From 53a2d9865b05d1557a7918e9719ecaac84adb020 Mon Sep 17 00:00:00 2001 From: jverzani Date: Wed, 3 Jun 2026 14:22:34 -0400 Subject: [PATCH 1/7] typos,start giac notes --- quarto/_quarto.yml | 1 + quarto/alternatives.qmd | 3 +- quarto/alternatives/giac.qmd | 2592 +++++++++++++++++++++++++++++ quarto/alternatives/make_pdf.jl | 1 + quarto/alternatives/symbolics.qmd | 2 +- quarto/basics/calculator.qmd | 10 +- 6 files changed, 2602 insertions(+), 7 deletions(-) create mode 100644 quarto/alternatives/giac.qmd diff --git a/quarto/_quarto.yml b/quarto/_quarto.yml index 504bb33..43b8ecb 100644 --- a/quarto/_quarto.yml +++ b/quarto/_quarto.yml @@ -117,6 +117,7 @@ book: - part: alternatives.qmd chapters: + - alternatives/giac.qmd - alternatives/symbolics.qmd - alternatives/SciML.qmd #- alternatives/interval_arithmetic.qmd diff --git a/quarto/alternatives.qmd b/quarto/alternatives.qmd index b917e5a..862daed 100644 --- a/quarto/alternatives.qmd +++ b/quarto/alternatives.qmd @@ -2,7 +2,8 @@ These notes use a particular selection of packages. This selection could have been different. For example: -* The symbolic math is provided by `SymPy`. [Symbolics](./alternatives/symbolics.html) (along with `SymbolicUtils` and `ModelingToolkit`) provides an alternative. +* The symbolic math is provided by `SymPy`. [Giac](./alternatives/giac.html) and +[Symbolics](./alternatives/symbolics.html) (along with `SymbolicUtils` and `ModelingToolkit`) provide alternatives. * The finding of zeros of scalar-valued, univariate functions is done with `Roots`. The [NonlinearSolve](./alternatives/SciML.html#nonlinearsolve) package provides an alternative for univariate and multi-variate functions. diff --git a/quarto/alternatives/giac.qmd b/quarto/alternatives/giac.qmd new file mode 100644 index 0000000..b2b3dcd --- /dev/null +++ b/quarto/alternatives/giac.qmd @@ -0,0 +1,2592 @@ +--- +title: "Symbolic math with Giac" +engine: julia +format: + html: + embed-resources: true + standalone: true +--- + +# Using the Giac symbolic math engine within Julia + +This page describes the use of symbolic math for some topics of Calculus within `Julia` utilizing the [Giac](http://www-fourier.ujf-grenoble.fr/~parisse/giac.html) library. Giac is accessed through `Giac.jl`. Giac is a computer algebra system (CAS) implemented in C++; `Giac.jl` is an interface. See this [pdf documentation](https://www-fourier.univ-grenoble-alpes.fr/~parisse/giac/cascmd_en.pdf) for a comprehensive set of topics or the `Julia` documention for the different commands discussed. + +There are other possible choices for symbolic math within the Julia ecosystem: + +* `SymPy.jl`: a package using either `PyCall` or `PythonCall` to connect to the sympy Python library. This is full featured computer algebra system, but does require the underlying python installation to work within `Julia`. +* `Symbolics.jl`: a numeric-oriented computer algebra system written in `Julia`, well suited for its intended use, but not a feature rich as either `Giac` or sympy. +* `SymEngine.jl`: a symbolic manipulation library designed for performance, written in C++, but is not intended to be full featured computer algebra systems, as any of the others mentioned. + +Whereas, Giac is a mature CAS and `Giac.jl` a natural interface from the perspective of `Julia` that does not require a glue package, as `SymPy` does. + +---- + +The `Giac` package is loaded as any other package is package is. We also add the `Plots` package to handle our plotting needs.^[Other plotting packages can be used] + + +```{julia} +using Giac +using Plots +``` + +There are a few basic ideas behind the package: + +* Symbolic variables and values can be easily created (`@giac_var`, `giac_eval`). +* Generic operations, such as math operations, have specialized methods for symbolic values and expressions that utilize the underlying Giac library to manipulate or create new symbolic expressions. +* Commands specific to the Giac library are accessed through the `Commands` submodule and are specialized on the first argument being of type `GiacExpr`. +* Most data exchange between `Julia` and the C++ library is done through strings. + + +## Symbolic variables + +The main way to create a symbolic object in `Giac` is the `@giac_var` macro + +```{julia} +@giac_var x +``` + + +One or more variables can be created. + +```{julia} +@giac_var x y +``` + +In this example, this command creates a range of values: + +```{julia} +@giac_several_vars z 3 +``` + + +Symbolic functions can be naturally specified + +```{julia} +@giac_var u(t) +u(x) +``` + +::: {.callout-note} +## GiacExpr + +`Giac` has a few basic types for its objects, the primary one being `GiacExpr`. This type may appear in some display methods and may be necessary for certain methods, such as `collect`. + +```{julia} +isa(x, GiacExpr), isa(sin(x^2), GiacExpr) +``` +::: + + +## Giac commands, Julia methods + + +Giac has over 2000 commands available. Many of these have counterparts within `Julia`. Commands in `Giac` which have base `Julia` counterparts or `LinearAlgebra` counterparts are extended to have methods defined for when the first argument is of type `GiacExpr`. For example, the trigonometric functions all have methods for `Giac` expressions: + +```{julia} +sin(x), cos(x), tan(x), asin(x), acos(x), atan(x) +``` + +The above method calls all return unevaluated symbolic expressions. + +Basic math operations do as well: + +```{julia} +x*y^(1/2)/(1 + x^2 + y^2) +``` + +Those commands which do not have a counterpart in base `Julia` or the `LinearAlgebra` module, do have new methods defined that narrow the type of the first argument to `GiacExpr`. These methods are generated in the `Giac.Commands` module and are available within that module. For example, the `simplify` command is defined along the lines of:^[With a notable absence not illustrated---the Giac documentation string is given to the automatically generated methods.] + +```{julia} +#| eval: false +simplify(first_arg::GiacExpr, rest...) = Giac.invoke_cmd(:simplify, first_arg, rest...) +``` + +(There are no keyword arguments passed.) + +The `Giac.Commands` module can be brought into scope via `using Giac.Commands` and all the commands would then be available, as they are exported when generated. + +However, this approach can cause many conflicts with methods defined in other packages---such as `Plots`, as `Giac` has underlying plotting functionality. + +We selectively import commands as needed. The basic syntax for importing a method looks like the following and will be used in the sequel: + +```{julia} +using Giac.Commands: factor, simplify, expand +``` + + +## Symbolic numbers + +If `x` is symbolic, the expression `2x` will also be symbolic + +```{julia} +2x +``` + +Behind the scenes, before multiplying, `2` and `x` the `2` is converted to the `GiacExpr` type^[Via `convert(GiacExpr, 2)`] and the two arguments are passed to an underlying C++ call^[`GiacCxxBindings.giac_eval(...)`] after a conversion to a string. So most regular `Julia` expressions involving one or more symbolic values will simply yield symbolic answers without modification. Integers and rational numbers promote to exact counterparts; floating point numbers convert to floating point values. + +Symbolic numbers can be directly produced *from strings* using `giac_eval`, for example: + +```{julia} +giac_eval("42") +``` + +There are some constants that can be produced this way. The following are in `Giac.Constants`: + +```{julia} +giac_eval("e"), giac_eval("pi"), giac_eval("i") +``` + +Values for infinity can be constructed from their names: + +```{julia} +giac_eval("infinity"), giac_eval("-infinity"), giac_eval("+infinity") +``` + +A more common `Julia` idiom would be to convert the number to the `GiacExpr` type, and this too can be done: + +```{julia} +convert(GiacExpr, 42), convert(GiacExpr, pi), convert(GiacExpr, Inf) +``` + +---- + +Care must be taken with some expressions, as the conversion to `GiacExpr` and back to `Float64` may not agree, given the digits used by `Giac` and the conversion to string. For example, this expression below fails with many inputs: + +```{julia} +v = 1.000000000222 +convert(Float64, convert(GiacExpr, v)) == v +``` + +---- + +Care must be taken to ensure promotion occurs before expressions are evaluated with `Julia`: + +```{julia} +x/2 == x * (1/2), x/2 == x * (1//2) +``` + +In the calculation `x * (1/2)` the value of `1/2` is evaluated returning a floating point value `0.5` and `x * 0.5` is not equal to `x/2` in `Giac`. + + +Similarly, ensuring the promotion occurs beforehand can be used: + +```{julia} +2*pi*x, 2*(pi*x) +``` + +In the following, we define `PI` to be the result of converting `pi` to a `Giac` expression so that such considerations need not be made. + +::: {.callout-note} +## giac_eval + +`Giac.jl` is a `Julia` interface to an underlying `C++` library. The primary protocol to transfer data from a `Julia` session to this library is to pass values after converting them to strings. The `giac_eval` command parses a string and produces a Giac expression in the `C++` library, which is then referenced with the `Julia` session. + +The `giac_eval` command can be used to quickly create symbolic expressions, and symbolic values. It is especially useful to avoid early floating point conversions: + +```{julia} +giac_eval("2 * pi * x"), giac_eval("sqrt(2)") +``` + +::: + +## Giac expressions + +Symbolic expressions can be created by applying functions to symbolic variables. For example + +```{julia} +@giac_var F +C = 9//5 * (F - 32) +``` + +A complicated expression is abstractly a tree with an enclosing operation and various arguments. Such expressions can be systematically iterated over until the arguments are either just symbolic variables or symbolic numbers. + +```{julia} +using Giac: arguments, operation # names from TermInterface +op, args = (operation(C), arguments(C)) +operation(last(args)), arguments(last(args)) +``` + +## Substitution + +Expressions are like function bodies---and not like regular Julia expressions---in that the symbols are evaluated only on request. For functions this is when the function is called; for expressions this is when values are substituted. + +Substituting one value in for another, is done in a few ways, for example there is `substitute`, called with a dictionary of substitutions. These substitutions can also be specified as pairs, as in `substitute(ex, old => new)`. + +The `substitute` function (or object method) replaces an old value for a new value after conversion to a symbolic object^[The `giac_eval` function takes a string and makes it symbolic, which can be used to convert numbers to symbolic numbers or strings into variables.] the arguments. + +```{julia} +@giac_var a b c x y +ex = a * x^2 + b * x + c +substitute(ex, x => y-1) +``` + +```{julia} +substitute(ex, a => 2) +``` + +```{julia} +substitute(ex, a=>3, b=>2, c=>1) +``` + +The latter is akin to function evaluation. As such, the call notation for symbolic expressions is defined to use `substitute` when called with paired data: + +```{julia} +ex(a=>3, b=>2, c=>1) +``` + +(There is a different method followed with non-paired data which can be used to create unevaluated function objects.) + +::: {.callout-note} +## Inconsistency with the variable `x` + +In this example, we show that `x` variables in substitution in Giac have a special priority, which can be hard to describe and hence best to be avoided. + +First, let's discuss that these two commands are not necessarily the same: + +```{julia} +ex(a=>3, b=>2, c=>1), ex(a=>3)(b=>2)(c=>1) +``` + +The reason being is the order *need not* be preserved in the first case, but in the second proceeds from left to right. + +Now we show that the variable name can make a difference when substitution is performed. + +```{julia} +@giac_var t x(t) y(t) u(t) v(t) +ex = sin(x(t))/cos(y(t)) +ex1 = sin(u(t))/cos(v(t)) +``` + +We compare the output of *identical operations*: + +```{julia} +ex(t=>2)(y(2)=>3)(x(2)=>4) +``` + +and + +```{julia} +ex1(t=>2)(v(2)=>3)(u(2)=>4) +``` + +The latter returns the expected result; the former has an unexpected warning. + +Next we compare the output of another *identical operation* + +```{julia} +ex(t=>2)(y(t)=>3)(x(t)=>4) +``` + +```{julia} +ex1(t=>2)(v(t)=>3)(u(t)=>4) +``` + +The latter again is what is expected. Once `ex1(t=>1)` is evaluated, there are no `v(t)` or `u(t)` variables to substitute. Not so with the initial case. In fact, we have this oddity + +```{julia} +ex(t=>2), ex(t=>2)(y(t) => 3) +``` + +The first substitution acting exactly as expected, the second replacing `x(t)` with `2`, which makes no sense. + +A similar oddity occurs just with `giac_eval`: + +```{julia} +giac_eval("x(2)"), giac_eval("y(2)") +``` + +::: + +### Domain of a univariate expression + +The domain of a univariate function is the set of $x$ for which the function is defined. Many functions have domains that are the entire real line, but not all functions do. The `domain` command returns the domain of an algebraic expression: + +```{julia} +using Giac.Commands: domain +@giac_var x +u = domain(log1p(x)) +``` + +We can check if a value is in the domain, through substitution. For example, this shows `-1` is not: + +```{julia} +u(x => -1) +``` + +The `domain` function works a little bit to try and sort this out. + +```{julia} +u = domain(log1p(sin(x))) +``` + +The result is more complicated than the previous. We can substitute in a value, `1` below, and then call any with a predicate. + +```{julia} +TRUE, FALSE = giac_eval("true"), giac_eval("false") +any(==(TRUE), u(x=>1)) +``` + +So `1` is in the interval, but not $3\pi/2$, which makes the interior `sin` function `-1` and hence out of the domain of the enclosing `log1p`: + +```{julia} +PI = giac_eval("pi") +any(==(TRUE), u(x => 3*PI/2)) +``` + +(This pattern with `any`, will work for all of the examples shown.) + +## Numerical evaluation + +An exact symbolic value may be the result of a computation, but it may be of interest to convert that value to a number (e.g. `1.414...` instead of $\sqrt{2}$). + +Giac has an `evalf` command that finds a floating-point value. + +```{julia} +t = giac_eval("2") +u = exp(t) - exp(-t) +u +``` + +The value of `u` is exact. In this next cell, we call the `evalf` command---after importing---and compare to a computation within `Julia`: + +```{julia} +using Giac.Commands: evalf +evalf(u), exp(2) - exp(-2) +``` + +The default for `evalf()` is 12 digits after the decimal point; similar but not the same as for Julia. The method allows a specification of the number of digits in its second argument. + +```{julia} +evalf(u, 50) +``` + + +::: {.callout-note} +## DIGITS + +The number of significant digits in `Giac` is stored in a configuration variable `DIGITS`. The value of 12 is the default, it can be changed with `giac_eval`, for example `giac_eval("DIGITS := 16")`. +::: + +### Conversion to a Julia numeric type + +The result of `evalf` looks numeric, but internally the values are still of the `GiacExpr` type. In `Giac.jl` the `to_julia` function converts the exact value to a number type in `Julia`, as does `convert`, `unwrap_const`, and the generic method `float`. + + +```{julia} +u = giac_eval("1 / 1000") +evalf(u), to_julia(u), convert(Float64, u), Giac.unwrap_const(u)#, float(u) +``` + +The differences are slight, and mostly due to other conventions: + +* `to_julia` is defined in `Giac.jl`. If the underlying Giac storage type is a number type, then a value is defined by conversion.^[The underlying number types known to `to_julia` are `INT`, `DOUBLE`, `REAL`, `ZINT`, `CPLX`, `FRAC`, `VECT`, and `STRING`.] For example a big integer is returned by `parse(BigInt, string(x))`. Otherwise, the `evalf` command is called and that value is converted. +* `unwrap_constant` is just `to_julia` with semantics that it will return non-constant values, to match its use in `Symbolics` +* `Base.convert(T,x)` uses the same machinery as `to_julia` with an additional call of `T`. +* `float` also uses a similar approach, but following its generic usage, will convert `VECT` types as well (lists of numbers). + + + +## Programming + +Both `Giac` and `Julia` implement the building blocks of computer programming such as typed numbers, strings, Boolean values, lists, arrays, conditional statements, flow control, etc. A key design of `Julia` is that there are generic methods to deal in a standard manner with different underlying data types. As much as possible, this set of notes defers to `Julia` the programming concepts necessary to prepare data or manipulate outputs. + +For example, iterable objects could be iterated within `Giac`, but as `GiacExpr` objects have an `iterate` interface, iteration can happen within `Julia` functions. + +This example gives a small taste. Here we define a list of numbers and show that it can easily be passed to `sum`, which supports summation of iterable objects, ultimately because `mapreduce` does: + +```{julia} +lst = Giac.Commands.list(1:8) +sum(lst) +``` + +The `map` function and list comprehensions work: + +```{julia} +map(sqrt, lst) +``` + +and + +```{julia} +[x^2 for x in lst if iseven(to_julia(x))] +``` + +### Broadcasting + +Broadcasting over symbolic containers should work as expected. + +For example: + +```{julia} +lst = Giac.Commands.list([1,2,3]) +lst .^ 2 +``` + +Or + +```{julia} +(lst .* lst')/2 +``` + + +## Plotting expressions + +`Giac` expressions do not plot directly within the `Plots` framework. To work with expressions as functions, we provide this `Plots` recipe: + +```{julia} +Plots.RecipesBase.@recipe f(::Type{T}, v::T) where {T<:GiacExpr} = lambdify(v) +``` + +The `lambdify` function creates a `Julia` function from a symbolic expressions. The work is done by +the `build_function` method whose basic pattern is `build_function(expr, vars)`. The default approach is convenient, but not performant. For more performant usage, the `Symbolics` package extension provides an alternate. + + +Below we write a `lambdify` function that identifies the free variable, if present, to create a univariate function from an expression. The `free_symbols` call extracts all the variables in the expression. If there are none, the expression is constant, if there is more than one, the order is dependent on `free_symbols` and might be a surprise.^[As such, we avoid this function when we plot multivariate functions.] + +```{julia} +function lambdify(v::GiacExpr) + vars = Giac.free_symbols(v) + isempty(vars) && return (val = Giac.unwrap_const(v); x -> val) + build_function(v, vars...) +end +``` + + +(The `lambdify` function isn't meant to be a general solution, just a convenience for this specific act of plotting an expression.). + +With this set up, we see that passing a Giac expression of a single variable to `plot` works as though a function were passed: + + +```{julia} +@giac_var x +plot(x^5 - x - 1, -1, 5/4; legend=false) +plot!(zero(x)) +``` + +## Polynomials + +Polynomials provide a very structured, yet flexible, family of functions. Their flexibility leads to widespread usage, the structure allows key properties to be defined and analyzed. Rational functions, or ratios of polynomials, add even more flexibility. + +Computer algebra systems use *algebraic extensions* of fields of numbers to perform many of the advanced tasks. These extensions are constructed as polynomials with more general fields of coefficients. As such, `Giac` has several methods for polynomials and rational expressions. + +### Construction + +Here we begin with how to create a polynomial object from its coefficients. + +A sparse, dense, univariate polynomial may be written as + +$$ +p = a_n x^n + a_{n-1}x^{n-1} + \cdots a_2 x^2 + a_1 x + a_0, \quad a_n \neq 0 +$$ + + +The $a_n$ above are the coefficients of $p$. These can be readily identified with the entries of a vector. Giac uses *decreasing* order (starting with $a_n$) and not *ascending* order (starting with $a_0$---more of a Julian convention). The `poly2symb` method is used. + +To construct the polynomial $x^5 - 2x - 3$ the coefficients are specified with $1,0,0,0,-2,-3$ and we can pass these in a vector to the constructor along with a variable: + +```{julia} +using Giac.Commands: poly2symb +p = poly2symb([1,0,0,0,-2,-3], x) +p, expand(p) +``` + + +Using base Julia commands, the same polynomial might be constructed directly, as follows, with $5$ being the degree for the polynomial and $5$ and $i-1$ for the traditional $0$-based indexing of a polynomial's powers. + +```{julia} +sum(cᵢ * x^(5 - (i - 1)) for (i, cᵢ) ∈ enumerate([1,0,0,0,-2,-3])) +``` + + +Similarly, the base `evalpoly` function can + +```{julia} +evalpoly(x, reverse([1,0,0,0,-2,-3])) +``` + +Both `poly2symb` and `evalpoly` use the efficient [Horner's](https://en.wikipedia.org/wiki/Horner%27s_method) method to evaluate a polynomial. + +---- + +For multivariate polynomials, a sparse format is used. In `Julia` it is natural to describe a monomial, $a x_1^{n_1}\cdot x_2^{n_2} \cdots x_m^{n_m}$ through the powers $\{n_1, n_2, \dots, n_m\}$ and the coefficient $a$. As---after combining---the powers are unique, a dictionary with keys given by the powers and coefficient given by the value is a natural data structure. + +In `Giac` a monomial is represented by a special container consisting of the coefficient and a vector of the powers (again, given some ordering of the variables). The syntax (paired delimiters `%%%{` and `%%%}`) has no natural counterpart, so this mapping using strings is useful: + + +```{julia} +function sparse_polynomial(d, vars) + coeffs = join(["%%%{$v,$k%%%}" for (k,v) in d], "+") + Giac.Commands.poly2symb(coeffs, vars) +end +``` + +To construct $3x^2\cdot y + x\cdot y^2 + y^3$ we have: + +```{julia} +@giac_var x y +d = Dict([2,1]=>3, [1,2] => 1, [0,3] => 1) +s = sparse_polynomial(d, [x,y]) +``` + +The Giac command `symb2poly` is the reverse, returning a "list" from a polynomial. The "list" in Julia for sparse polynomials can be easily converted to a readable format with the Giac `convert` method, which, again, does not follow Julia's generic calling pattern. + +```{julia} +sc = Giac.Commands.symb2poly(s, [x,y]) +``` + +```{julia} +l = convert(sc) +``` + +More directly, the `coeff` method can be used: + +```{julia} +using Giac.Commands: coeff +coeff(p, x) +``` + +We specified the variable `x` and we see the coefficients are returned starting with the highest order. For the multi-variate polynomial, `s`, the specification of coefficients is important: + +```{julia} +coeff(s, x), coeff(s, y) +``` + +The `coeff(s,y)` output mirrors $s = (1)\cdot y^3 + (x) \cdot y^2 + (3x^2) \cdot y + (0)\cdot y^0$ treating a multivariate polynomial as a univariate polynomial with a wider set of coefficients (polynomials in the other variables). + +The coefficients of the sparse representation used in construction of the polynomial can be returned for each monomial ($x^ny^m$) by specifying the unique powers with a vector: + +```{julia} +coeff(s, [x,y], [2,1]) +``` + +To avoid specifying the powers, the `op` command will break a polynomial into monomials, then `coeff` can be applied: + +```{julia} +[coeff(mᵢ, [x,y]) for mᵢ ∈ Giac.Commands.op(s)] +``` + +---- + +The degree of a univariate polynomial is returned by `degree`. For a multivariate polynomial it returns the degrees when treated as polynomials over a single variable: + +```{julia} +using Giac.Commands: degree +degree(p), degree(s, [x,y]) +``` + +Combining the above, we might get the original structure from the sparse polynomial with + +```{julia} +vars = [x,y] +Dict(degree(m, vars) => only(coeff(m,vars)) for m in Giac.Commands.op(s)) +``` + + +---- + +The leading coefficient of a univariate polynomial is the coefficient of the monomial of highest degree. This +can be extracted by `lcoeff`. The `tcoeff` finds the trailing coefficient, defined by the coefficient of the monomial of lowest degree. + +### Polynomial evaluation + +To evaluate a polynomial, substitution can be used: + +```{julia} +s(x=>3, y=>5) +``` + +For univariate polynomials, the `horner` method will use the efficient Horner algorithm for evaluation: + +```{julia} +using Giac.Commands: horner +horner(p, 2) +``` + +To use `evalpoly` from base `Julia` requires more effort: + +```{julia} +evalpoly(2, reverse(collect(GiacExpr, coeff(p)))) +``` + +### Polynomial shifts + +Univariate polynomials are represented in the standard basis $1, x, x^2, \dots$. This basis can be shifted around a point, say $a$, using the basis $1, (x-a), (x-a)^2, \dots$. The `ptayl` function (finding the polynomial Taylor series) does this computation. + +```{julia} +using Giac.Commands: ptayl +a = 2 +cs = ptayl(coeff(p), a) +``` + +We can see this polynomial with a bit of work + +```{julia} +pa = sum(cᵢ * (x-a)^((1 + degree(p)) - i) for (i, cᵢ) ∈ enumerate(cs)) +``` + +The two polynomials are indeed the same: + +```{julia} +p == pa +``` + +### simplify + +The fact that `p` and `pa` are equal can be checked as above, but could also have been done by *simplifying* both expressions. There are a number of related commands for simplification: + +```{julia} +using Giac.Commands: simplify, expand, collect, factor, factors, sqrfree +simplify(p), simplify(pa) +``` + +The `simplify` method simplifies an expression, in this case a polynomial. There are other means to do this, we mention `factor` and Giac's `collect`; there are others like `normal`. + + +### collect + +The `collect` generic of `Julia` is used to take an iterator and realize it as an Vector/Matrix/Array. In `Giac` we have seen that `collect(GiacExpr, expr)` will collect an underlying `GiacExpr` which is iterable into a vector. + + +However, the `collect` method for `GiacExpr` has different semantics. In particular, `collect` collects like terms in a polynomial expression. The method returns something very similar to `factor`, though for polynomials of degree $2$ factor will factor over the real numbers: + +```{julia} +collect(x^4 - 1), collect(x^4 - 4) +``` + +### factor + +Factoring a polynomial is the task of `factor`, though `collect` also does this task. In an algebra class, factoring of a polynomial over the integers over rational terms is learned. But, there are other techniques available. The factor command works more or less as expected: + +```{julia} +factor(x^4 - 1) +``` + +This could have been done by hand, as the terms are integers (hence rational). However, `factor` can also do more, in this case recognize a perfect square and then factor `x^2 - 2`: + +```{julia} +factor(x^4 - 4) +``` + +#### factors + +The `factors` command returns a list of each identified factor *along* with its multiplicity. This has no new information over `factor`, but allows subsequent manipulation of the individual factors: + +```{julia} +factor(x^6-14*x^5+80*x^4-238*x^3+387*x^2-324*x+108) +``` + +```{julia} +u = factors(x^6-14*x^5+80*x^4-238*x^3+387*x^2-324*x+108) +``` + +```{julia} +(irreducible_factors = u[1:2:end], multiplicities=u[2:2:end]) +``` + + +### Square-free factorization + +The square-free factorization of a polynomial is needed for some algorithms. In the above, the polynomial formed by just multiplying the factors together without their multiplicities gives a polynomial with the exact same set of roots as $p$, though without the multiplicities. What characterizes a square-free factor is the factor and its derivative have no common root. + +The `sqrfree` function returns a factorization where each factor has that property. As seen is it may not factor as far as `factor`, which returns irreducible factors: + +```{julia} +using Giac.Commands: sqrfree +u = (x^2-1)^2*(x-1)*(x+2)^2 +sqrfree(u) +``` + +Whereas, `factor` returns + +```{julia} +factor(u) +``` + +### Roots of univariate polynomials + +Suppose $p = (x-r)\cdot q$ where $p$ and $q$ are polynomials in the symbol $x$. The term $(x-r)$ is called a factor of $p$, the value $r$ is called a root of $r$. When $r$ is substituted in for $x$, then $p$ is zero. The factor theorem states a bit more: $r$ is a root if and only if $(x-r)$ is a factor. + +By the fundamental theorem of algebra, the roots of a univariate polynomial of degree $n$ with real coefficients must be $n$ in number and either real or in complex conjugate pairs. + +Finding the roots of such polynomials if of interest and Giac provides different methods. The `roots` function returns a list of roots and their multiplicities: + +```{julia} +using Giac.Commands: roots +roots(u) +``` + +The above can be read off from the output of `factor`. + +The values alone can be accessed as follows: + +```{julia} +rts = map(first, roots(u)) +mults = map(last, roots(u)) +(; rts, mults) +``` + +The variable can be passed as a second argument allowing parameters to be used. This example shows the quadratic equation in action: + +```{julia} +@giac_var a b c d +roots(a*x^2 + b*x + c, x) +``` + +The roots of a cubic polynomial, known in part for [centuries](https://en.wikipedia.org/wiki/Cubic_equation#History), are not identified in this example: + +```{julia} +roots(a*x^3 + b*x^2 + c*x + d, x) +``` + + + +#### Real roots + +There is one real root of the polynomial $x^5 - x - 1$ and four complex ones: + +```{julia} +collect(GiacExpr, roots(x^5 - x - 1)) +``` + +There are methods to isolate the one real root, which is irrational in this case, so doesn't have an algebraic formulation such as results with the quadratic formula. + +Giac uses the [Vincent-Akritas-Strzebonski](https://en.wikipedia.org/wiki/Vincent%27s_theorem) algorithm to isolate the real roots of a square-free polynomial through a series of transformations and a simple test on whether a polynomial has a single sign variation in its coefficients. + +A necessary ingredient is a bound on the size of any real roots. A few algorithms are implemented in Giac, here we see "a (non-optimal) upper bound" + +```{julia} +Giac.Commands.posubLMQ(x^5 - x - 1) +``` + +This says any positive real root is no more than $2$ in value. In this case, the lone real root is around $1.167\cdots$. + +The roots can *isolated* in intervals, after which a numeric algorithm like the bisection method can be used to numerically approximate the root. The *maximum* size of the isolating interval is specified: + +```{julia} +using Giac.Commands: realroot +realroot(x^5 - x - 1, 1/10) +``` + +This polynomial has nearby real roots: + +```{julia} +p = expand(poly2symb([1, 0, 0, 0, 0, -16129, 254, -1], x)) +u = realroot(p, 1/10) +``` + +The real roots can be filtered out, though cumbersomely, by: + +```{julia} +rs = map(first, roots(p)) +rs′ = filter(isreal∘to_julia, rs) +``` + +We can see that the first two differ in the seventh decimal point. The value `1/127` separates the two: + +```{julia} +r₁, r₂ = [first(u) for u in rs′[1:2]] +r₁ < 1//127, 1//127 < r₂ +``` + +Passing a smaller value for the interval, will identify smaller isolating intervals: + +```{julia} +ϵ = 1e-10 +u = realroot(p, ϵ) +``` + +Which we can see has narrow intervals, for example, the first identified one is: + +```{julia} +-(first(first(u))...) < ϵ +``` + + +## Rational functions + +Polynomial long division is an algorithm mirroring the familiar long division algorithm. If $p$ and $q$ are polynomials, then the algorithm produces polynomials $d$ and $r$ with $p = q \cdot d + r$ where the degree of $r$ is less than the degree of $q$. + +The `Julia` generic is `divrem`. In Giac, the commands are `quo`, `rem`, and `quorem`: + +```{julia} +using Giac.Commands: quo, quorem # rem is overloaded +p = (x-1) * (x-2) * (x-3)^3 +q = (x-1) * (x-2) +d, r = quorem(p, q) +``` + +Confirming the two key assumptions: + + +```{julia} +simplify(d *q + r - p), degree(r) < degree(q) +``` + +Or with larger polynomials with random coefficients: + +```{julia} +ps, qs, = rand(Int, 25), rand(Int, 15) +p, q = evalpoly(x, ps), evalpoly(x, qs) +d, r = quorem(p, q) +simplify(d *q + r - p), degree(r) < degree(q) +``` + +Polynomial long division means $p/q = d + r/q$. This form can be directly returned by the `propfrac` command applied to a rational function. + +```{julia} +q = x^2 + 3x + 4 +p = 2q + (x-3) +Giac.Commands.propfrac(p/q) +``` + +Alternatively, given a rational function `p/q`, this could be done with: + +```{julia} +using Giac.Commands: numer, denom +u = p / q +p′, q′ = numer(u), denom(u) +d, r = quorem(p, q) +d + r / q′ +``` + + + + +The `numer` and `denom` commands find the numerator and denominator **after** the rational function has been reduced by simplifying. For the above use, this detail isn't important. However, to retrieve the terms *before* a reduction, the `getNum` and `getDenom` commands are available. + +```{julia} +p = (x-1)*(x^2 + x + 1) +q = (x-1)*(x^3 - x - 1) +numer(p/q), Giac.Commands.getNum(p/q) +``` + +and + +```{julia} +denom(p/q), Giac.Commands.getDenom(p/q) +``` + +When the remainder is $0$, for rational expressions, the `simplify` function re-expresses in a canonical, *reduced* form. + +```{julia} +p/q, simplify(p/q) +``` + + +Related to polynomial division are the notions of a greatest common divisor (`gcd`) or least common multiple (`lcm`). These are generic methods in base `Julia` and given methods for Giac expressions: + +```{julia} +gcd(p, q) +``` + +and + +```{julia} +lcm(p, q) # just one (x-1) term +``` + +### Partial fractions + +A partial fraction decomposition of a rational function expresses a rational function as polynomial plus a sum of other rational functions of a certain type. + +A formal statement says the for polynomials $f(x)$ and $g(x) = p_1(x)^{n_1} \cdot p_2(x) \cdots p_k(x)^{n_k}$, each $p_i$ being *irreducible*. Then there are unique polynomials $b(x)$ and $a_{ij}(x)$ with + +$$ +\frac{f(x)}{g(x)} = b(x) + \sum_{i=1}^k \sum_{j=1}^{n_i} \frac{a_{ij}(x)}{p_i(x)^j} +$$ + +where for each $j$, $a_{ij}(x)$ has degree less than the degree of $p_i(x)$. + + +```{julia} +using Giac.Commands: partfrac +p,q = 3x + 5, (1 - 2x)^2 * (x + 2) +u = partfrac(p/q) +collect(GiacExpr, Giac.Commands.op(u)) +``` + + +For polynomials with real coefficients, a partial fractions decomposition has then $p_i(x)$ being linear or quadratic and $a_{ij}(x)$ being linear or constant. + +## Algebraic manipulation + + +`Giac` provides other tools to algebraically manipulate expressions, even non-polynomial ones. Common algebraic manipulations include + +* `simplify`: simplify expressions. +* `expand`: expand algebraic expressions . +* `combine`: joins subexpressions of various types. + +### Trigonometry + +There are trigonometric variants: `trigsimplify`, `texpand` + +As well: + +* `tcollect`: linearizes trigonometric expressions in terms of sin(nx) and +cos(nx) and combines sines and cosines of the same angle. +* `tlin`: linearizes products and integer powers of the trigonometric functions +* `trigsin`: trigonometric functions rewritten in terms of sines and cosines, with as many cosines as possible transformed to sines. Also `acos2asin` and `atan2asin`. +* `trigcos`: trigonometric functions rewritten in terms of sines and cosines, with as many sines as possible transformed to cosines. Also `asin2acos` and `atan2acos`. +* `trigtan`: rewrites trigonometric expressions into expressions where as many trigonometric functions as possible are written in terms of tangents. Also `acos2atan` and `asin2atan`. +* `tan2sincos`: writes `tan` as `sin/cos`, similarly `sin2costan` and `cos2sintan`. +* `halftan` rewrites `sin`, `cos`, and `tan` terms using `tan(x/2)`. + +### Logs, exponentials, and powers + +* `expexpand`: expands $e^{x+y}$ as a product, +* `powexpand`: expands $a^{x+y}$ as a product +* `lnexpand`: expands $\log(xy)$ as a sum. +* `texpand`: combines `expexpand`, `lnexpand`, and `trigexpand` + +Also + +* `pow2exp`: changes base of $a^x$ to $e$. +* `exp2pow`: changes $e^{n\log(x)}$ to $x^n$ +* `lncollect`: rewrites sums of logs as a product + +In addition: + +* `lin`: linearize expressions involving exponentials; namely, it will replace products of exponentials by exponentials of sums; replace any hyperbolic functions by exponentials. +* `exp2trig` rewrites $e^{ix}$ using sines and cosines; `trig2exp` reverses. +* `atrig2ln` writes inverse trigonometric functions in terms of logarithms +* `hyp2exp` rewrites hyperbolic functions in terms of exponentials + + +### Examples + +```{julia} +using Giac.Commands: simplify, expand, factor +``` + +Symbolic expressions are not routinely simplified in `Giac`. + +In this example, the `Gamma` function^[This is `gamma` in `SpecialFunctions.jl`] is $\Gamma(n) = (n-1)!$ for integer $n > 0$. We see by simplifying, the cancellation occurs: + +```{julia} +using Giac.Commands: Gamma +@giac_var n +Giac.Commands.assume(n, "integer") # assume n is integer +u = Gamma(n) / Gamma(n-1) +u, simplify(u) +``` + + +The `expand` function takes an expression and basically multiplies it out. If new *default* cancellations occur, expand can actually result in shorter expressions: + +```{julia} +expand((x-1)^3) +``` + +```{julia} +ex = (x + 1) * (x - 2) - (x - 1) * x +ex, expand(ex) +``` + +The `factor` function applied to polynomials is the opposite of expand. In Giac, factorization of polynomials isn't limited to rational terms: + +```{julia} +factor(x^2 - 4), factor(x^2 - 3) +``` + + +## Solving equations + +`Giac` can be used to solve many algebraically solvable equations and many numerically solvable equations. + +The `solve` function is used. + +Equations are specified with a left- and right-hand side using a tilde, `~`. Later we illustrate the commands `left` and `right` to access the two sides of an equation. + + +Here the equation $x^2 = 2$ is solved for: + + +```{julia} +using Giac.Commands: solve +@giac_var x +solve(x^2 ~ 2, x) +``` + +Equations can have symbolic variables included. In which case, passing in the second argument the variable(s) to solve for is done: + +```{julia} +@giac_var a b c +out = solve(a*x^2 + b*x + c ~ 0, x) +collect(GiacExpr, out) +``` + +The use of `collect`, as used above, iterates over the `Giac` container to produce a vector (in this case). + +Systems of equations are specified using vectors, as are the variables: + +```{julia} +@giac_var a b c d x y +system = [a*x + b*y ~ 1, c*x + d*y ~ 1] +out = solve(system, [x,y]) +``` + +The return value is a `list` holding `[x,y]` pairs. As above, these can be "collected" into a `GiacExpr` values as follows^[For a list with a single object, the generic `only` method can return the single object as a `GiacExpr`, as the underlying list is iterable.]: + +```{julia} +collect(GiacExpr, out) +``` + +This example shows a possible pattern to collect multiple solutions, each into a dictionary keyed by the variables solved for: + +```{julia} +@giac_var x y +vars = [x, y] +eqns = [x^2 + y^2 ~ 1, y ~ x^2] +out = solve(eqns, vars) +[Dict(zip(vars, u)) for u in out] +``` + + +### Assumptions + +Assumptions on variables can be made through `assume`. +These assumptions are locked and can't be modified through `assume`, rather `additionally` is used to add additional assumptions; `purge` is used to remove all assumptions. The current assumptions on a variable can be queried with `about`. + +```{julia} +using Giac.Commands: assume, additionally, purge, about +@giac_var r +purge(r) # clean out any assumptions +assume(r >= -1) +about(r) +``` + +The `line[-1,+infinity]` indicates the domain for `r`. We can also put an upper limit + +```{julia} +additionally(r < 100) +about(r) +``` + + +We see the lower bound impacts what is solved for in these two examples: + +```{julia} +solve(r^2 ~ 1, r) +``` + +The above finds the negative solution, `-1`, but not in the following, as it is $-2$ is not greater or equal to $-1$: + +```{julia} +solve(r^2 ~ 4, r) +``` + +Compound assumptions can be specified using strings, such as: `"(a>=2 and a<4) or a>6"`. + +```{julia} +@giac_var a +purge(a) +assume("(a>=2 and a<4) or a>6") +a < -1 +``` + +---- + +The output can be narrowed to a certain *domain*. Compare the two solution sets to the same equation once `r` is assumed to be an integer: + +```{julia} +solve(r^2 ~ 1//2, r) +``` + +```{julia} +additionally(r, "integer") +solve(r^2 ~ 1//2, r) +``` + +The possible domains are `"real"`, `"complex"`, `"integer"`, and `"rational"`. + +### Numerically solving an equation + +Not all equations can be solved symbolically. For example, finding the solution(s) to $\cos(x) = x$ will result in a numeric approximation: + +```{julia} +solve(cos(x) ~ x) +``` + +The `solve` call resolves to `fsolve`, as can be read in the provided output, `fsolve` attempts to find all solutions over the real line, as the mapping $x = \tan(t)$ over $(-\pi/2, \pi/2)$ scans over the original equation over $(-\infty, \infty)$. In this case, there is just one solution. However, with a flatter slope of the right hand side, we will see more: + +```{julia} +eqn = cos(x) ~ x/3 +out = solve(eqn) +``` + +Numeric solutions are typically "approximate" zeros. In this case, we can see the residuals through: + +```{julia} +import Giac.Commands: left, right +u = left(eqn) - right(eqn) # also just equal2diff(eqn) +[u(x => a) for a in out] +``` + +::: {.callout-note} +## Roots +The `Roots` package is used in the sequel, when `fsolve` doesn't work as desired. The `fsolve` function works similarly to `Roots.find_zeros` but can miss some zeros. +::: + + +## Calculus-specific functions + +Giac provides functions for specific topics of Calculus I and II. Here we import some commands for illustration purposes below: + +```{julia} +using Giac.Commands: limit, diff, integrate +using Giac: D +``` + +### Limits + +Limits may be taken symbolically using `limit`. The standard notation for a limit involves 3 things---an expression, a value of `c` and a variable name, a direction, when given, adds a fourth: + +$$ +\lim_{x \rightarrow c} f(x) = L +$$ + +If `ex` is `f(x)` in the above, then the call to `limit` looks like `lim(ex, x, c)`. Right limits have a fourth positional argument of `1`, left limits of `-1`: + +```{julia} +limit(sin(x)/x, x, 0), limit(sign(x), x, 0, 1), limit(sign(x), x, 0, -1) +``` + +The variable and `c` (the $x \rightarrow c$ part) can be paired off with equation notation: + +```{julia} +limit((1 + x)^(1/x), x ~ 0) +``` + + +#### Errors + +A limit exists if the left- and right-hand limits exist and are equal. The above showed the left and right and limits at `0` of `sign(x)` are different. The shows the resulting error message if a direction is not specified: + +```{julia} +#| eval: false +limit(sign(x), x ~ 0) +``` + +``` +ERROR: Unidirectional limits are distinct -1,1 Error: Bad Argument Value +``` + +#### Infinite limits + +Infinite limits can use `Inf` or `Inf` as the value for `c`: + +```{julia} +@giac_var n +assume(n > 0) +limit(x^n * exp(-x), x ~ Inf) +``` + +Not all limit problems have an answer. This shows how the oscillating `sin` function is treated at infinity + +```{julia} +limit(sin(x), x ~ Inf) +``` + + +::: {.callout-note} +##### Conversion of arguments + +Function calls for Giac methods are processed by `invoke_cmd(:fn_symbol, args...)`. +The argument values passed to a Giac function call are processed by conversion to a string by `_arg_to_giac_string`. Passing a floating point value, like `Inf`, is not an issue as it converts to an underlying infinity in `Giac`, as shown here after evaluation of the string: + +```{julia} +a,b,c = giac_eval(Giac._arg_to_giac_string(Inf)), giac_eval("inf"), giac_eval("Inf") +``` + +We see `a` and `b` are equivalent, but `c` is not. The value assigned to `c` is **not** infinite, rather just a name. The `is_constant` method shows if a value has a free symbol or not: + +```{julia} +Giac.is_constant(b), Giac.is_constant(c) +``` + +(The command `convert(GiacExpr, Inf)` takes essentially the same code path, but this special case is handled by an `isinf` test.) + +::: + +#### Limits with parameters + +The following shows that limits can be computed with parameters. + + + +```{julia} +@giac_var a b c x +purge(a); purge(b) +assume(a > 0); assume(b > 0) +ex = ((a^x - x*log(a)) / (b^x - x*log(b)))^(1/x^2) +L = limit(ex, x ~ 0, 1) +``` + +For a given set of values of `a` and `b`, we can see different answers: + +```{julia} +L(a=>2, b=>3) +``` + +::: {.callout-note} +## Watch out for early conversion! + +The above example will fail if instead of making `a` and `b` symbolic and then substituting values of `2` and `3`, the expression were substituted in at the outset: + +```{julia} +@giac_var x +a, b = 2, 3 +ex = ((a^x - x*log(a)) / (b^x - x*log(b)))^(1/x^2) +L = limit(ex, x ~ 0, 1) +``` + +This is because both `log(a)` and `log(b)` are then inexact values for $\log(2)$ and $\log(3)$ respectively. +::: + + +### Differentiation + +Derivatives are taken through `diff(ex, var, ...)` with variants for multiple or mixed derivatives. The basic usage is pretty straightforward. The derivative with respect to $x$ of an expression is found as follows: + +```{julia} +@giac_var x a b +ex = sin(a*x - b) +diff(ex, x) +``` + +Second derivatives can be more succinctly expressed than repeated usage of `diff` by adding more variables: + +```{julia} +diff(ex, x, x) # 2nd derivative +``` + + +Alternatively, an order can be specified after the variable + +```{julia} +diff(ex, x, 30) +``` + +For mixed partials, the variables are specified sequentially + +```{julia} +diff(ex, x, a) +``` + +#### Symbolic functions and derivatives + +A symbolic function can be defined as follows: + +```{julia} +@giac_var u(t) +``` + +XXX -- update to Differential(t) ----XXX +The `D` function differentiates this formally: + +```{julia} +D(u) +``` + +#### Critical points + +The zeros of the derivative are critical points of a function (in addition to where a derivative is undefined in the domain of the function). + + +Here is an example of finding critical points of the periodic function $f(x) = \sin(x) + 3 \cdot \sin(5x)$: + +```{julia} +ex = sin(x) + 3 * sin(5x) +u = solve(diff(ex, x) ~ 0, x) +to_julia(u) +``` + +Critical points can also occur when the derivative is undefined at a point in the domain of the original function. For this example, we can see that $-1$ and $1$ are critical points with some squinting: + +```{julia} +ex = sqrt(abs(1 - x^2)) +solve(diff(ex,x) ~ 0, x) +``` + +That is $0$ is a critical point. But also are $1$ and $-1$: + +```{julia} +u = domain(ex, x) +``` + +Which is an odd way of writing all $x$ (as the union of $x \leq -1$, $-1 \leq x \leq 1$, and $1 \leq x$. + +However, this domain excludes $-1$ and $1$ (through `x<>-1` and `x<>1`): + +```{julia} +domain(diff(ex,x), x) +``` + +The `tabvar` function generates data for a table of variations, but it doesn't pretty print as easily as would be useful. + +```{julia} +using Giac.Commands: tabvar +ex = sqrt(abs(1-x^2)) +u = tabvar(ex, x) +``` + +Here is a possible pattern to print the output in a structured manner: + +```{julia} +function print_tabvar(u) + v = [string.(ui) for ui in u] + N = length(first(v)) + ls = [maximum(length(vi[j]) for vi in v) for j in 1:N] .+ 2 + nms = ["𝑥", "𝑓′(𝑥)", "𝑓(𝑥)", "𝑓′′(𝑥)"] + function centerprintline(io, x, n) + m,r = divrem(n-length(x), 2) + Δ = startswith(x, "-") ? 1 : 0 + print(io, " "^(m-Δ), x, " "^(m+r+Δ)) + end + io = IOBuffer() + println(io, "====================\n") + for (vi,nm) in zip(v, nms) + for j in 2:N + centerprintline(io, replace(vi[j], "\"" => " "), ls[j]) + end + println(io, "\t⋮ ", nm, "\n") + end + print(String(take!(io))) +end +print_tabvar(u) +``` + +This output shows the behavior at the key points $0$, $1$ and $\infty$ (this is an *even* function, so the behavior at $-1$ and $-\infty$ can be inferred). + +Reading down, we see that at $0$ there is a critical point with $f'(0) = 0$, $f(0) = 1$, and $f''(0) = -1$. On $(0,1)$, reading down we see: $f'(x)$ is negative, $f(x)$ is decreasing and concave down , and $f''(x)$ is negative. + +At $1$, we see $f'(1)$ is "`||`" or undefined, $f''(1)$ as well. +On the interval $(1, \infty)$ the derivative is positive, the function is increasing, and the function is concave down. + +At $\infty$ we see the limit of $f'(x)$ is $1$, the limit of $f(x)$ is $\infty$, the limit of $f''(x)$ is $0$. + +This should match this graph, the bounds of which are suggested from the printed output of `tabvar`: + +```{julia} +plot(ex, -1.64897066667, 1.67903066667) +``` + +Here is another example for a polynomial: + +```{julia} +u = tabvar(x^5 - x - 1) +print_tabvar(u) +``` + + + +#### Optimization + +Optimization problems can be solved with familiar commands. + +Consider the task + +> Find the rectangle with largest area amongst all rectangles with perimeter $P$. + +Below, we set up a constraint and objective equation. Using `solve` we first + +* solve for `h` from the constraint in terms of `w`; then +* substitute that into the objective; and then +* find a critical point in the area as a function of `w`. + +```{julia} +@giac_var w h P A +constraint = P ~ 2w + 2h +objective = A ~ w * h +h0 = solve(constraint, h) |> only +dA = diff(objective(h=>h0), w) +w0 = solve(dA, w) |> only +``` + +Finally, we find values for the width, height, and area: + +```{julia} +w0, h0(w=>w0), objective(w => w0, h=>h0) +``` + +---- + +The following is another example. + +> What angle $\theta$ between two edges of length $x$ will result in an isosceles triangle with the largest area? + +We define two variables and then package them to get an expression for the area, `a`: + +```{julia} +@giac_var x θ +b = x*sin(θ) +h = x*cos(θ) +a = b*h +``` + +Solving for critical points is easy: + +```{julia} +u = solve(diff(a, θ), θ) +``` + +We filter out the negative values: + +```{julia} +θs = filter(x -> to_julia(x > 0), collect(GiacExpr, u)) +``` + +There are still two critical points. Which leads to a maxima? We compute the area for each and observe, seeing $\pi/4$, as might have been guessed: + +```{julia} +[θs [simplify(a(θ => u)) for u in θs]] +``` + + + +#### Implicit differentiation + +The implicit function theorem provides conditions so that the graph defined through the equation $F(x,y) =0$ can---in part---be described by the graph of a function $y=f(x)$. The function $f(x)$ being *implicitly* defined. (The function $F$ is continuously differentiable and the *partial* of $F$ in $y$ at $(x_0,y_0)$ is non zero.) + +Implicit differentiation is a technique to find the derivative of $y$ with respect to $x$ without needing to solve for $y$ in terms of $x$. The method is implemented in the function `implicitdiff`. The simplest calling pattern is `implicitdiff(eqn, y, x)` to differentiate the variable `y` with respect to `x`. + +This illustrates that when $y = f(x)$ the function returns $f'(x)$: + +```{julia} +using Giac.Commands: implicitdiff +@giac_var x y +implicitdiff(y ~ sin(x), y, x) +``` + +Now for a figure eight curve: + +```{julia} +@giac_var a +eqn = x^4 ~ a^2*(x^2 - y^2) +dy_dx = implicitdiff(eqn, y, x) +``` + +We can plot this for a given value of `a` in the following way (this graphic will be discussed later): + +```{julia} +using Giac.Commands: left, right + +eqn′ = eqn(a => 2) +u = build_function(left(eqn′)-right(eqn′), x, y) + +xs = ys = range(-3, 3, 100) +contour(xs, ys, u, levels=[0]; legend=false) + +# find tangent line expression at a point +P = (x0, y0) = (giac_eval("sqrt(2)"), giac_eval("1")) +m = dy_dx(a=>2, x=>x0, y=>y0) +tl = y0 + m*(x-x0) + +# plot a tangent line at a point on the curve +plot!(tl) +scatter!([to_julia.(P)]; markersize=5) +``` + +We added a tangent line at a known point, but the pattern could be used at any point on the curve, save for $x=2, 0$, and $-2$ where the partial derivative in $y$ is undefined. + + +### Integration + +In Calculus II there are techniques of integration to be learned: + +* integration by parts +* trigonometric integrals +* partial fractions +* The definition of integration can be extended to incorporate infinities, or improper integrals + +Save for special cases, such techniques and extensions are handled by `Giac` and its `integrate` command. + +Further, there are several formulas where an integral takes on a geometric meaning beyond the area under a curve, including: + + +* Area between 2 curves: $A = \int_a^b (f(x) - g(x)) dx$. +* volume of revolution: $V = \int_a^b \pi r(x)^2 dx$. +* cylindrical shell: $V = \int_a^b 2\pi x f(x) dx$. +* arc length: $L = \int_a^b \sqrt{1 + f'(x)^2} dx$, +* surface area: $SA = \int_a^b 2\pi f(x) \sqrt{1+ f'(x)^2} dx$. + + + +In `Giac`, the `integrate` function computes integrals symbolically by finding an anti-derivative. The Risch algorithm is used and can be directly called through `risch`. Bear in mind, not all integrands have an antiderivative that can be algebraically expressed, the Risch algorithm only works for certain functions. + +The *indefinite integral*, $\int f(x) dx$, is computed with `integrate(f(x), x)` where `f(x)` is some expression depending on `x` and may have symbolic parameters. + +Definite integrals, $\int_a^b f(x)dx$, are computed with `integrate(f(x), x, a, b)`. + +For example: + +```{julia} +@giac_var x c a b +integrate(x * exp(x^2), x) # integration by parts +``` + +```{julia} +integrate(sin(x)^2, x) # integration by parts +``` + + +```{julia} +integrate(x^5/(36x^2 + 1)^(3//2), x, 0, 1//6) # trig subs with x = tan(theta)/6 +``` + + +```{julia} +ex = (x^4 + 1) / ( (x^2 + 1)^2 * (x^2 - 4)^2) +integrate(ex, x) # partial fraction +``` + +We can see how this is done by taking the partial fraction decomposition provided by `partfrac`: + +```{julia} +us = partfrac(ex) +``` + +and then integrating term-by-term: + +```{julia} +[integrate(a, x) for a in Giac.arguments(us)] +``` + +For polynomials with real-valued coefficients, the partial fraction decomposition leads to terms that can always be integrated: + +```{julia} +@giac_var a b c d e x +u = (a + b*x) / (c + d*x + e*x^2) +integrate(u, x) +``` + +The underlying algorithm to integrate rational functions doesn't necessarily need the factorization of the denominator into *irreducible* factors, as only a [square-free factorization](https://www.researchgate.net/profile/Sam-Blake-6/publication/259815903_Symbolic_Integration_I_Transcendental_Functions_by_Manuel_Bronstein/links/5e5fab4c92851cefa1dc85c6/Symbolic-Integration-I-Transcendental-Functions-by-Manuel-Bronstein.pdf) is needed. + + +#### Examples of integration + +This example finds the area bounded by two parabolas: + +```{julia} +f(x) = 1 - 2x^2 +g(x) = x^2 +purge(x); assume(x, "real") +as = solve(f(x) ~ g(x), x) +a,b = sort(as) +integrate(f(x) - g(x), x, a, b) +``` + +---- + +Following an example from an [AP](https://apcentral.collegeboard.org/media/pdf/Volumes_of_Solids_of_Revolution_Calculus_CM.pdf) calculus test, we look at the solid formed by rotating the region bounded by $y = \sqrt{x+2}$ and $y=e^x$ about the line $y = -2$. + +We first find the intersection points, solutions to $f(x) = g(x)$. +We numerically solve for these with the `Roots` package, as `solve` had no success and `fsolve` only identifies the value we assign to `b` below. + +```{julia} +@giac_var x +f(x) = exp(x) +g(x) = sqrt(x + 2) +import Roots +a,b = Roots.find_zeros(x -> f(x) - g(x), (-2, 10)) +``` + + +The integral requires two radii from the line $y=-2$. We have on $(a,b)$ that $g(x) > f(x) > 0$, so the distance of $g(x)$ from $y=-2$ is greater than that of $f(x)$, hence the ordering below: + +```{julia} +r1, r2 = (g(x) - (-2)), (f(x) - (-2)) +v = integrate(pi * (r1^2 - r2^2) ,x ,a, b) +``` + +---- + +This example finds the length of the graph of $x^2$ between $0$ and $1$: + +```{julia} +f(x) = x^2 +dL = sqrt(1 + diff(f(x),x)^2) +p = integrate(dL, x, 0, 1) +``` + +---- + +To find the surface area of the volume formed by rotating the graph of $f(x) = \sqrt{9 - x^2}$ between $-1 \leq x \leq 2$ we have: + +```{julia} +f(x) = sqrt(9 - x^2) +dSA = 2 * (pi * f(x)) * sqrt(1 + diff(f(x), x)^2) +``` + +And + +```{julia} +integrate(dSA, x, -1, 2) +``` + +### Taylor polynomial + +A Taylor polynomial is a polynomial expansion of a function at a point, $c$, up to an given order $n$. For a function $f(x)$ with $n+1$ derivatives on an open interval including $c$ the $n$th-order Taylor polynomial is: + +$$ +T_n(x) = f(c) + f'(c)(x-c) + \frac{f^{2}(c)}{2!}(x-c)^2 + \cdots + \frac{f^{n}(c)}{n!}(x-c)^n +$$ + +The Taylor series is the is the expansion is the "best" polynomial of degree $n$ (or less) that approximates the function at the point. Let the remainder be $R_n(x) = f(x) - T_n(x)$. Then using the mean value there exists a $\xi$ in the open interval such that + +$$ +R_{n+1}(x) = \frac{f^{n+1}(\xi)}{(n+1)!}(x - c)^{n+1} +$$ + +This is a constant times $(x-c)^{n+1}$ and is shortened to $O((x-c)^{n+1})$. + +The Taylor polynomial is computed by `series`.^[The Taylor series is a formal power series with infinitely many terms, the `n`th term in the series of the type above.] The calling pattern is `series(ex, var, c, n)`. + +This computes the Taylor expansion about $0$ up to order $n=4$: + +```{julia} +using Giac.Commands: series +@giac_var x +series(exp(x), x, 0, 4) +``` + + + +A series around `0` is also called a Maclaurin series. + +The following computes the expansion about $c=1$: + + +```{julia} +series(exp(x), x, 1, 3) +``` + +---- + +To convert a Taylor series (with the error term as above) to a polynomial can be done with a `convert` method for `GiacExpr` values---which does not follow the semantics of `Base.convert`. + +We illustrate the pattern: + +```{julia} +p₃ = series(exp(x), x, 1, 3) +POLYNOM = giac_eval("polynom") # polynomial type +P₃ = convert(p₃, POLYNOM) # not convert(T, x)! +``` + +This new object can be manipulated as any other symbolic expression. + + + +## Parametric description of functions + +Vectors can be used to describe curves parameterically (where the $x$ and $y$ motions are modeled by a third variable, typically $t$ for time). We describe this in a bit more detail later when discussing univariate, vector-valued functions. + + +Consider a person on an advanced ferris wheel with coordinates $x(t)$ and $y(t)$ describing position. The wheel of radius $r$ is centered $R$ units above and circles at radius $\omega$, the car has radius $r_0$ and rotates $6$ times faster. + +The car has axis described by: + +$$ +(r\sin(\omega\cdot t), R - r\cos(\omega \cdot t)). +$$ + +The rotation of the car around the axis by: + +$$ +(r_0 \sin(6\omega \cdot t), r_0 \cos(6 \omega\cdot t) +$$ + +(This could be modified by adjusting the starting angle.) + +The superposition (or addition) of these two give the position: + +```{julia} +@giac_var R r r0 t ω +ρ = [r*sin(ω*t), R - r*cos(ω*t)] + [r0*sin(6ω*t), r0*cos(6ω*t)] +``` + +We can plot for specific values: + +```{julia} +PI = Giac.Constants.pi # symbolic pi, or giac_eval("pi") +d = Dict(R => 35, r => 30, r0 => 4, ω => 2PI/120) +rt = substitute.(ρ, Ref(d)) # broadcast over `rho` not `d` +``` + +```{julia} +plot(rt..., 0, 100) # uses lambdify through a recipe +``` + +::: {.callout-note} +## Symbolic constants +Just as `Julia` has some constants made available in its `Base.MathConstants` module, `Giac` has some constants available in its `Giac.Constants` module. In the above, `pi` is retrieved and assigned to `PI`, so as not to conflict with the value of `pi` from `Base.MathConstants`. +::: + + +## Polar coordinates + +Polar coordinates are a special case of a representation. They are an alternate to Cartesian descriptions and use $(r,\theta)$ as an alternate to $(x,y)$. The translation is straightforward: $(x,y) = (r\cdot \cos(\theta), r\cdot\sin(\theta))$. + + +There isn't much special support for polar coordinates within `Giac` or `Julia`. The `plot` function has a `projection=:polar` argument that plots polar plots where $r=r(\theta)$. + +For example, + +```{julia} +rho(theta) = 1 + cos(theta) * sin(theta)^2 +@giac_var θ +``` +```{julia} +plot(rho(θ), 0, 2pi; projection=:polar, legend=false) +``` + + + +## Sequences and series + +A finite sequence can be generated in a `Julia`n manner using a comprehension. For example, + +```{julia} +[1/giac_eval("$i")^2 for i in 1:10] +``` + +The `seq` command can also be used to generate sequences. The syntax is `seq(expr, var, from ,to)`. The above could be computed through: + +```{julia} +using Giac.Commands: seq +@giac_var j +s = seq(1/j^2, j, 1, 10) +collect(GiacExpr, s) +``` + +The use of `seq` is a convenience, but it seems more idiomatic to just use comprehensions. + + +::: {.callout-note} +### Symbolic endpoints + +The number of terms is not symbolic, as can be the case in `SymPy`. + +::: + + + +## Differential equations + +A differential equation is an equation involving a variable, a function, and its derivatives. + +The `desolve` command can solve several different types of differential equations^[ +the [documentation](https://www-fourier.univ-grenoble-alpes.fr/~parisse/giac/doc/en/cascmd_en/cascmd_en458.html) says: linear differential equations with constant coefficients, +first order linear differential equations, +first order differential equations without `y`, +first order differential equations without `x`, +first order differential equations with separable variables, +first order homogeneous differential equations: `y′=F(y/x)`, +first order differential equations with integrating factor, +first order Bernoulli differential equations: `a(x)y′+b(x)y=c(x)y^n`, +first order Clairaut differential equations: `y=x\cdot y′+f(y′)`.] + +In the following we define an operator, using `D` to simplify the specification of the equation $y'(x) = y(x)$.^[`D` is to be *deprecated* in favor of `Differential`] Alternatively, `diff` can be used to specify a formal derivative which gives some flexibility in the variable name. + + +```{julia} +@giac_var x u(t) +eqn = D(u) ~ u +``` + + + +The `desolve` function is then used to solve differential equations. It is illustrated by passing in the equation (or a vector of equations including initial conditions) and the unknown function. The variable is inferred from the function object, or can be passed through, e.g. `u` or `u(x)` + +```{julia} +using Giac.Commands: desolve +desolve(eqn, u) +``` + +The constants come from integration. Initial conditions reduce the number of constants. These are specified as extra equations using a vector: + +```{julia} +desolve([eqn, u(0) ~ 3], u) +``` + +---- + + +The initial conditions can also involve constraints on derivatives, +etc. as with this next example, which comes from the `Giac.jl` +documentation. + + +The [damped harmonic oscillator](https://en.wikipedia.org/wiki/Harmonic_oscillator#Damped_harmonic_oscillator) models what happens to oscillating motion in a drag force. The basic form of the equation is + +$$ +\frac{d^2x}{dt^2} + 2\zeta \omega_0 \frac{dx}{dt} + \omega_0^2 x = 0. +$$ + +The value $\zeta$ is the damping ratio; the value $\omega_0$ is the angular frequency. Solutions will exhibit different behaviors depending on the damping ratio. + +To setup this problem, we have: + +```{julia} +@giac_var t x u(t) ω₀ ζ +uₜ = D(u) +uₜₜ = D(D(u)) +damped = uₜₜ + 2 * ζ * ω₀ * uₜ + ω₀^2 * u ~ 0 +``` + +The initial conditions are: + +```{julia} +ics = [u(0) ~ 1, # start at 1 + uₜ(0) ~ 0] # no initial velocity +``` + +We can now solve: + +```{julia} +out = desolve([damped, ics...], u) +``` + +We can plot for given values of $\zeta$. For values bigger than $1$ the oscillator just decays, for values less than $1$ it oscillates: + +```{julia} +sol = out(ω₀ => 1) +p1 = plot(sol(ζ => 3/2), 0, 15; title="ζ = 3/2") +p2 = plot(sol(ζ => 1/8), 0, 15; title="ζ = 1/8") +plot(p1, p2) +``` + +---- + + +We follow with a more involved example. The following equations model projectile motion using Newton's laws and projectile motion with a drag force proportional to velocity. This proportion is given by a constant $\gamma$. + +We use `u` and `v` to model the $x$ and $y$ coordinates of motion over time, $t$. Inverting $x$ to get $t$, allows the solution of $y$ in terms of $t$. + +```{julia} +@giac_var x0 y0 v0 γ 𝑔 +@giac_var t x u(t) v(t) +Dₜu = D(u) +Dₜₜu = D(D(u)) +Dₜv = D(v) +Dₜₜv = D(D(v)) +eq₁ = Dₜₜu ~ -γ * Dₜu +eq₂ = Dₜₜv ~ -𝑔 -γ * Dₜv +``` + +If we set $\gamma = 0$ and solve, we get the $y$ values as a function of $t$: + +```{julia} +a1 = desolve(eq₁(γ=>0), u(t)) +a2 = desolve(eq₂(γ=>0), v(t)) +tₓ = only(solve(a1 ~ x, t)) # invert; only one solution +yₓ = a2(t => tₓ) +``` + +We can see the form of a parabola, with two parameters adjusting its shape related to the initial conditions. This is anticipated from physics, where projectile motion s an early example. + +Here, we specify initial values on position and velocity so that we can ultimately plot a solution: + +```{julia} +@giac_var α v₀ +icx = (u(0) ~ 0, Dₜu(0) ~ v₀*cos(α)) +icy = (v(0) ~ 0, Dₜv(0) ~ v₀*sin(α)) + +a1 = desolve([eq₁(γ => 0), icx...], u(t)) +a2 = desolve([eq₂(γ => 0), icy...], v(t)) + +tₓ = only(solve(x ~ a1, t)) # only one solution +yₓ = a2(t => tₓ) +``` + + +If there is a positive $\gamma$, there will be drag and the formulas will change to reflect that. + +```{julia} +𝑎1 = desolve([eq₁, icx...], u(t)) +𝑎2 = desolve([eq₂, icy...], v(t)) + +𝑡ₓ = only(solve(x ~ 𝑎1, t)) # only one solution +𝑦ₓ = 𝑎2(t => 𝑡ₓ) +``` + +The difference can be visualized through plotting. The proper way to get the values for the constants is to specify numeric values in the initial conditions: + + +```{julia} +PI = giac_eval("pi") +a = yₓ(𝑔 => 32, v₀ => 200, α => PI/4) +M = maximum(solve(a ~ 0)) +``` + +```{julia} +plot(a, 0, to_julia(M)) +``` + +```{julia} +𝑎 = 𝑦ₓ(𝑔 => 32, v₀ => 200, α => PI/4, γ => 1//2) +M = maximum(solve(𝑎 ~ 0)) +``` + +Peeking at the values show they are explicit, but complicated: + +```{julia} +𝑎, M +``` + +Adding the plot with a drag force, shows the models track each other initially, but evenutally the drag force leads to a big difference: + +```{julia} +plot!(𝑎, 0, to_julia(M)) +``` + +---- + +There are other methods available for other types of equations in this [Giac documentation](https://www-fourier.univ-grenoble-alpes.fr/~parisse/giac/doc/en/cascmd_en/index.html#sec1024). + + +## Vectors and matrices + +Julia has an `AbstractArray` type which can be used to store vectors and matrices. Similarly, `Giac` has internal vector and matrix structures. These have an iteration protocol that usually allows them to be treated as a Julia vector or matrix. + +For example, a solution returned by `solve` might be in an internal list (vector) and below we see how to filter the value. + +```{julia} +out = solve(cos(x) ~ x/4, x) +``` + +To find just the positive solutions, we have: + +```{julia} +filter(>(0) ∘ to_julia, collect(GiacExpr, out)) +``` + +Or, perhaps if `Float64` values are preferred over `GiacExpr` values: + +```{julia} +filter(>(0), collect(Float64, out)) +``` + +The collection is necessary as `filter` doesn't specialize on its second argument being of type `GiacExpr`.^[Other functional programming operations, like `map`, `reduce`, `foldr`, work, but the generic `filter` is defined for `AbstractArray` and not a general iterator.] The `collect` call turns the list into a vector of `GiacExpr` objects or `Float64` objects. + +Alternatively, a comprehension could have been used: + +```{julia} +v = [x for x in out if 0 < to_julia(x)] +``` + +Giac vectors can also be directly produced by the `list `command. + +```{julia} +using Giac.Commands: list +vv = list([1,2,3,x]) +``` + +---- + + +Vectors have a *dot* product and a `dot` command is available for `GiacExpr` objects in addition to a generic `dot` method for vectors of `GiacExpr` type: + +```{julia} +using LinearAlgebra # gives access to methods dot, cross, ... that Giac extends +dot(vv, vv), dot(out,out) +``` + +---- + +Three-dimensional vectors have a *cross* product and a `cross` command is available for `GiacExpr` objects in addition to a generic `cross` method for vectors of `GiacExpr` type: + +```{julia} +v,w = [0,1,x], [2,1,x^2] +vv, ww = list(v), list(w) +cross(v, w), cross(vv, ww) +``` + +### Matrices + +Some Giac commands return "matrices". + +For example, consider the roots of this polynomial $p= x^5-9*x^4+30*x^3-46*x^2+33*x-9$: + +```{julia} +p = x^5-9*x^4+30*x^3-46*x^2+33*x-9 +Giac.Commands.roots(p) +``` + +This is a vector of vectors, but cam also be viewed as a matrix with each row given by a vector. Matrices can be approached in several different manners. + +---- + +Matrices with `GiacExpr` elements are produced as any other matrix in `Julia`: + +```{julia} +@giac_var x +M = [1 2; 3 x] +``` + +Some, but decidedly not all, generic linear algebra operations will work with such matrices. (E.g, `tr` will, `det` will not.) + +---- + +Matrices stored as a Giac object are directly produced by calling `giac_eval` with the matrix specified as a vector of row vectors: + +```{julia} +mat_string = "matrix[[1,2], [2,3], [3,4]]" +m = giac_eval(mat_string) +``` + +We have the expected values: + +```{julia} +size(m), length(m), eachindex(m) +``` + +The iteration protocol should respect the shape: + +```{julia} +sin.(m) +``` + +---- + +`Giac.jl` provides a `GiacMatrix` class to turn a `Julia` array into a symbolic matrix that has methods defined for it: + +```{julia} +M = GiacMatrix([1 2; 3 4]) +``` + +```{julia} +using LinearAlgebra +det(M), tr(M) +``` + +In particular, there is support for `det`, `inv`, `tr`, and `transpose` along with other commands from `Giac`---which may not follow Julia's patterns---such as `lu`. + +::: {.callout-note} +## But not all +There are some generic linear algebra functions which *will not* work with either a vector of `GiacExpr`s or a `GiacMatrix`. Notably, `norm` which will fail with a recursion. There are specific `Giac` commands for norms: `maxnorm`, `l1norm`, `l2norm`, ... +::: + +#### Switching between matrix forms + +As an example of the different data structures, we return to the output for the roots of a polynomial: + +```{julia} +u = Giac.Commands.roots(p) +``` + +This vector of vectors, can be viewed as a matrix, but needs to be manipulated into being one. + +To get an internal Giac matrix, we create a string for the matrix constructor using interpolation and then evaluate: + +```{julia} +m_giac = giac_eval("matrix$u") +``` + +To get a Julia matrix, we collect each row into a `Julia` vector, transpose (or permute dimensions) into row vectors, and then concatenate with `vcat(collection...)` or by a reduction: + +```{julia} +m_julia = reduce(vcat, transpose.(collect.(GiacExpr, u))) +``` + +To get a `GiacMatrix`, the last result can be passed to `GiacMatrix`: + +```{julia} +GiacMatrix(m_julia) +``` + +`GiacMatrix` has a method for a `GiacExpr` object of the proper form allowing the value `u` to be passed directly to the constructor: + +```{julia} +GiacMatrix(u) +``` + +Finally, `GiacMatrix` has a method for `Vector{<:Vector}` which might be convenient, depending on the data. This structure is returned by broadcasting `collect` over `u`: + +```{julia} +m_GiacMatrix = GiacMatrix(collect.(GiacExpr, u)) +``` + +Finally, a `GiacMatrix` objected can be collected to produce a `Julia` matrix with `GiacExpr` entries: + +```{julia} +collect(GiacExpr, m_GiacMatrix) +``` + + + +## Functions from $R^n \rightarrow R^m$ + +We turn to discussing scalar-or-vector-valued functions of one or more input variables. + +### $R \rightarrow R^m$ + +Earlier, we discussed parameterized curves, which are an example of a univariate vector-valued function, in that case with $m=2$. For different value of $m$, univariate, vector-valued functions can also be represented by vectors of symbolic expressions using `Julia` vectors. + +#### Visualize + +For $m=2$ or $3$, a typical plot is to plot the vectors as though they are anchored a the origin. The `plot(f1,f2,[f3], t0, t1)` recipe will do this, as was illustrated in the case $m=2$. In this example, we let $m=3$ to get a space curve: + +```{julia} +@giac_var r R a b t +spiral = [(R + r*cos(b*t))*cos(a*t), (R + r*cos(b*t))*sin(a*t), r*sin(b*t)] +``` + +Broadcasting is used to conveniently substitute in values for the parameters so that a specific expression can be plotted. The prime in the name is just notation---not a derivative---derivatives will be illustrated shortly. + +```{julia} +spiral′ = substitute.(spiral,R=>3, r=>1, a=>3, b=>2) +plot(spiral′..., 0, 2pi) +``` + + The splatting done in the `plot` command turns the 3-dimensional vector into 3 symbolic expressions which `plot` treats as function objects by the previously defined recipe. (As an alternative to calling `build_function` on each component.) + + +#### Derivatives + +Univariate, Vector-valued functions have derivatives that are taken component by component. As such, simply broadcasting `diff` over a vector will yield the derivative: + +```{julia} +dv = diff.(spiral′, t) +``` + +To add the tangent vector to the curve, we might do: + +```{julia} +plot(spiral′..., 0, 2pi) +t0 = 1 +p0 = to_julia.(Tuple(substitute.(spiral′, t=>t0))) +p1 = to_julia.(Tuple(substitute.(dv, t=>t0))) +plot!([p0, p1]; arrow=true) +``` + +The above line is added as a vector of tuples, each tuple representing a point. The `arrow=true` is only hopeful, as 3-dimensional arrows are not supported for all backends. + +XXX --- a float method would avoid broadcasting `to_julia` + +#### Integration + +Component-by-component integration, like differentiation, can be done by broadcasting the `integrate` command over the data structure.: + +```{julia} +using Giac.Commands: integrate +integrate.(spiral, t, 0, pi) +``` + +##### Line integrals + +A line integral might be generically written over a curve $C$ or with a paramterization, $r(t)$, of $C$, yielding + +$$ +I = \int_C f(\vec{x}) ds = \int_a^b f(r(t)) dt +$$ + +An example is the computation of arc-length which is found by integrating the *norm* of the tangent vector: + +$$ +L = \int_a^b \lVert r'(t) \rVert dt. +$$ + +The `spiral` curve doesn't have an analytical anti-derivative; instead we look at the parameterization of a circle, with its known arc-length of $2\pi*r$: + +```{julia} +@giac_var r t +purge(r); assume(r > 0) +PI = giac_eval("pi") +v = [r*cos(t), r*sin(t)] +``` + +We avoid calling `norm` to form `dL` below, as that method isn't happy with a vector of `GiacExpr` objects (though we could do this internally with `Giac` commands): + +```{julia} +dL = sqrt(sum(xi^2 for xi in diff.(v, t))) +integrate(dL, t, 0, 2PI) +``` + +We avoided using `LinearAlgebra.norm`, as that method fails due some expected generic assumptions. `Giac` provides a norm function `l2norm` which we might have used, after converting `v` to a Giac list: + +```{julia} +using Giac.Commands: list, l2norm +vv = list(v) +dL = l2norm(diff(vv, t)) # no need to broadcast within Giac +integrate(dL, t, 0, 2PI) +``` + +---- + +Another integral along a path is the work integral: + +$$ +W = \int_C F \cdot dr = \int_a^b F(r(t)) \cdot r'(t) dt, +$$ + +where $r$ is a space-curve describing the path and $F$ some field (a function from $R^m$ into $R^m$). + +Here is an example: compute the work moving around a circle at the origin of radius $\rho$ through the field $F(x,y) = [x, 3xy]$. + +A parameterization is simply + +```{julia} +@giac_var ρ t +r = ρ .* [cos(t), sin(t)] +``` + +We set up $F$: + + +```{julia} +@giac_var x y +F = [x, 3*x*y] +``` + +The composition $F(r(t))$ can be done through substitution, here we combine with a dot product using the infix operation. Notice the broadcasting in the first two commands: + +```{julia} +Fr = substitute.(F, x=>r[1], y=>r[2]) +dr = diff.(r, t) +dW = Fr ⋅ dr +``` + +Finally we integrate `dW` over $[0,2\pi]$ to get the work done: + +```{julia} +W = integrate(dW, t, 0, 2PI) +``` + +### $R^n \rightarrow R$ + +Functions taking $R^n$ to $R$ are called multivariate, scalar-valued functions. + +#### Define + +An expression with $n$ free variables that yields a scalar quantity can be viewed as a function from $R^n$ into $R$. The `build_function` method can make a concrete function which takes and returns `Julia` values with its rule being the expression. + +For example, + +$$ +F(x,y) = (y-5)\cdot \cos(4\cdot \sqrt{(4-x)^2 + y^2}) - x\cdot \sin(2\sqrt{x^2 + y^2}) +$$ + +could be modeled with: + +```{julia} +@giac_var x y +ex = (y-5)*cos(4*sqrt((4-x)^2 + y^2)) - x*sin(2sqrt(x^2 + y^2)) +``` + +Evaluation can be achieved through substitution: + +```{julia} +ex(x=>1, y=>2) +``` + +or after calling `build_function` and calling the new function within `Julia`: + +```{julia} +λ = build_function(ex, x, y) +λ(1, 2) +``` + +#### Visualization + +The common graphics for visualization are contour plots and surface plots. For each, the call is similar and we defer the work to the function that was built. + +```{julia} +xs = ys = range(-10, 10, 250) +surface(xs, ys, λ) +``` + +```{julia} +contour(xs, ys, λ) +``` + +##### Implicit plots + +The contour plot graphs $F(x,y) = c$ for different values of $c$. The set of zeros of $F$, solutions to $F(x,y)=0$, is often of particular interest. For this equation, a plot *implicitly* defines a function $y(x)$ where $F(x, y(x)) = 0$ for neighborhoods around most points of the resulting curve. + +We can generate the plot by passing in a value to the `levels` argument: + +```{julia} +contour(xs, ys, λ; levels=[0]) +``` + +Through some algebra, the same approach can be used to plot the solution set to any equation involving two variables. + + + + +#### Derivatives + +For multivariate functions the partial derivatives are important. These are found using `diff` and specifying the variable to differentiate by: + +For example, + +```{julia} +diff(ex, x) +``` + +The mixed partial, can be computed by, say, `diff(ex, x, y)`. Under assumptions, the order doesn't matter, as this computation illustrates: + + +```{julia} +diff(ex, x, y) - diff(ex, y, x) |> simplify +``` + + +The gradient, is the vector of partial derivatives. For an expression, this can be computed through a comprehension: + +```{julia} +[diff(ex, xᵢ) for xᵢ ∈ (x,y)] +``` + +Or, the `grad` command can be used: + +```{julia} +using Giac.Commands: grad +grad(ex, [x,y]) +``` + +The *hessian* is the matrix of mixed partials, which, again, can be computed through a comprehension. As the output is lengthy, we compare just one term computed in two different ways: + +```{julia} +vars = [x,y] +out = [diff(ex, xᵢ, xⱼ) for (xᵢ, xⱼ) ∈ Iterators.product(vars, vars)] +out[1,1] - diff(ex, x, x) |> simplify +``` + +Or, using the built-in `hessian` command: + +```{julia} +using Giac.Commands: hessian +out = hessian(ex, [x,y]) +out[1][1] - diff(ex, x, x) |> simplify +``` + +The `hessian` command returns a Giac vector of vectors (not a matrix, though related) hence, the double call to `getindex` above. + +#### integration + +Integration of a multivariate, scalar valued function is typically performed by Fubini's theorem, which allows the integration to be performed variable by variable: + +$$ +\int_{X \times Y} f(x,y) dA = \int_X (\int_Y f(x,y) dy) dx = +\int_Y (\int_X f(x,y) dx) dy. +$$ + +For example, to integrate $F(x,y) = x^2 \cdot y^3$ over the triangular region formed by the vertices $(0,0)$, $(1,0)$, $(0,1)$ we might have: + +```{julia} +@giac_var x y +F = x^2 * y^3 +Iy = integrate(F, y, 0, 1-x) # integrating int_{x=0}^1 int_{y=0}^{1-x} F(x,y) dy dx +integrate(Iy, x, 0, 1) +``` + +(Integrating is made easy, but *not* the task of identifying valid endpoints to describe the region integrated over.) + +### $R^n \rightarrow R^m,\quad m,n > 1$ + +Multivariate, vector-valued functions can be modeled as vector-valued functions where the expressions are multivariate scalar-valued functions. + + +There are two special operations for such functions, the divergence (`divergence`) and the the curl (`curl`) (when $n = m= 3$). + +```{julia} +using Giac.Commands: divergence, curl, grad +@giac_var x y z +vars = [x, y, z] +F = [x*sin(y), x*cos(y), z] +divergence(F, vars) +``` + +The divergence, $\nabla \cdot F$ can also be realized via a dot product: + +```{julia} +sum(diff(Fi, v) for (Fi, v) ∈ zip(F, vars)) +``` + +The above commands used a vector of `GiacExpr` objects. The same works for an internal `Giac` vector. Here we use `list` to create such: + +```{julia} +divergence(list(F), list(vars)) +``` + + +The `curl` command is similarly called as `divergence`: + +```{julia} +curl(F, vars) +``` + +This $F$ has rotation about the $z$ axis, so the $x$--$y$ terms of the curl are $0$. + +#### Visualization + +The visualization of vector fields is most familiar in the $2\times 2$ case, in which each point in the $x$--$y$ plane has a $2$-dimensional vector, $F(x,y)$, anchored at the point. + +The `quiver` function can be used to draw arrows in two dimensions. It is passed the $x$ values and $y$ values of the positions, and the $u$ and $v$ values of the directions. These can be produced with comprehensions as follows.^[The construction of `F = [y/d, -x/d]` would be more succinctly performed by `[x,y]/d` but that fails, as there isn't (currently) a specialized operation for such vector-scalar operations. However, broadcasting would still work, `[x,y] ./ d`.] + +```{julia} +@giac_var x y +d = 1 + x^2 + y^2 # scale factor +F = [y/d, -x/d] +us = build_function.(F, (x,), (y,)) +xs = ys = range(-2, 2, 10) +``` + +That sets up a function, $F$, from $R^2$ to $R^2$ and the $x$ and $y$ values for the grid of points. To expand this grid, we generate fours vectors looping over the paired values: + +```{julia} +xxs = [x for x in xs for y in ys] +yys = [y for x in xs for y in ys] +uus = [us[1](x,y) for x in xs for y in ys] +vvs = [us[2](x,y) for x in xs for y in ys]; +``` + +These vectors can then be passed along to `quiver`: + +```{julia} +quiver(xxs, yys, quiver=(uus, vvs)) +``` + +As an aside, an alternate means to produce the four vectors could be to use `invert` from `SplitApplyCombine`: + +```{julia} +#| eval: false +import SplitApplyCombine: invert +uu = build_function(list(F), x, y) + +xxs, yys = invert([(x,y) for x in xs for y in ys]) +uus, vvs = invert([uu(x,y) for x in xs for y in ys]); +``` + + +#### Derivatives + +The Jacobian of a vector-valued function is defined by + +$$ +J_F = [\frac{\partial F_1}{\partial x_1} \cdots \frac{\partial F_n}{\partial x_n}] += \left[ +\begin{array}{c} +\nabla F_1^T \\ +\nabla F_2^T \\ +\cdots \\ +\nabla F_n^T \\ +\end{array} +\right] +$$ + +The Jacobian is the *best* linear approximation in the sense + +$$ +\lVert(F(x) - F(a)) - J \cdot (x - a)\rVert = o(\lVert x - a\rVert) +$$ + + +If we express our vector-valued function using a vector of `GiacExpr` objects, we can compute this with a comprehension:^[The comprehension syntax `[F(x,y) for x in xs, y in ys]` creates a 2-d dense matrix, with additional variables leading to higher dimensional arrays. The comprehension syntax `[F(x,y) for x in xs for y in ys]` is like a short cut to two `for` loops and produces a vector with the last iteration happening for each value of the inner variable (as would be done with nested `for` loops). Alternatively, something like `vcat(map(F, Iterators.product(ys, xs)))` will also work similarly, though `F` would be passed a tuple.] + +```{julia} +[diff(Fi, vj) for Fi in F, vj in vars] +``` + +If the function is a giac vector, the syntax shifts and gets a bit cumbersome. + +To compare, we take an easier example: + +```{julia} +F = [x, 2x*y] +J = [diff(Fi, vj) for Fi in F, vj in vars] +``` + +To compute with a the gradient, we might have: + +```{julia} +out = [grad(Fi, vars) for Fi in F] +``` + +We see a vector of vectors, but we convert to a `Vector` to pass to `GiacMatrix` + +```{julia} +JJ = GiacMatrix(collect.(GiacExpr, out)) +``` + +This compares with `J` above. + + +#### Integration + +Let's illustrate integration via a confirmation of Stokes' Theorem: + +$$ +\int_C F \cdot dr = \iint_S (\nabla \times F) \cdot n dS. +$$ + + +Let $F = \langle x + y, 2xy, y^2 \rangle$. + +Suppose, we have a curve that sits on the plane given by $f(x.y) = c - (a*x + b*y)$ above the circle of radius $\rho$ centered at the origin. Then we can write $r(t)$ by putting the parameterization $\langle \rho \cos(t), \rho \sin(t)\rangle$ through $f$: + +```{julia} +@giac_var a b c ρ t x y +PI = giac_eval("pi") +F = [x + y, 2x*y, y^2] +f(x,y) = c - (a*x + b*y) +𝑟 = ρ .* [cos(t), sin(t)] +r = vcat(𝑟, f(𝑟...)) +``` + +The line integral can be computed as follows: + +```{julia} +Fr = substitute.(F, x=>r[1], y=>r[2], z=>r[3]) +dr = diff.(r, t) +line_integral = integrate(Fr ⋅ dr, t, 0, 2PI) +``` + +This simplifies to: + +```{julia} +line_integral = simplify(line_integral) +``` + +For the surface integral, we take the disc to be the surface. A parameterization for that comes from $r(u,v) = \langle u\cdot\cos(t), u\cdot\sin(t), c - (a\cdot u\cdot\cos(t) + b\cdot u \cdot\sin(t)) \rangle$, where $0 \leq u \leq \rho$ and $0 \leq v \leq 2 \pi$. + +With the surface so parameterized, we compute: + +$$ +\oint_S (\nabla \times F) \cdot dS = \iint_D (\nabla \times F)(r(u,v)) \cdot (r_u \times r_v) du dv, +$$ + +With $r_u$ and $r_v$ the partial derivatives. Continuing: + +```{julia} +@giac_var u v +𝑟 = u .* [cos(v), sin(v)] +r = vcat(𝑟, f(𝑟...)) +ru, rv = diff.(r, u), diff.(r, v) +curlF = substitute.(curl(F, [x,y,z]), x=>r[1], y=>r[2], z=>r[3]) +surface_integral = integrate(integrate(curlF ⋅ (ru × rv), v, 0, 2PI), u, 0, ρ) +``` + +The two integrals being equal, Stokes' theorem is confirmed for this example. + + +::: {.callout-note collapse="true"} +## Environment of `Julia` when generated + +`Julia` version: + +```{julia} +VERSION +``` + +Packages and versions: + +```{julia}u +sing Pkg +Pkg.status() +``` + +::: diff --git a/quarto/alternatives/make_pdf.jl b/quarto/alternatives/make_pdf.jl index aab5b1d..4b15b4f 100644 --- a/quarto/alternatives/make_pdf.jl +++ b/quarto/alternatives/make_pdf.jl @@ -5,6 +5,7 @@ module Make dir = "alternatives" files = ( + "giac", "symbolics", "SciML", "plotly_plotting", diff --git a/quarto/alternatives/symbolics.qmd b/quarto/alternatives/symbolics.qmd index f4a3225..aec506f 100644 --- a/quarto/alternatives/symbolics.qmd +++ b/quarto/alternatives/symbolics.qmd @@ -275,7 +275,7 @@ For example, `0+x` should simplify to `x`, as well `1*x`, `x^0`, or `x^1` should * `-x` becomes `(-1)*x` * `x * x` becomes `x^2` (and `x^n` if more terms). Meaning this expression is represented as a power, not a product - * `x + x` becomes `2*x` (and `n*x` if more terms). Similarly, this represented as a product, not a sum. + * `x + x` becomes `2*x` (and `n*x` if more terms). Similarly, this is represented as a product, not a sum. * `p/q * x` becomes `(p*x)/q)`, similarly `p/q * x/y` becomes `(p*x)/(q*y)`. (Division wraps multiplication.) diff --git a/quarto/basics/calculator.qmd b/quarto/basics/calculator.qmd index 75443dc..a498dab 100644 --- a/quarto/basics/calculator.qmd +++ b/quarto/basics/calculator.qmd @@ -238,7 +238,7 @@ Numeric combinations, as above, will be easier to check for correctness when var The calculator must use some rules to define how it will evaluate its instructions when two or more operations are involved. We know mathematically, that when $1 + 2 \cdot 3$ is to be evaluated the multiplication is done first then the addition. -With the Google Calculator, typing `1 + 2 x 3 =` will give the value $7$, but *if* we evaluate the `+` sign first, via `1` `+` `2` `=` `x` `3` `=` the answer will be 9, as that will force the addition of `1+2` before multiplying. The more traditional way of performing that calculation is to use *parentheses* to force an evaluation. That is, `(1 + 2) * 3 =` will produce `9` (though one must type it in, and not use a mouse to enter). Except for the most primitive of calculators, there are dedicated buttons for parentheses to group expressions. +With the Google Calculator, typing `1 + 2 x 3 =` will give the value $7$, but *if* we evaluate the `+` sign first, via `1` `+` `2` `=` `x` `3` `=` the answer will be 9, as that will force the addition of `1+2` before multiplying. The more traditional way of performing that calculation is to use *parentheses* to force an evaluation. That is, `(1 + 2) * 3 =` will produce `9` (though one must type it in, and not use a mouse to enter it). Except for the most primitive of calculators, there are dedicated buttons for parentheses to group expressions. In `Julia`, the entire expression is typed in before being evaluated, so the usual conventions of mathematics related to the order of operations may be used. These are colloquially summarized by the acronym [PEMDAS](http://en.wikipedia.org/wiki/Order_of_operations). @@ -374,7 +374,7 @@ The Google calculator has two built in constants, `e` and `π`. Julia provides t pi ``` -Whereas, `e` is is not simply the character `e`, but *rather* a [Unicode](../unicode.html) character typed in as `\euler[tab]`. +Whereas, `e` is not simply the character `e`, but *rather* a [Unicode](../unicode.html) character typed in as `\euler[tab]`. ```{julia} @@ -383,7 +383,7 @@ Whereas, `e` is is not simply the character `e`, but *rather* a [Unicode](../uni :::{.callout-note} ## Note -However, when the accompanying package, `CalculusWithJulia`, is loaded, the character `e` will refer to a floating point approximation to the Euler constant . +However, when the accompanying package, `CalculusWithJulia`, is loaded, the character `e` will refer to a floating point approximation to the Euler constant. ::: @@ -515,7 +515,7 @@ So we have all these different, but related, uses to find logarithms: log(e), log(2, e), log(10, e), log(e, 2) ``` -In `Julia`, the "generic" function `log` not only has different implementations for different types of arguments (real or complex), but also has a different implementation depending on the number of arguments. +In `Julia`, the "generic" function `log` not only has different implementations for different types of arguments (real or complex), but also a different implementation depending on the number of arguments. ### Examples @@ -797,7 +797,7 @@ numericq(val) ###### Question -You are asked to cook chicken is an unfamiliar kitchen. Your recipe says to turn the oven to 200 Celsius, but you the oven is calibrated in Fahrenheit. Which value is closest? +You are asked to cook chicken in an unfamiliar kitchen. Your recipe says to turn the oven to 200 Celsius, but you the oven is calibrated in Fahrenheit. Which value is closest? ```{julia} #| echo: false From d4b3be00a0df94407ee95cd783be2aa03b258372 Mon Sep 17 00:00:00 2001 From: Typo Fix Bot Date: Wed, 3 Jun 2026 18:37:33 +0000 Subject: [PATCH 2/7] Fix prose typos and grammatical errors across 26 .qmd files Scan of all .qmd files under quarto/ found 43 genuine prose errors in 26 files. Changes by category: Duplicate words removed: - alternatives/makie_plotting.qmd: 'can can' -> 'can'; 'the the' -> 'the' - basics/vectors.qmd: 'the the' -> 'the'; 'which which' -> 'which' - derivatives/condition.qmd: 'the the' -> 'the' - derivatives/derivatives.qmd: 'At at' -> 'At' - derivatives/lhospitals_rule.qmd: 'the the' -> 'the' - derivatives/mean_value_theorem.qmd: 'the the' -> 'the' - derivatives/more_zeros.qmd: 'the the' -> 'the' - differentiable_vector_calculus/matrix_calculus_notes.qmd: 3 instances - differentiable_vector_calculus/plots_plotting.qmd: 'The the' -> 'The' - differentiable_vector_calculus/vector_fields.qmd: 'the the'; 'a a' -> 'a' - differentiable_vector_calculus/vector_valued_functions.qmd: 'the the' - differentiable_vector_calculus/vectors.qmd: 'the the' - integral_vector_calculus/div_grad_curl.qmd: 'the the' - integral_vector_calculus/double_triple_integrals.qmd: 'The the'; 'over over' - integral_vector_calculus/line_integrals.qmd: 'the the' - integral_vector_calculus/review.qmd: 2x 'the the' - integral_vector_calculus/stokes_theorem.qmd: 2x 'the the' - integrals/improper_integrals.qmd: 'the the' - integrals/substitution.qmd: 'that that' -> 'that' - integrals/surface_area.qmd: 'the the' - precalc/functions.qmd: 'that that width' -> 'that the width' Article (a/an) corrections: - basics/calculator.qmd: 'A overview' -> 'An overview' - basics/vectors.qmd: 'A example' -> 'An example'; 'are a implemented' -> 'are implemented' - derivatives/optimization.qmd: 'an trigonometry-free' -> 'a trigonometry-free' - derivatives/taylor_series_polynomials.qmd: 'a error' -> 'an error' - differentiable_vector_calculus/scalar_functions_applications.qmd: 'a optimization' -> 'an optimization' - differentiable_vector_calculus/vectors.qmd: 'a another' -> 'another'; 'a an angle' -> 'an angle' - integral_vector_calculus/double_triple_integrals.qmd: 'a azimuthal' -> 'an azimuthal' - integral_vector_calculus/line_integrals.qmd: 'an current' -> 'a current'; 'an simply' -> 'a simply'; 'an rotational' -> 'a rotational' - limits/intermediate_value_theorem.qmd: 'an local' -> 'a local'; 'an minimum' -> 'a minimum' Other typos: - basics/calculator.qmd: 'is is not' -> 'is not'; 'chicken is an unfamiliar' -> 'chicken in an unfamiliar'; 'but you the oven' -> 'but the oven' - differentiable_vector_calculus/matrix_calculus_notes.qmd: 'symmteric' -> 'symmetric' --- quarto/alternatives/makie_plotting.qmd | 4 ++-- quarto/basics/calculator.qmd | 6 +++--- quarto/basics/vectors.qmd | 8 ++++---- quarto/derivatives/condition.qmd | 2 +- quarto/derivatives/derivatives.qmd | 2 +- quarto/derivatives/lhospitals_rule.qmd | 2 +- quarto/derivatives/mean_value_theorem.qmd | 2 +- quarto/derivatives/more_zeros.qmd | 2 +- quarto/derivatives/optimization.qmd | 2 +- quarto/derivatives/taylor_series_polynomials.qmd | 2 +- .../matrix_calculus_notes.qmd | 6 +++--- quarto/differentiable_vector_calculus/plots_plotting.qmd | 2 +- .../scalar_functions_applications.qmd | 2 +- quarto/differentiable_vector_calculus/vector_fields.qmd | 4 ++-- .../vector_valued_functions.qmd | 2 +- quarto/differentiable_vector_calculus/vectors.qmd | 6 +++--- quarto/integral_vector_calculus/div_grad_curl.qmd | 2 +- .../integral_vector_calculus/double_triple_integrals.qmd | 6 +++--- quarto/integral_vector_calculus/line_integrals.qmd | 8 ++++---- quarto/integral_vector_calculus/review.qmd | 4 ++-- quarto/integral_vector_calculus/stokes_theorem.qmd | 4 ++-- quarto/integrals/improper_integrals.qmd | 2 +- quarto/integrals/substitution.qmd | 2 +- quarto/integrals/surface_area.qmd | 2 +- quarto/limits/intermediate_value_theorem.qmd | 4 ++-- quarto/precalc/functions.qmd | 2 +- 26 files changed, 45 insertions(+), 45 deletions(-) diff --git a/quarto/alternatives/makie_plotting.qmd b/quarto/alternatives/makie_plotting.qmd index 38af02a..4434a49 100644 --- a/quarto/alternatives/makie_plotting.qmd +++ b/quarto/alternatives/makie_plotting.qmd @@ -207,7 +207,7 @@ lines(a..b, f) As with `scatter`, `lines` returns an object that produces a graphic when displayed. -As with `scatter`, `lines` can can also be drawn using a vector of points: +As with `scatter`, `lines` can also be drawn using a vector of points: ```{julia} @@ -978,7 +978,7 @@ arrows(pts, dvs) The rotational pattern becomes much clearer now. -The `streamplot` function also illustrates this phenomenon. This implements an "algorithm [that] puts an arrow somewhere and extends the streamline in both directions from there. Then, it chooses a new position (from the remaining ones), repeating the the exercise until the streamline gets blocked, from which on a new starting point, the process repeats." +The `streamplot` function also illustrates this phenomenon. This implements an "algorithm [that] puts an arrow somewhere and extends the streamline in both directions from there. Then, it chooses a new position (from the remaining ones), repeating the exercise until the streamline gets blocked, from which on a new starting point, the process repeats." The `streamplot` function expects a `Point` not a pair of values, so we adjust `f` slightly and call the function using the pattern `streamplot(g, xs, ys)`: diff --git a/quarto/basics/calculator.qmd b/quarto/basics/calculator.qmd index 75443dc..7297ffb 100644 --- a/quarto/basics/calculator.qmd +++ b/quarto/basics/calculator.qmd @@ -190,7 +190,7 @@ A right triangle has sides $a=11$ and $b=12$. Find the length of the hypotenus ##### Example -A overview of a research paper published in [theconversation.com](https://theconversation.com/earth-harbours-20-000-000-000-000-000-ants-and-they-weigh-more-than-wild-birds-and-mammals-combined-190831) reviews six authors' work on estimating the number of ants currently on earth. This was covered in an +An overview of a research paper published in [theconversation.com](https://theconversation.com/earth-harbours-20-000-000-000-000-000-ants-and-they-weigh-more-than-wild-birds-and-mammals-combined-190831) reviews six authors' work on estimating the number of ants currently on earth. This was covered in an article in the [Washington Post](https://www.washingtonpost.com/climate-environment/2022/09/19/ants-population-20-quadrillion/). @@ -374,7 +374,7 @@ The Google calculator has two built in constants, `e` and `π`. Julia provides t pi ``` -Whereas, `e` is is not simply the character `e`, but *rather* a [Unicode](../unicode.html) character typed in as `\euler[tab]`. +Whereas, `e` is not simply the character `e`, but *rather* a [Unicode](../unicode.html) character typed in as `\euler[tab]`. ```{julia} @@ -797,7 +797,7 @@ numericq(val) ###### Question -You are asked to cook chicken is an unfamiliar kitchen. Your recipe says to turn the oven to 200 Celsius, but you the oven is calibrated in Fahrenheit. Which value is closest? +You are asked to cook chicken in an unfamiliar kitchen. Your recipe says to turn the oven to 200 Celsius, but the oven is calibrated in Fahrenheit. Which value is closest? ```{julia} #| echo: false diff --git a/quarto/basics/vectors.qmd b/quarto/basics/vectors.qmd index dafa9b9..8a5c6d4 100644 --- a/quarto/basics/vectors.qmd +++ b/quarto/basics/vectors.qmd @@ -102,7 +102,7 @@ Don't spend time thinking about the formulas if they are unfamiliar. The point e Initially, our primary use of vectors will be as containers, but it is worthwhile to spend some time to discuss properties of vectors and their visualization. -A line segment in the plane connects two points $(x_0, y_0)$ and $(x_1, y_1)$. The length of a line segment (its magnitude) is given by the distance formula $\sqrt{(x_1 - x_0)^2 + (y_1 - y_0)^2}$. A line segment can be given a direction by assigning an initial point and a terminal point. A directed line segment has both a direction and a magnitude. A vector is an abstraction where just these two properties $-$ a **direction** and a **magnitude** $-$ are intrinsic. While a directed line segment can be represented by a vector, a single vector describes all such line segments found by translation. That is, how the the vector is located when visualized is for convenience, it is not a characteristic of the vector. In the figure above, all vectors are drawn with their tails at the position of the projectile over time. +A line segment in the plane connects two points $(x_0, y_0)$ and $(x_1, y_1)$. The length of a line segment (its magnitude) is given by the distance formula $\sqrt{(x_1 - x_0)^2 + (y_1 - y_0)^2}$. A line segment can be given a direction by assigning an initial point and a terminal point. A directed line segment has both a direction and a magnitude. A vector is an abstraction where just these two properties $-$ a **direction** and a **magnitude** $-$ are intrinsic. While a directed line segment can be represented by a vector, a single vector describes all such line segments found by translation. That is, how the vector is located when visualized is for convenience, it is not a characteristic of the vector. In the figure above, all vectors are drawn with their tails at the position of the projectile over time. We can visualize a (two-dimensional) vector as an arrow in space. This arrow has two components. We represent a vector mathematically as $\langle x,~ y \rangle$. For example, the vector connecting the point $(x_0, y_0)$ to $(x_1, y_1)$ is $\langle x_1 - x_0,~ y_1 - y_0 \rangle$. @@ -419,7 +419,7 @@ These properties may not all be desirable for one reason or the other and `Juli ### Arrays -Vectors are $1$-dimensional, but there are desires for other dimensions. Vectors are a implemented as a special case of a more general array type. Arrays are of dimension $N$ for various non-negative values of $N$. A common, and somewhat familiar, mathematical use of a $2$-dimensional array is a matrix. +Vectors are $1$-dimensional, but there are desires for other dimensions. Vectors are implemented as a special case of a more general array type. Arrays are of dimension $N$ for various non-negative values of $N$. A common, and somewhat familiar, mathematical use of a $2$-dimensional array is a matrix. Arrays can have their entries accessed by dimension and within that dimension their components. By default these are $1$-based, but other offsets are possible through the `OffsetArrays.jl` package. A matrix can refer to its values either by row and column indices or, as a matrix has linear indexing by a single index. @@ -841,7 +841,7 @@ map(sin, xs) The `map` function can be used with one or more iterators. -The `map` function can also be used in combination with `reduce`, a reduction. Reductions take a container with one or more dimensions and reduces the number of dimensions. A example might be: +The `map` function can also be used in combination with `reduce`, a reduction. Reductions take a container with one or more dimensions and reduces the number of dimensions. An example might be: ```{julia} sum(map(sin, xs)) @@ -861,7 +861,7 @@ There are other specialized reduction functions that reverse the order of the ma sum(xs), prod(xs) ``` -These are reductions, which which fall back to a `mapreduce` call. They require a starting value (`init`) of `0` and `1` (which in this case can be determined from `xs`). The `sum` and `prod` function also allow as a first argument an initial function to map over the collection: +These are reductions, which fall back to a `mapreduce` call. They require a starting value (`init`) of `0` and `1` (which in this case can be determined from `xs`). The `sum` and `prod` function also allow as a first argument an initial function to map over the collection: ```{julia} sum(sin, xs) diff --git a/quarto/derivatives/condition.qmd b/quarto/derivatives/condition.qmd index b7ff1b8..2dfc602 100644 --- a/quarto/derivatives/condition.qmd +++ b/quarto/derivatives/condition.qmd @@ -64,7 +64,7 @@ $$ &= 0 + f'(r)\delta + \epsilon \end{align*} $$ -Rearranging gives $\lVert\delta/\epsilon\rVert \approx 1/\lVert f'(r)\rVert$. But the $|\delta|/|\epsilon|$ ratio is related to the the condition number: +Rearranging gives $\lVert\delta/\epsilon\rVert \approx 1/\lVert f'(r)\rVert$. But the $|\delta|/|\epsilon|$ ratio is related to the condition number: > The absolute condition number is $\hat{\kappa}_r = |f'(r)|^{-1}$. diff --git a/quarto/derivatives/derivatives.qmd b/quarto/derivatives/derivatives.qmd index 327aa22..d64b1f0 100644 --- a/quarto/derivatives/derivatives.qmd +++ b/quarto/derivatives/derivatives.qmd @@ -1574,7 +1574,7 @@ radioq(choices, answ) ###### Question -The rate of change of volume with respect to height is $3h$. The rate of change of height with respect to time is $2t$. At at $t=3$ the height is $h=14$ what is the rate of change of volume with respect to time when $t=3$? +The rate of change of volume with respect to height is $3h$. The rate of change of height with respect to time is $2t$. At $t=3$ the height is $h=14$ what is the rate of change of volume with respect to time when $t=3$? ```{julia} diff --git a/quarto/derivatives/lhospitals_rule.qmd b/quarto/derivatives/lhospitals_rule.qmd index 5e9160b..25d5218 100644 --- a/quarto/derivatives/lhospitals_rule.qmd +++ b/quarto/derivatives/lhospitals_rule.qmd @@ -498,7 +498,7 @@ $$ \lim_{x \rightarrow \infty} \frac{1 - \cos(x)}{1}, $$ -as the function just oscillates. This shows that L'Hospital's rule does not apply when the limit of the the ratio of the derivatives does not exist. +as the function just oscillates. This shows that L'Hospital's rule does not apply when the limit of the ratio of the derivatives does not exist. ##### Example: the assumptions matter diff --git a/quarto/derivatives/mean_value_theorem.qmd b/quarto/derivatives/mean_value_theorem.qmd index 554fe1e..3700d1a 100644 --- a/quarto/derivatives/mean_value_theorem.qmd +++ b/quarto/derivatives/mean_value_theorem.qmd @@ -134,7 +134,7 @@ This insight holds for other types of functions: -When the derivative exists, this says the tangent line is flat. (If it had a slope, then the the function would increase by moving left or right, as appropriate, a point we pursue later.) +When the derivative exists, this says the tangent line is flat. (If it had a slope, then the function would increase by moving left or right, as appropriate, a point we pursue later.) For a continuous function $f(x)$, call a point $c$ in the domain of $f$ where either $f'(c)=0$ or the derivative does not exist a **critical** **point**. diff --git a/quarto/derivatives/more_zeros.qmd b/quarto/derivatives/more_zeros.qmd index 14dec55..0644ed3 100644 --- a/quarto/derivatives/more_zeros.qmd +++ b/quarto/derivatives/more_zeros.qmd @@ -461,7 +461,7 @@ $$ &= 0 + f'(r)\delta + \epsilon \end{align*} $$ -Rearranging gives $\lVert\delta/\epsilon\rVert \approx 1/\lVert f'(r)\rVert$. But the $|\delta|/|\epsilon|$ ratio is related to the the condition number: +Rearranging gives $\lVert\delta/\epsilon\rVert \approx 1/\lVert f'(r)\rVert$. But the $|\delta|/|\epsilon|$ ratio is related to the condition number: > The absolute condition number is $\hat{\kappa}_r = |f'(r)|^{-1}$. diff --git a/quarto/derivatives/optimization.qmd b/quarto/derivatives/optimization.qmd index ce7f820..df306c0 100644 --- a/quarto/derivatives/optimization.qmd +++ b/quarto/derivatives/optimization.qmd @@ -1361,7 +1361,7 @@ radioq(choices, answ) ###### Question -The ladder problem has an trigonometry-free solution. We show one attributed to [Asma](http://www.mathematische-basteleien.de/ladder.htm). +The ladder problem has a trigonometry-free solution. We show one attributed to [Asma](http://www.mathematische-basteleien.de/ladder.htm). ```{julia} diff --git a/quarto/derivatives/taylor_series_polynomials.qmd b/quarto/derivatives/taylor_series_polynomials.qmd index 3c3c23d..10f51de 100644 --- a/quarto/derivatives/taylor_series_polynomials.qmd +++ b/quarto/derivatives/taylor_series_polynomials.qmd @@ -1186,7 +1186,7 @@ answ = 1 radioq(choices, answ) ``` -Assuming the above is right, find the smallest value $k$ guaranteeing a error no more than $10^{-16}$. +Assuming the above is right, find the smallest value $k$ guaranteeing an error no more than $10^{-16}$. ```{julia} diff --git a/quarto/differentiable_vector_calculus/matrix_calculus_notes.qmd b/quarto/differentiable_vector_calculus/matrix_calculus_notes.qmd index bee6d93..23e5172 100644 --- a/quarto/differentiable_vector_calculus/matrix_calculus_notes.qmd +++ b/quarto/differentiable_vector_calculus/matrix_calculus_notes.qmd @@ -415,7 +415,7 @@ In calculus, we typically have $n$ and $m$ are $1$, $2$,or $3$. But that need no ## Derivatives of matrix functions -What is the the derivative of $f(A) = A^2$? +What is the derivative of $f(A) = A^2$? The function $f$ takes a $n\times n$ matrix and returns a matrix of the same size. @@ -564,7 +564,7 @@ all(l == r for (l, r) ∈ zip(L, R)) Now to use this relationship to recognize $df = A dA + dA A$ with the Jacobian computed from $\text{vec}(f(a))$. -We have $\text{vec}(A dA + dA A) = \text{vec}(A dA) + \text{vec}(dA A)$, by obvious linearity of $\text{vec}$. Now inserting an identity matrix, $I$, which is symmteric, in a useful spot we have: +We have $\text{vec}(A dA + dA A) = \text{vec}(A dA) + \text{vec}(dA A)$, by obvious linearity of $\text{vec}$. Now inserting an identity matrix, $I$, which is symmetric, in a useful spot we have: $$ \text{vec}(A dA) = \text{vec}(A dA I^T) = (I \otimes A) \text{vec}(dA), @@ -861,7 +861,7 @@ $$ d(f')[dx] = f''(x)[d\tilde{x}][dx] = f''(x)[d\tilde{x}, dx]. $$ -The last equality a definition. As $f''$ is linear in the the application to $d\tilde{x}$ and also linear in application to $dx$, $f''(x)$ is a bilinear operator. +The last equality a definition. As $f''$ is linear in the application to $d\tilde{x}$ and also linear in application to $dx$, $f''(x)$ is a bilinear operator. Moreover, the following shows it is *symmetric*: diff --git a/quarto/differentiable_vector_calculus/plots_plotting.qmd b/quarto/differentiable_vector_calculus/plots_plotting.qmd index 9f9b0b2..b46048e 100644 --- a/quarto/differentiable_vector_calculus/plots_plotting.qmd +++ b/quarto/differentiable_vector_calculus/plots_plotting.qmd @@ -308,7 +308,7 @@ arrow!(p, v) ### The tangent plane -Let $z = f(x,y)$ describe a surface, and $F(x,y,z) = f(x,y) - z$. The the gradient of $F$ at a point $p$ on the surface, $\nabla F(p)$, will be normal to the surface and for a function, $f(p) + \nabla f \cdot (x-p)$ describes the tangent plane. We can visualize each, as follows: +Let $z = f(x,y)$ describe a surface, and $F(x,y,z) = f(x,y) - z$. The gradient of $F$ at a point $p$ on the surface, $\nabla F(p)$, will be normal to the surface and for a function, $f(p) + \nabla f \cdot (x-p)$ describes the tangent plane. We can visualize each, as follows: ```{julia} diff --git a/quarto/differentiable_vector_calculus/scalar_functions_applications.qmd b/quarto/differentiable_vector_calculus/scalar_functions_applications.qmd index d48728e..2b74579 100644 --- a/quarto/differentiable_vector_calculus/scalar_functions_applications.qmd +++ b/quarto/differentiable_vector_calculus/scalar_functions_applications.qmd @@ -1544,7 +1544,7 @@ $$ \int \sqrt{1 + y'(x)^2} dx = L. $$ -The latter being the formula for arc length. This is very much like a optimization problem that Lagrange's method could help solve, but with one big difference: the answer is *not* a point but a *function*. +The latter being the formula for arc length. This is very much like an optimization problem that Lagrange's method could help solve, but with one big difference: the answer is *not* a point but a *function*. This is a variant of [Dido](http://www.ams.org/publications/journals/notices/201709/rnoti-p980.pdf)'s problem, described by Bandle as diff --git a/quarto/differentiable_vector_calculus/vector_fields.qmd b/quarto/differentiable_vector_calculus/vector_fields.qmd index f18e2b1..70136e0 100644 --- a/quarto/differentiable_vector_calculus/vector_fields.qmd +++ b/quarto/differentiable_vector_calculus/vector_fields.qmd @@ -531,7 +531,7 @@ $$ J = [\nabla{f}']. $$ - * For $f:R^2 \rightarrow R$, the Hessian matrix, was the matrix of $2$nd partial derivatives. This may be viewed as the total derivative of the the gradient function, $\nabla{f}$: + * For $f:R^2 \rightarrow R$, the Hessian matrix, was the matrix of $2$nd partial derivatives. This may be viewed as the total derivative of the gradient function, $\nabla{f}$: $$ @@ -1341,7 +1341,7 @@ With this, we get the following possibilities for $f$ with a zero of order $k$ a * If $l$ is odd and $k$ is even and $f^{(k)}(b_0)$ and $f^{(l)}(c_0)$ have *opposite* signs, the $(b_0, c_0)$ is an isolated solution. -* If $l$ is add and $k$ is odd, then there are two continuous solutions, but only defined in a a one-sided neighborhood of $b_0$ where $f^{(k)}(b_0) f^{(l)}(c_0) (b - b_0) > 0$. +* If $l$ is add and $k$ is odd, then there are two continuous solutions, but only defined in a one-sided neighborhood of $b_0$ where $f^{(k)}(b_0) f^{(l)}(c_0) (b - b_0) > 0$. To visualize these four cases, we take $(l=2,k=1)$, $(l=3, k=2)$ (twice) and $(l=3, k=3)$. diff --git a/quarto/differentiable_vector_calculus/vector_valued_functions.qmd b/quarto/differentiable_vector_calculus/vector_valued_functions.qmd index eb20710..cae6217 100644 --- a/quarto/differentiable_vector_calculus/vector_valued_functions.qmd +++ b/quarto/differentiable_vector_calculus/vector_valued_functions.qmd @@ -558,7 +558,7 @@ vvf = [cos(t), sin(t), t] We will see working with these expressions is not identical to working with a vector-valued function. -To plot, we can avail ourselves of the the parametric plot syntax. The following expands to `plot(cos(t), sin(t), t, 0, 2pi)`: +To plot, we can avail ourselves of the parametric plot syntax. The following expands to `plot(cos(t), sin(t), t, 0, 2pi)`: ```{julia} diff --git a/quarto/differentiable_vector_calculus/vectors.qmd b/quarto/differentiable_vector_calculus/vectors.qmd index 8e625c1..e9e0bbb 100644 --- a/quarto/differentiable_vector_calculus/vectors.qmd +++ b/quarto/differentiable_vector_calculus/vectors.qmd @@ -247,7 +247,7 @@ quiver([0],[0], quiver=([1],[2])) The cumbersome syntax, `quiver(x, y, quiver=(u, v))`, is typical here. We naturally describe vectors and points using `[a,b,c]` to combine them, but the plotting functions want to plot many such at a time and expect vectors containing just the `x` values, just the `y` values, etc. The above usage looks a bit odd, as these vectors of `x` and `y` values have only one entry. -Converting from the one representation to the other requires reshaping the data. We will use the `unzip` function from `CalculusWithJulia` which in turn just uses the the `invert` function of the `SplitApplyCombine` package ("return a new nested container by reversing the order of the nested container") for the bulk of its work. +Converting from the one representation to the other requires reshaping the data. We will use the `unzip` function from `CalculusWithJulia` which in turn just uses the `invert` function of the `SplitApplyCombine` package ("return a new nested container by reversing the order of the nested container") for the bulk of its work. This function takes a vector of vectors, and returns a tuple containing the `x` values, the `y` values, etc. So if `u=[1,2,3]` and `v=[4,5,6]`, then `unzip([u,v])` becomes `([1,4],[2,5],[3,6])`, etc. (The `zip` function in base does essentially the reverse operation, hence the name.) Notationally, `A = [u,v]` can have the third element of the first vector (`u`) accessed by `A[1][3]`, where as `unzip(A)[3][1]` will do the same. We use `unzip([u])` in the following, which for this `u` returns `([1],[2],[3])`. (Note the `[u]` to make a vector of a vector.) @@ -679,7 +679,7 @@ But the associative property does not, as $(\vec{u} \cdot \vec{v}) \cdot \vec{w} ### Cross product -In three dimensions, there is a another operation between vectors that is similar to multiplication, though we will see with many differences. +In three dimensions, there is another operation between vectors that is similar to multiplication, though we will see with many differences. Let $\vec{u}$ and $\vec{v}$ be two $3$-dimensional vectors, then the *cross* product, $\vec{u} \times \vec{v}$, is defined as a vector with length: @@ -875,7 +875,7 @@ $$ \|\vec{u} \times \vec{v}\| \| \vec{w}\| \cos(\theta), $$ -that is, the area of the parallelepiped. Wait, what about $(\vec{v}\times\vec{u})\cdot\vec{w}$? That will have an opposite sign. Yes, in the above, there is an assumption that $\vec{n}$ and $\vec{w}$ have a an angle between them within $[0, \pi/2]$, otherwise an absolute value must be used, as volume is non-negative. +that is, the area of the parallelepiped. Wait, what about $(\vec{v}\times\vec{u})\cdot\vec{w}$? That will have an opposite sign. Yes, in the above, there is an assumption that $\vec{n}$ and $\vec{w}$ have an angle between them within $[0, \pi/2]$, otherwise an absolute value must be used, as volume is non-negative. :::{.callout-note} diff --git a/quarto/integral_vector_calculus/div_grad_curl.qmd b/quarto/integral_vector_calculus/div_grad_curl.qmd index 877eadc..1ad0921 100644 --- a/quarto/integral_vector_calculus/div_grad_curl.qmd +++ b/quarto/integral_vector_calculus/div_grad_curl.qmd @@ -1360,7 +1360,7 @@ $$ \nabla \cdot F = \lim \frac{1}{\Delta{V}}\oint_S F \cdot \hat{N} dS. $$ -Taking $V$ as a box in the curvilinear coordinates, with side lengths $h_udu$, $h_vdv$, and $h_wdw$ the surface integral is computed by projecting $F$ onto each normal area element and multiplying by the area. The task is similar to how the the divergence was derived above, only now the terms are like $\partial{(F_uh_vh_w)}/\partial{u}$ due to the scale factors ($F_u$ is the u component of $F$.) The result is: +Taking $V$ as a box in the curvilinear coordinates, with side lengths $h_udu$, $h_vdv$, and $h_wdw$ the surface integral is computed by projecting $F$ onto each normal area element and multiplying by the area. The task is similar to how the divergence was derived above, only now the terms are like $\partial{(F_uh_vh_w)}/\partial{u}$ due to the scale factors ($F_u$ is the u component of $F$.) The result is: $$ diff --git a/quarto/integral_vector_calculus/double_triple_integrals.qmd b/quarto/integral_vector_calculus/double_triple_integrals.qmd index 2252711..a2fe088 100644 --- a/quarto/integral_vector_calculus/double_triple_integrals.qmd +++ b/quarto/integral_vector_calculus/double_triple_integrals.qmd @@ -193,7 +193,7 @@ a1, b1 = 0, 5 # R is area 20, so V = 60 = 3 ⋅ 20 hcubature(f, (a0, a1), (b0, b1)) ``` - * A wedge. Let $f(x,y) = x$ and $R= [0,1] \times [0,1]$. The the volume is a wedge, and should be half the value of the unit cube, or simply $1/2$: + * A wedge. Let $f(x,y) = x$ and $R= [0,1] \times [0,1]$. The volume is a wedge, and should be half the value of the unit cube, or simply $1/2$: ```{julia} @@ -795,7 +795,7 @@ $$ \iint_{(x,y): \phi(x,y) \leq 0} f(x,y) dx dy. $$ -It can also integrate over over boundaries of the form $\phi(x) = 0$. The latter can be visualized through `implicit_plot`. +It can also integrate over boundaries of the form $\phi(x) = 0$. The latter can be visualized through `implicit_plot`. The main function from `ImplicitIntegration` is `integrate`. The package is imported below to avoid naming conflicts with `SymPy`'s `integrate` function: @@ -1762,7 +1762,7 @@ integrate(1 * r, (z, -sqrt(4-r^2), sqrt(4-r^2)), (r, 0, a), (theta,0, 2PI)) #### Spherical integrals -Spherical coordinates describe a point in space by a radius from the origin, $r$ or $\rho$; a azimuthal angle $\theta$ in $[0, 2\pi]$ and an *inclination* angle $\phi$ (also called polar angle) in $[0, \pi]$. The $z$ axis is the direction of the zenith and gives a reference line to define the inclination angle. The $x$-$y$ plane is the reference plane, with the $x$ axis giving a reference direction for the azimuth measurement. +Spherical coordinates describe a point in space by a radius from the origin, $r$ or $\rho$; an azimuthal angle $\theta$ in $[0, 2\pi]$ and an *inclination* angle $\phi$ (also called polar angle) in $[0, \pi]$. The $z$ axis is the direction of the zenith and gives a reference line to define the inclination angle. The $x$-$y$ plane is the reference plane, with the $x$ axis giving a reference direction for the azimuth measurement. The exact formula to relate $(\rho, \theta, \phi)$ to $(x,y,z)$ is given by diff --git a/quarto/integral_vector_calculus/line_integrals.qmd b/quarto/integral_vector_calculus/line_integrals.qmd index 06afe25..b901a7c 100644 --- a/quarto/integral_vector_calculus/line_integrals.qmd +++ b/quarto/integral_vector_calculus/line_integrals.qmd @@ -152,7 +152,7 @@ $$ which is $1$ on the path $C$. So $\int_C B\cdot\hat{T} ds = \int_C ds = 2\pi$. So the current satisfies $2\pi = \mu_0 I$, so $I = (2\pi)/\mu_0$. -(Ampere's law is more typically used to find $B$ from an current, then $I$ from $B$, for special circumstances. The Biot-Savart does this more generally.) +(Ampere's law is more typically used to find $B$ from a current, then $I$ from $B$, for special circumstances. The Biot-Savart does this more generally.) ### Line integrals and vector fields; work and flow @@ -355,7 +355,7 @@ The fact that work in a potential field is path independent is a consequence of ::: {.callout-note icon=false} ## The Fundamental Theorem of Line [Integrals](https://en.wikipedia.org/wiki/Gradient_theorem): -Let $U$ be an open subset of $R^n$, $f: U \rightarrow R$ a *differentiable* function and $\vec{r}: R \rightarrow R^n$ a differentiable function such that the the path $C = \vec{r}(t)$, $a\leq t\leq b$ is contained in $U$. Then +Let $U$ be an open subset of $R^n$, $f: U \rightarrow R$ a *differentiable* function and $\vec{r}: R \rightarrow R^n$ a differentiable function such that the path $C = \vec{r}(t)$, $a\leq t\leq b$ is contained in $U$. Then $$ \int_C \nabla{f} \cdot d\vec{r} = @@ -430,7 +430,7 @@ quadgk(t -> Radial(r₂(t)) ⋅ r₂'(t), 0, pi) Not all vector fields are conservative. How can a vector field in $U$ be identified as conservative? For now, this would require either finding a scalar potential *or* showing all line integrals are path independent. -In dimension $2$ there is an easy to check method assuming $U$ is *simply connected*: If $F=\langle F_x, F_y\rangle$ is continuously differentiable in an simply connected region *and* $\partial{F_y}/\partial{x} - \partial{F_x}/\partial{y} = 0$ then $F$ is conservative. A similarly statement is available in dimension $3$. The reasoning behind this will come from the upcoming Green's theorem. +In dimension $2$ there is an easy to check method assuming $U$ is *simply connected*: If $F=\langle F_x, F_y\rangle$ is continuously differentiable in a simply connected region *and* $\partial{F_y}/\partial{x} - \partial{F_x}/\partial{y} = 0$ then $F$ is conservative. A similarly statement is available in dimension $3$. The reasoning behind this will come from the upcoming Green's theorem. ### Flow across a curve @@ -521,7 +521,7 @@ quadgk(G, a, b)[1] ##### Example -Example, let $F(x,y) = \langle -y, x\rangle$ be a vector field. (It represents an rotational flow.) What is the flow across the unit circle? +Example, let $F(x,y) = \langle -y, x\rangle$ be a vector field. (It represents a rotational flow.) What is the flow across the unit circle? ```{julia} diff --git a/quarto/integral_vector_calculus/review.qmd b/quarto/integral_vector_calculus/review.qmd index b7a538d..a95e9dc 100644 --- a/quarto/integral_vector_calculus/review.qmd +++ b/quarto/integral_vector_calculus/review.qmd @@ -299,7 +299,7 @@ The gradient is then scalar "multiplication" on the left: $\nabla{f}$. The divergence is the dot product on the left: $\nabla\cdot{F}$. -The curl is the the cross product on the left: $\nabla\times{F}$. +The curl is the cross product on the left: $\nabla\times{F}$. These operations satisfy two vanishing properties: @@ -393,7 +393,7 @@ To compute integrals over non-box-like regions, Fubini's theorem may be utilized For a parameterized curve, $\vec{r}(t)$, the **line integral** of a scalar function between $a \leq t \leq b$ is defined by: $\int_a^b f(\vec{r}(t)) \| \vec{r}'(t)\| dt$. For a path parameterized by arc-length, the integral is expressed by $\int_C f(\vec{r}(s)) ds$ or simply $\int_C f ds$, as the norm is $1$ and $C$ expresses the path. -A Jordan curve in two dimensions is a non-intersecting continuous loop in the plane. The Jordan curve theorem states that such a curve divides the plane into a bounded and unbounded region. The curve is *positively* parameterized if the the bounded region is kept on the left. A line integral over a Jordan curve is denoted $\oint_C f ds$. +A Jordan curve in two dimensions is a non-intersecting continuous loop in the plane. The Jordan curve theorem states that such a curve divides the plane into a bounded and unbounded region. The curve is *positively* parameterized if the bounded region is kept on the left. A line integral over a Jordan curve is denoted $\oint_C f ds$. Some interpretations: $\int_a^b \| \vec{r}'(t)\| dt$ computes the *arc-length*. If the path represents a wire with density $\rho(\vec{x})$ then $\int_a^b \rho(\vec{r}(t)) \|\vec{r}'(t)\| dt$ computes the mass of the wire. diff --git a/quarto/integral_vector_calculus/stokes_theorem.qmd b/quarto/integral_vector_calculus/stokes_theorem.qmd index 0ed531e..fd0e8dc 100644 --- a/quarto/integral_vector_calculus/stokes_theorem.qmd +++ b/quarto/integral_vector_calculus/stokes_theorem.qmd @@ -89,7 +89,7 @@ In an abstract setting, Stokes' theorem says exactly this with the relationship The related functions will involve the divergence and the curl, previously discussed. -Many of the the examples in this section come from either [Strang](https://ocw.mit.edu/resources/res-18-001-calculus-online-textbook-spring-2005/) or [Schey](https://www.amazon.com/Div-Grad-Curl-All-That/dp/0393925161/). +Many of the examples in this section come from either [Strang](https://ocw.mit.edu/resources/res-18-001-calculus-online-textbook-spring-2005/) or [Schey](https://www.amazon.com/Div-Grad-Curl-All-That/dp/0393925161/). To make the abstract concrete, consider the one dimensional case of finding the definite integral $\int_a^b F'(x) dx$. The Riemann sum picture at the *microscopic* level considers a figure like: @@ -419,7 +419,7 @@ annotate!(p, [(a,del, "a"), (b,-del,"b")]) p ``` -Let $A$ label the red line, $B$ the green curve, $C$ the blue line, and $D$ the black line. Then the area is given from Green's theorem by considering half of the the line integral of $F(x,y) = \langle -y, x\rangle$ or $\oint_C (xdy - ydx)$. To that matter we have: +Let $A$ label the red line, $B$ the green curve, $C$ the blue line, and $D$ the black line. Then the area is given from Green's theorem by considering half of the line integral of $F(x,y) = \langle -y, x\rangle$ or $\oint_C (xdy - ydx)$. To that matter we have: $$ diff --git a/quarto/integrals/improper_integrals.qmd b/quarto/integrals/improper_integrals.qmd index 0cf16b3..8194e52 100644 --- a/quarto/integrals/improper_integrals.qmd +++ b/quarto/integrals/improper_integrals.qmd @@ -621,7 +621,7 @@ numericq(val) ###### Question -Compute the the integral $\int_1^\infty \log(x)/x^2 dx$. +Compute the integral $\int_1^\infty \log(x)/x^2 dx$. ```{julia} diff --git a/quarto/integrals/substitution.qmd b/quarto/integrals/substitution.qmd index 48a5e31..04f967e 100644 --- a/quarto/integrals/substitution.qmd +++ b/quarto/integrals/substitution.qmd @@ -585,7 +585,7 @@ $$ $$ -Keeping in mind that that a circle with radius $a$ is an ellipse with $b=a$, we see that this gives the correct answer for a circle. +Keeping in mind that a circle with radius $a$ is an ellipse with $b=a$, we see that this gives the correct answer for a circle. ## Questions diff --git a/quarto/integrals/surface_area.qmd b/quarto/integrals/surface_area.qmd index eacde05..e6b8f8c 100644 --- a/quarto/integrals/surface_area.qmd +++ b/quarto/integrals/surface_area.qmd @@ -837,7 +837,7 @@ radioq(choices, answ) ##### Questions -Find the surface area of the dome of sphere generated by rotating the the curve generated by $g(t) = \cos(t)$ and $f(t) = \sin(t)$ for $t$ in $0$ to $\pi/6$. +Find the surface area of the dome of sphere generated by rotating the curve generated by $g(t) = \cos(t)$ and $f(t) = \sin(t)$ for $t$ in $0$ to $\pi/6$. Numerically find the value. diff --git a/quarto/limits/intermediate_value_theorem.qmd b/quarto/limits/intermediate_value_theorem.qmd index 8aa9ab9..c80eb02 100644 --- a/quarto/limits/intermediate_value_theorem.qmd +++ b/quarto/limits/intermediate_value_theorem.qmd @@ -750,7 +750,7 @@ Related but different is the concept of a relative of *local extrema*: ::: {.callout-note icon=false} ## Local maximum, local minimum -A local maxima for $f$ is a value $f(c)$ where $c$ is in **some** *open* interval $I=(a,b)$, $I$ in the domain of $f$, and $f(c)$ is an absolute maxima for $f$ over $I$. Similarly, an local minima for $f$ is a value $f(c)$ where $c$ is in **some** *open* interval $I=(a,b)$, $I$ in the domain of $f$, and $f(x)$ is an absolute minima for $f$ over $I$. +A local maxima for $f$ is a value $f(c)$ where $c$ is in **some** *open* interval $I=(a,b)$, $I$ in the domain of $f$, and $f(c)$ is an absolute maxima for $f$ over $I$. Similarly, a local minima for $f$ is a value $f(c)$ where $c$ is in **some** *open* interval $I=(a,b)$, $I$ in the domain of $f$, and $f(x)$ is an absolute minima for $f$ over $I$. The term *local extrema* is used to describe either a local maximum or local minimum. @@ -883,7 +883,7 @@ plot(x -> x * exp(-x), 0, 5) ##### Example -The tangent function does not have a *guarantee* of an absolute maximum or an minimum over $(-\pi/2, \pi/2),$ as it is not *continuous* at the endpoints. In fact, it doesn't have either extrema - it has vertical asymptotes at each endpoint of this interval. +The tangent function does not have a *guarantee* of an absolute maximum or a minimum over $(-\pi/2, \pi/2),$ as it is not *continuous* at the endpoints. In fact, it doesn't have either extrema - it has vertical asymptotes at each endpoint of this interval. ##### Example diff --git a/quarto/precalc/functions.qmd b/quarto/precalc/functions.qmd index c647dbd..d27aac2 100644 --- a/quarto/precalc/functions.qmd +++ b/quarto/precalc/functions.qmd @@ -835,7 +835,7 @@ For example, jumping ahead a bit, the `plot` function of `Plots` expects functi plot(Area, 0, 10) ``` -From the graph, we can see that that width for maximum area is $w=5$ and so $h=5$ as well. +From the graph, we can see that the width for maximum area is $w=5$ and so $h=5$ as well. From ae461659e0392df11b9125f7e1ab3bf8372c8ea2 Mon Sep 17 00:00:00 2001 From: jverzani Date: Wed, 3 Jun 2026 20:46:39 -0400 Subject: [PATCH 3/7] claude typos; plotly+scatter3d issue; google analytics --- quarto/.gitignore | 3 +- quarto/_quarto.yml | 5 +- quarto/alternatives.qmd | 2 - quarto/alternatives/symbolics.qmd | 23 +- ...functions_applications-backup-question.qmd | 1433 +++++++++++++++++ .../scalar_functions_applications.qmd | 48 +- .../differentiable_vector_calculus/test.html | 641 ++++++++ quarto/differentiable_vector_calculus/test.jl | 827 ++++++++++ .../differentiable_vector_calculus/test.qmd | 98 ++ .../vector_fields.qmd | 4 +- 10 files changed, 3042 insertions(+), 42 deletions(-) create mode 100644 quarto/differentiable_vector_calculus/scalar_functions_applications-backup-question.qmd create mode 100644 quarto/differentiable_vector_calculus/test.html create mode 100644 quarto/differentiable_vector_calculus/test.jl create mode 100644 quarto/differentiable_vector_calculus/test.qmd diff --git a/quarto/.gitignore b/quarto/.gitignore index b19a2da..ea9ed1f 100644 --- a/quarto/.gitignore +++ b/quarto/.gitignore @@ -5,4 +5,5 @@ /*/*.ipynb/ /*/bonepile.qmd /*/references.bib -weave_support.jl \ No newline at end of file +weave_support.jl +**/*.quarto_ipynb diff --git a/quarto/_quarto.yml b/quarto/_quarto.yml index 43b8ecb..2535ef9 100644 --- a/quarto/_quarto.yml +++ b/quarto/_quarto.yml @@ -1,4 +1,4 @@ -version: "0.25" +version: "0.27" engines: ['julia'] project: @@ -10,6 +10,7 @@ comments: book: title: "Calculus with Julia" author: "John Verzani" + google-analytics: "G-LXFE6MTM4M" date: now search: true repo-url: https://github.com/jverzani/CalculusWithJuliaNotes.jl @@ -117,7 +118,7 @@ book: - part: alternatives.qmd chapters: - - alternatives/giac.qmd + #- alternatives/giac.qmd - alternatives/symbolics.qmd - alternatives/SciML.qmd #- alternatives/interval_arithmetic.qmd diff --git a/quarto/alternatives.qmd b/quarto/alternatives.qmd index 862daed..d59ba08 100644 --- a/quarto/alternatives.qmd +++ b/quarto/alternatives.qmd @@ -2,8 +2,6 @@ These notes use a particular selection of packages. This selection could have been different. For example: -* The symbolic math is provided by `SymPy`. [Giac](./alternatives/giac.html) and -[Symbolics](./alternatives/symbolics.html) (along with `SymbolicUtils` and `ModelingToolkit`) provide alternatives. * The finding of zeros of scalar-valued, univariate functions is done with `Roots`. The [NonlinearSolve](./alternatives/SciML.html#nonlinearsolve) package provides an alternative for univariate and multi-variate functions. diff --git a/quarto/alternatives/symbolics.qmd b/quarto/alternatives/symbolics.qmd index aec506f..cac5003 100644 --- a/quarto/alternatives/symbolics.qmd +++ b/quarto/alternatives/symbolics.qmd @@ -1,6 +1,8 @@ # Symbolics.jl +XXX This needs updating! XXX +XXX add https://docs.sciml.ai/SymbolicIntegration/stable/ XXX There are a few options in `Julia` for symbolic math, for example, the `SymPy` package which wraps a Python library. This section describes a collection of native `Julia` packages providing many features of symbolic math. @@ -238,27 +240,6 @@ w = x^3 + y^3 - 2z^3 substitute(w, Dict(x=>2, y=>3)) ``` -The `fold` argument can be passed `false` to inhibit evaluation of values. Compare: - - -```{julia} -ex = 1 + sqrt(x) -substitute(ex, x=>2), substitute(ex, x=>2, fold=false) -``` - -Or - - -```{julia} -ex = sin(x) -substitute(ex, x=>π), substitute(ex, x=>π, fold=false) -``` - -For the latter, it is more efficient to directly use `Term`, which creates the symbolic expression representing the calling of `sin(π)`: - -```{julia} -Symbolics.Term(sin, [π]) -``` ### Simplify diff --git a/quarto/differentiable_vector_calculus/scalar_functions_applications-backup-question.qmd b/quarto/differentiable_vector_calculus/scalar_functions_applications-backup-question.qmd new file mode 100644 index 0000000..d7f0c5b --- /dev/null +++ b/quarto/differentiable_vector_calculus/scalar_functions_applications-backup-question.qmd @@ -0,0 +1,1433 @@ + + +##### Example: Steiner's problem + + +This is from [Strang](https://ocw.mit.edu/resources/res-18-001-calculus-online-textbook-spring-2005/textbook/MITRES_18_001_strang_13.pdf) p 506. + + +We have three points in the plane, $(x_1, y_1)$, $(x_2, y_2)$, and $(x_3,y_3)$. A point $p=(p_x, p_y)$ will have $3$ distances $d_1$, $d_2$, and $d_3$. Broadly speaking we want to minimize to find the point $p$ "nearest" the three fixed points within the triangle. Locating a facility so that it can service $3$ separate cities might be one application. The answer depends on the notion of what measure of distance to use. + + +If the measure is the Euclidean distance, then $d_i^2 = (p_x - x_i)^2 + (p_y - y_i)^2$. If we sought to minimize $d_1^2 + d_2^2 + d_3^2$, then we would proceed as follows: + + +```{julia} +@syms x1 y1 x2 y2 x3 y3 +d2(p,x) = (p[1] - x[1])^2 + (p[2]-x[2])^2 +d2_1, d2_2, d2_3 = d2((x,y), (x1, y1)), d2((x,y), (x2, y2)), d2((x,y), (x3, y3)) +exₛ = d2_1 + d2_2 + d2_3 +``` + +We then find the gradient, and solve for when it is $\vec{0}$: + + +```{julia} +gradfₛ = diff.(exₛ, [x,y]) +xstarₛ = solve(gradfₛ, [x,y]) +``` + +There is only one critical point, so must be a minimum. + + +We confirm this by looking at the Hessian and noting $H_{11} > 0$: + + +```{julia} +Hₛ = subs.(hessian(exₛ, [x,y]), x=>xstarₛ[x], y=>xstarₛ[y]) +``` + +As it occurs at $(\bar{x}, \bar{y})$ where $\bar{x} = (x_1 + x_2 + x_3)/3$ and $\bar{y} = (y_1+y_2+y_3)/3$---the averages of the three values---the critical point is an interior point of the triangle. + + +As mentioned by Strang, the real problem is to minimize $d_1 + d_2 + d_3$. A direct approach with `SymPy`---just replacing `d2` above with the square root fails. Consider instead the gradient of $d_1$, say. To avoid square roots, this is taken implicitly from $d_1^2$: + + +$$ +\frac{\partial}{\partial{x}}(d_1^2) = 2 d_1 \frac{\partial{d_1}}{\partial{x}}. +$$ + +But computing directly from the expression yields $2(x - x_1)$ Solving, yields: + + +$$ +\frac{\partial{d_1}}{\partial{x}} = \frac{(x-x_1)}{d_1}, \quad +\frac{\partial{d_1}}{\partial{y}} = \frac{(y-y_1)}{d_1}. +$$ + +The gradient is then $(\vec{p} - \vec{x}_1)/\|\vec{p} - \vec{x}_1\|$, a *unit* vector, call it $\hat{u}_1$. Similarly for $\hat{u}_2$ and $\hat{u}_3$. + + +Let $f = d_1 + d_2 + d_3$. Then $\nabla{f} = \hat{u}_1 + \hat{u}_2 + \hat{u}_3$. At the minimum, the gradient is $\vec{0}$, so the three unit vectors must cancel. This can only happen if the three make a "peace" sign with angles $120^\circ$ between them. To find the minimum then within the triangle, this point and the boundary must be considered, when this point falls outside the triangle. + + +Here is a triangle, where the minimum would be within the triangle: + + +```{julia} +usₛ = [[cos(t), sin(t)] for t in (0, 2pi/3, 4pi/3)] +polygon(ps) = unzip(vcat(ps, ps[1:1])) # easier way to plot a polygon + +pₛ = scatter([0],[0], markersize=2, legend=false, aspect_ratio=:equal) + +asₛ = (1,2,3) +plot!(polygon([a*u for (a,u) in zip(asₛ, usₛ)])...) +[arrow!([0,0], a*u, alpha=0.5) for (a,u) in zip(asₛ, usₛ)] +pₛ +``` + +For this triangle we find the Steiner point outside of the triangle. + + +```{julia} +asₛ₁ = (1, -1, 3) +scatter([0],[0], markersize=2, legend=false) +psₛₗ = [a*u for (a,u) in zip(asₛ₁, usₛ)] +plot!(polygon(psₛₗ)...) +``` + +Let's see where the minimum distance point is by constructing a plot. The minimum must be on the boundary, as the only point where the gradient vanishes is the origin, not in the triangle. The plot of the triangle has a contour plot of the distance function, so we see clearly that the minimum happens at the point `[0.5, -0.866025]`. On this plot, we drew the gradient at some points along the boundary. The gradient points in the direction of greatest increase---away from the minimum. That the gradient vectors have a non-zero projection onto the edges of the triangle in a direction pointing away from the point indicates that the function `d` would increase if moved along the boundary in that direction, as indeed it does. + + +```{julia} +euclid_dist(x; ps=psₛₗ) = sum(norm(x-p) for p in ps) +euclid_dist(x,y; ps=psₛₗ) = euclid_dist([x,y]; ps=ps) +``` + +```{julia} +#| hold: true +xs = range(-1.5, 1.5, length=100) +ys = range(-3, 1.0, length=100) + +p = plot(polygon(psₛₗ)..., linewidth=3, legend=false) +scatter!(p, unzip(psₛₗ)..., markersize=3) +contour!(p, xs, ys, euclid_dist) + +# add some gradients along boundary +li(t, p1, p2) = p1 + t*(p2-p1) # t in [0,1] +for t in range(1/100, 1/2, length=3) + pt = li(t, psₛₗ[2], psₛₗ[3]) + arrow!(pt, ForwardDiff.gradient(euclid_dist, pt)) + pt = li(t, psₛₗ[2], psₛₗ[1]) + arrow!(pt, ForwardDiff.gradient(euclid_dist, pt)) +end + +p +``` + +The following graph, shows distance along each edge: + + +```{julia} +#| hold : true +li(t, p1, p2) = p1 + t*(p2-p1) +p = plot(legend=false) +for i in 1:2, j in (i+1):3 + plot!(p, t -> euclid_dist(li(t, psₛₗ[i], psₛₗ[j]); ps=psₛₗ), 0, 1) +end +p +``` + +The smallest value is when $t=0$ or $t=1$, so at one of the points, as `li` is defined above. + + +##### Example: least squares + + +We know that two points determine a line. What happens when there are more than two points? This is common in statistics where a bivariate data set (pairs of points $(x,y)$) are summarized through a linear model $\mu_{y|x} = \alpha + \beta x$, That is the average value for $y$ given a particular $x$ value is given through the equation of a line. The data is used to identify what the slope and intercept are for this line. We consider a simple case---$3$ points. The case of $n \geq 3$ being similar. + + +We have a line $l(x) = \alpha + \beta(x)$ and three points $(x_1, y_1)$, $(x_2, y_2)$, and $(x_3, y_3)$. Unless these three points *happen* to be collinear, they can't possibly all lie on the same line. So to *approximate* a relationship by a line requires some inexactness. One measure of inexactness is the *vertical* distance to the line: + + +$$ +d1(\alpha, \beta) = |y_1 - l(x_1)| + |y_2 - l(x_2)| + |y_3 - l(x_3)|. +$$ + +Another might be the vertical squared distance to the line: + +$$ +\begin{align*} +d2(\alpha, \beta) &= (y_1 - l(x_1))^2 + (y_2 - l(x_2))^2 + (y_3 - l(x_3))^2 \\ +&= (y1 - (\alpha + \beta x_1))^2 + (y2 - (\alpha + \beta x_2))^2 + (y3 - (\alpha + \beta x_3))^2 +\end{align*} +$$ + +Another might be the *shortest* distance to the line: + + +$$ +d3(\alpha, \beta) = \frac{\beta x_1 - y_1 + \alpha}{\sqrt{1 + \beta^2}} + \frac{\beta x_2 - y_2 + \alpha}{\sqrt{1 + \beta^2}} + \frac{\beta x_3 - y_3 + \alpha}{\sqrt{1 + \beta^2}}. +$$ + +The method of least squares minimizes the second one of these. That is, it chooses $\alpha$ and $\beta$ that make the expression a minimum. + + +```{julia} +@syms xₗₛ[1:3] yₗₛ[1:3] α β +li(x, alpha, beta) = alpha + beta * x +d₂(alpha, beta) = sum((y - li(x, alpha, beta))^2 for (y,x) in zip(yₗₛ, xₗₛ)) +d₂(α, β) +``` + +To identify $\alpha$ and $\beta$ we find the gradient: + + +```{julia} +grad_d₂ = diff.(d₂(α, β), [α, β]) +``` + +```{julia} +outₗₛ = solve(grad_d₂, [α, β]) +``` + +As found, the formulas aren't pretty. If $x_1 + x_2 + x_3 = 0$ they simplify. For example: + + +```{julia} +subs(outₗₛ[β], sum(xₗₛ) => 0) +``` + +Let $\vec{x} = \langle x_1, x_2, x_3 \rangle$ and $\vec{y} = \langle y_1, y_2, y_3 \rangle$ this is simply $(\vec{x} \cdot \vec{y})/(\vec{x}\cdot \vec{x})$, a formula that will generalize to $n > 3$. The assumption is not a restriction---it comes about by subtracting the mean, $\bar{x} = (x_1 + x_2 + x_3)/3$, from each $x$ term (and similarly subtract $\bar{y}$ from each $y$ term). A process called "centering." + + +With this observation, the formulas can be re-expressed through: + + +$$ +\beta = \frac{\sum{(x_i - \bar{x})(y_i - \bar{y})}}{\sum(x_i-\bar{x})^2}, +\quad +\alpha = \bar{y} - \beta \bar{x}. +$$ + +Relative to the centered values, this may be viewed as a line through $(\bar{x}, \bar{y})$ with slope given by $(\vec{x}-\bar{x})\cdot(\vec{y}-\bar{y}) / \|\vec{x}-\bar{x}\|^2$. + + +As an example, if the point are $(1,1), (2,3), (5,8)$ we get: + + +```{julia} +[k => subs(v, xₗₛ[1]=>1, yₗₛ[1]=>1, xₗₛ[2]=>2, yₗₛ[2]=>3, + xₗₛ[3]=>5, yₗₛ[3]=>8) for (k,v) in outₗₛ] +``` + +### Gradient descent + + +As seen in the examples above, extrema may be identified analytically by solving for when the gradient is $0$. Here we discuss some numeric algorithms for finding extrema. + + +An algorithm to identify where a surface is at its minimum is [gradient descent](https://en.wikipedia.org/wiki/Gradient_descent). The gradient points in the direction of the steepest ascent of the surface and the negative gradient the direction of the steepest descent. To move to a minimum then, it make intuitive sense to move in the direction of the negative gradient. How far? That is a different question and one with different answers. Let's formulate the movement first, then discuss how far. + + +Let $\vec{x}_0$, $\vec{x}_1$, $\dots$, $\vec{x}_n$ be the position of the algorithm for $n$ steps starting from an initial point $\vec{x}_0$. The difference between these points is given by: + + +$$ +\vec{x}_{n+1} = \vec{x}_n - \gamma \nabla{f}(\vec{x}_n), +$$ + +where $\gamma$ is some scaling factor for the gradient. The above quantifies the idea: to go from $\vec{x}_n$ to $\vec{x}_{n+1}$, move along $-\nabla{f}$ by a certain amount. + + +Let $\Delta_x =\vec{x}_{n}- \vec{x}_{n-1}$ and $\Delta_y = \nabla{f}(\vec{x}_{n}) - \nabla{f}(\vec{x}_{n-1})$ A variant of the Barzilai-Borwein method is to take $\gamma_n = | \Delta_x \cdot \Delta_y / \Delta_y \cdot \Delta_y |$. + + +To illustrate, take $f(x,y) = - e^{-((x-1)^2 + 2(y-1/2)^2)}$ and a starting point $\langle 0, 0 \rangle$. We have, starting with $\gamma_0 = 1$ there are $5$ steps taken: + + +```{julia} +f₂(x,y) = -exp(-((x-1)^2 + 2(y-1/2)^2)) +f₂(x) = f₂(x...) + +xs₂ = [[0.0, 0.0]] # we store a vector +gammas₂ = [1.0] + +for n in 1:5 + xn = xs₂[end] + gamma₀ = gammas₂[end] + xn1 = xn - gamma₀ * gradient(f₂)(xn) + dx, dy = xn1 - xn, gradient(f₂)(xn1) - gradient(f₂)(xn) + gamman1 = abs( (dx ⋅ dy) / (dy ⋅ dy) ) + + push!(xs₂, xn1) + push!(gammas₂, gamman1) +end + +[(x, f₂(x)) for x in xs₂] +``` + +We now visualize, using the `Contour` package to draw the contour lines in the $x-y$ plane: + + +```{julia} +#| hold: true +function surface_contour(xs, ys, f; offset=0) + p = surface(xs, ys, f, legend=false, fillalpha=0.5) + + ## we add to the graphic p, then plot + zs = [f(x,y) for x in xs, y in ys] # reverse order for use with Contour package + for cl in levels(contours(xs, ys, zs)) + lvl = level(cl) # the z-value of this contour level + for line in lines(cl) + _xs, _ys = coordinates(line) # coordinates of this line segment + _zs = offset * _xs + plot!(p, _xs, _ys, _zs, alpha=0.5) # add curve on x-y plane + end + end + p +end + + +offset = 0 +us = vs = range(-1, 2, length=100) +surface_contour(us, vs, f₂, offset=offset) +pts = [[pt..., offset] for pt in xs₂] +scatter3d!(unzip(pts)...) +plot!(unzip(pts)..., linewidth=3) +``` + +### Newton's method for minimization + + +A variant of Newton's method can be used to minimize a function $f:R^2 \rightarrow R$. We look for points where both partial derivatives of $f$ vanish. Let $g(x,y) = \partial f/\partial x(x,y)$ and $h(x,y) = \partial f/\partial y(x,y)$. Then applying Newton's method, as above to solve simultaneously for when $g=0$ and $h=0$, we considered this matrix: + + +$$ +M = [\nabla{g}'; \nabla{h}'], +$$ + +and had a step expressible in terms of the inverse of $M$ as $M^{-1} [g; h]$. In terms of the function $f$, this step is $H^{-1}\nabla{f}$, where $H$ is the Hessian matrix. [Newton](https://en.wikipedia.org/wiki/Newton%27s_method_in_optimization#Higher_dimensions)'s method then becomes: + + +$$ +\vec{x}_{n+1} = \vec{x}_n - [H_f(\vec{x}_n)]^{-1} \nabla(f)(\vec{x}_n). +$$ + +The Wikipedia page states where applicable, Newton's method converges much faster towards a local maximum or minimum than gradient descent. + + +We apply it to the task of characterizing the following function, which has a few different peaks over the region $[-3,3] \times [-2,2]$: + + +```{julia} +function peaks(x, y) + z = 3 * (1 - x)^2 * exp(-x^2 - (y + 1)^2) + z += -10 * (x / 5 - x^3 - y^5) * exp(-x^2 - y^2) + z += -1/3 * exp(-(x+1)^2 - y^2) + return z +end +peaks(v) = peaks(v...) +``` + +```{julia} +#| hold: true +xs = range(-3, stop=3, length=100) +ys = range(-2, stop=2, length=100) +Ps = surface(xs, ys, peaks, legend=false) +Pc = contour(xs, ys, peaks, legend=false) +plot(Ps, Pc, layout=2) # combine plots +``` + +As we will solve for the critical points numerically, we consider the contour plot as well, as it shows better where the critical points are. + + +Over this region we see clearly 5 peaks or valleys: near $(0, 1.5)$, near $(1.2, 0)$, near $(0.2, -1.8)$, near $(-0.5, -0.8)$, and near $(-1.2, 0.2)$. To classify the $5$ critical points we need to first identify them, then compute the Hessian, and then, possibly compute $f_{xx}$ at the point. Here we do so for one of them using a numeric approach. + + +For concreteness, consider the peak or valley near $(0,1.5)$. We use Newton's method to numerically compute the critical point. The Newton step, specialized here is: + + +```{julia} +function newton_stepₚ(f, x) + M = ForwardDiff.hessian(f, x) + b = ForwardDiff.gradient(f, x) + x - M \ b +end +``` + +We perform $3$ steps of Newton's method, and see that it has found a critical point. + + +```{julia} +xₚ = [0, 1.5] +xₚ = newton_stepₚ(peaks, xₚ) +xₚ = newton_stepₚ(peaks, xₚ) +xₚ = newton_stepₚ(peaks, xₚ) +xₚ, ForwardDiff.gradient(peaks, xₚ) +``` + +The Hessian at this point is given by: + + +```{julia} +Hₚ = ForwardDiff.hessian(peaks, xₚ) +``` + +From which we see: + + +```{julia} +#| hold: true +fxx = Hₚ[1,1] +d = det(Hₚ) +fxx, d +``` + +Consequently we have a local maximum at this critical point. + + +:::{.callout-note} +## Note + +::: + +The `Optim.jl` package provides efficient implementations of these two numeric methods, and others. + + +## Constrained optimization, Lagrange multipliers + + +We considered the problem of maximizing a function over a closed region. This maximum is achieved at a critical point *or* a boundary point. Investigating the critical points isn't so difficult and the second partial derivative test can help characterize the points along the way, but characterizing the boundary points usually involves parameterizing the boundary, which is not always so easy. However, if we put this problem into a more general setting a different technique becomes available. + + +The different setting is: maximize $f(x,y)$ subject to the constraint $g(x,y) = k$. The constraint can be used to describe the boundary used previously. + + +Why does this help? The key is something we have seen prior: If $g$ is differentiable, and we take $\nabla{g}$, then it will point at directions *orthogonal* to the level curve $g(x,y) = 0$. (Parameterize the curve, then $(g\circ\vec{r})(t) = 0$ and so the chain rule has $\nabla{g}(\vec{r}(t)) \cdot \vec{r}'(t) = 0$.) For example, consider the function $g(x,y) = x^2 +2y^2 - 1$. The level curve $g(x,y) = 0$ is an ellipse. Here we plot the level curve, along with a few gradient vectors at points satisfying $g(x,y) = 0$: + + +```{julia} +#| hold: true +g(x,y) = x^2 + 2y^2 -1 +g(v) = g(v...) + +xs = range(-3, 3, length=100) +ys = range(-1, 4, length=100) + +p = plot(aspect_ratio=:equal, legend=false) +contour!(xs, ys, g, levels=[0]) + +gi(x) = sqrt(1/2*(1-x^2)) # solve for y in terms of x +pts = [[x, gi(x)] for x in (-3/4, -1/4, 1/4, 3/4)] + +for pt in pts + arrow!(pt, ForwardDiff.gradient(g, pt) ) +end + +p +``` + +From the plot we see the key property that $\nabla g$ is orthogonal to the level curve. + + +Now consider $f(x,y)$, a function we wish to maximize. The gradient points in the direction of *greatest* increase, provided $f$ is smooth. We are interested in the value of this gradient along the level curve of $g$. Consider this figure representing a portion of the level curve, it's tangent, normal, the gradient of $f$, and the contours of $f$: + + +```{julia} +#| hold: true +#| echo: false +r(t) = [cos(t), sin(t)/2] +plot_parametric(pi/12..pi/3, r, legend=false, aspect_ratio=true, linewidth=3) +T(t) = -r'(t) / norm(r'(t)) +No(t) = T'(t) / norm(T'(t)) +t = pi/4 +lambda=1/10 +scatter!(unzip([r(t)])...) +arrow!(r(t), T(t)*lambda) +arrow!(r(t), No(t)* lambda) + +f(x,y)= x^2 + y^2 +f(v) = f(v...) +arrow!(r(t), lambda*ForwardDiff.gradient(f, r(t))) + +xs = range(0.5,1, length=100) +ys = range(0.1, 0.5, length=100) +contour!(xs, ys, f) +``` + +We can identify the tangent, the normal, and subsequently the gradient of $f$. Is the point drawn a maximum of $f$ subject to the constraint $g$? + + +The answer is no, but why? By adding the contours of $f$, we see that moving along the curve from this point will increase or decrease $f$, depending on which direction we move in. As the *gradient* is the direction of greatest increase, we can see that the *projection* of the gradient on the tangent will point in a direction of *increase*. + + +It isn't just because the point picked was chosen to make a pretty picture, and not be a maximum. Rather, the fact that $\nabla{f}$ has a non-trivial projection onto the tangent vector. What does it say if we move the point in the direction of this projection? + + +The gradient points in the direction of greatest increase. If we first move in one component of the gradient we will increase, just not as fast. This is because the directional derivative in the direction of the tangent will be non-zero. In the picture, if we were to move the point to the right along the curve $f(x,y)$ will increase. + + +Now consider this figure at a different point of the figure: + + +```{julia} +#| hold: true +#| echo: false +r(t) = [cos(t), sin(t)/2] +plot_parametric(-pi/6..pi/6,r, legend=false, aspect_ratio=true, linewidth=3) +T(t) = -r'(t) / norm(r'(t)) +No(t) = T'(t) / norm(T'(t)) +t = 0 +lambda=1/10 +scatter!(unzip([r(t)])...) +arrow!(r(t), T(t)*lambda) +arrow!(r(t), No(t)* lambda) + +f(x,y)= x^2 + y^2 +f(v) = f(v...) +arrow!(r(t), lambda*ForwardDiff.gradient(f, r(t))) + +xs = range(0.5,1.5, length=100) +ys = range(-0.5, 0.5, length=100) +contour!(xs, ys, f, levels = [.7, .85, 1, 1.15, 1.3]) +``` + +We can still identify the tangent and normal directions. What is different about this point is that local movement on the constraint curve is also local movement on the contour line of $f$, so $f$ doesn't increase or decrease here, as it would if this point were an extrema along the constraint. The key to seeing this is the contour lines of $f$ are *tangent* to the constraint. The respective gradients are *orthogonal* to their tangent lines, and in dimension $2$, this implies they are parallel to each other. + +::: {.callout-note icon=false} +## The method of Lagrange multipliers + +To optimize $f(x,y)$ subject to a constraint $g(x,y) = k$ we solve for all *simultaneous* solutions to + +$$ +\begin{align*} +\nabla{f}(x,y) &= \lambda \nabla{g}(x,y), \text{and}\\ +g(x,y) &= k. +\end{align*} +$$ + + +These *possible* points are evaluated to see if they are maxima or minima. + +::: + + +The method will not work if $\nabla{g} = \vec{0}$ or if $f$ and $g$ are not differentiable. + + +--- + + +##### Example + + +We consider [again]("../derivatives/optimization.html") the problem of maximizing all rectangles subject to the perimeter being $20$. We have seen this results in a square. This time we use the Lagrange multiplier technique. We have two equations: + + +$$ +A(x,y) = xy, \quad P(x,y) = 2x + 2y = 20. +$$ + +We see $\nabla{A} = \lambda \nabla{P}$, or $\langle y, x \rangle = \lambda \langle 2, 2\rangle$. We see the solution has $x = y$ and from the constraint $x=y = 5$. + + +This is clearly the maximum for this problem, though the Lagrange technique does not imply that, it only identifies possible extrema. + + +##### Example + + +We can reverse the question: what are the ranges for the perimeter when the area is a fixed value of $25$? We have: + + +$$ +P(x,y) = 2x + 2y, \quad A(x,y) = xy = 25. +$$ + +Now we look for $\nabla{P} = \lambda \nabla{A}$ and will get, as the last example, that $\langle 2, 2 \rangle = \lambda \langle y, x\rangle$. So $x=y$ and from the constraint $x=y=5$. + + +However this is *not* the maximum perimeter, but rather the minimal perimeter. The maximum is $\infty$, which comes about in the limit by considering long skinny rectangles. + + +##### Example: A rephrasing + + +An slightly different formulation of the Lagrange method is to combine the equation and the constraint into one equation: + + +$$ +L(x,y,\lambda) = f(x,y) - \lambda (g(x,y) - k). +$$ + +The we have + + +$$ +\begin{align*} +\frac{\partial L}{\partial{x}} &= \frac{\partial{f}}{\partial{x}} - \lambda \frac{\partial{g}}{\partial{x}}\\ +\frac{\partial L}{\partial{y}} &= \frac{\partial{f}}{\partial{y}} - \lambda \frac{\partial{g}}{\partial{y}}\\ +\frac{\partial L}{\partial{\lambda}} &= 0 + (g(x,y) - k). +\end{align*} +$$ + + +But if the Lagrange condition holds, each term is $0$, so Lagrange's method can be seen as solving for point $\nabla{L} = \vec{0}$. The optimization problem in two variables with a constraint becomes a problem of finding and classifying zeros of a function with *three* variables. + + +Apply this to the optimization problem: + + +Find the extrema of $f(x,y) = x^2 - y^2$ subject to the constraint $g(x,y) = x^2 + y^2 = 1$. + + +We have: + + +$$ +L(x, y, \lambda) = f(x,y) - \lambda(g(x,y) - 1) +$$ + +We can solve for $\nabla{L} = \vec{0}$ by hand, but we do so symbolically: + + +```{julia} +@syms lambda +fₗₐ(x, y) = x^2 - y^2 +gₗₐ(x, y) = x^2 + y^2 +Lₗₐ(x, y, lambda) = fₗₐ(x,y) - lambda * (gₗₐ(x,y) - 1) +dsₗₐ = solve(diff.(Lₗₐ(x, y, lambda), [x, y, lambda])) +``` + +This has $4$ easy solutions, here are the values at each point: + + +```{julia} +[fₗₐ(d[x], d[y]) for d in dsₗₐ] +``` + +So $1$ is a maximum value and $-1$ a minimum value. + + +##### Example: Dido's problem + + +Consider a slightly different problem: What shape should a rope (curve) of fixed length make to *maximize* the area between the rope and $x$ axis? + + +Let $L$ be the length of the rope and suppose $y(x)$ describes the curve. Then we wish to + + +$$ +\text{Maximize } \int y(x) dx, \quad\text{subject to } +\int \sqrt{1 + y'(x)^2} dx = L. +$$ + +The latter being the formula for arc length. This is very much like an optimization problem that Lagrange's method could help solve, but with one big difference: the answer is *not* a point but a *function*. + + +This is a variant of [Dido](http://www.ams.org/publications/journals/notices/201709/rnoti-p980.pdf)'s problem, described by Bandle as + + +> *Dido’s problem*: The Roman poet Publius Vergilius Maro (70–19 B.C.) tells in his epic Aeneid the story of queen Dido, the daughter of the Phoenician king of the 9th century B.C. After the assassination of her husband by her brother she fled to a haven near Tunis. There she asked the local leader, Yarb, for as much land as could be enclosed by the hide of a bull. Since the deal seemed very modest, he agreed. Dido cut the hide into narrow strips, tied them together and encircled a large tract of land which became the city of Carthage. Dido faced the following mathematical problem, which is also known as the isoperimetric problem: Find among all curves of given length the one which encloses maximal area. Dido found intuitively the right answer. + + + +The problem as stated above and method of solution follows notes by [Wang](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.368.1522&rep=rep1&type=pdf) though Bandle attributes the ideas back to a 19-year old Lagrange in a letter to Euler. + + +The method of solution will be to *assume* we have the function and then characterize this function in such a way that it can be identified. + + +Following Lagrange, we generalize the problem to the following: maximize $\int_{x_0}^{x_1} f(x, y(x), y'(x)) dx$ subject to a constraint $\int_{x_0}^{x_1} g(x,y(x), y'(x)) dx = K$. Suppose $y(x)$ is a solution. + + +The starting point is a *perturbation*: $\hat{y}(x) = y(x) + \epsilon_1 \eta_1(x) + \epsilon_2 \eta_2(x)$. There are two perturbation terms, were only one term added, then the perturbation may make $\hat{y}$ not satisfy the constraint, the second term is used to ensure the constraint is not violated. If $\hat{y}$ is to be a possible solution to our problem, we would want $\hat{y}(x_0) = \hat{y}(x_1) = 0$, as it does for $y(x)$, so we *assume* $\eta_1$ and $\eta_2$ satisfy this boundary condition. + + +With this notation, and fixing $y$ we can re-express the equations in terms of $\epsilon_1$ and $\epsilon_2$: + + +$$ +\begin{align*} +F(\epsilon_1, \epsilon_2) &= \int f(x, \hat{y}, \hat{y}') dx = +\int f(x, y + \epsilon_1 \eta_1 + \epsilon_2 \eta_2, y' + \epsilon_1 \eta_1' + \epsilon_2 \eta_2') dx,\\ +G(\epsilon_1, \epsilon_2) &= \int g(x, \hat{y}, \hat{y}') dx = +\int g(x, y + \epsilon_1 \eta_1 + \epsilon_2 \eta_2, y' + \epsilon_1 \eta_1' + \epsilon_2 \eta_2') dx. +\end{align*} +$$ + + +Then our problem is restated as: + + +$$ +\text{Maximize } F(\epsilon_1, \epsilon_2) \text{ subject to } +G(\epsilon_1, \epsilon_2) = L. +$$ + +Now, Lagrange's method can be employed. This will be fruitful---even though we know the answer---it being $\epsilon_1 = \epsilon_2 = 0$! + + +Forging ahead, we compute $\nabla{F}$ and $\lambda \nabla{G}$ and set $\epsilon_1 = \epsilon_2 = 0$ where the two are equal. This will lead to a description of $y$ in terms of $y'$. + + +Lagrange's method has: + + +$$ +\frac{\partial{F}}{\partial{\epsilon_1}}(0,0) - \lambda \frac{\partial{G}}{\partial{\epsilon_1}}(0,0) = 0, \text{ and } +\frac{\partial{F}}{\partial{\epsilon_2}}(0,0) - \lambda \frac{\partial{G}}{\partial{\epsilon_2}}(0,0) = 0. +$$ + +Computing just the first one, we have using the chain rule and assuming interchanging the derivative and integral is possible: + + +$$ +\begin{align*} +\frac{\partial{F}}{\partial{\epsilon_1}} +&= \int \frac{\partial}{\partial{\epsilon_1}}( +f(x, y + \epsilon_1 \eta_1 + \epsilon_2 \eta_2, y' + \epsilon_1 \eta_1' + \epsilon_2 \eta_2')) dx\\ +&= \int \left(\frac{\partial{f}}{\partial{y}} \eta_1 + \frac{\partial{f}}{\partial{y'}} \eta_1'\right) dx\quad\quad(\text{from }\nabla{f} \cdot \langle 0, \eta_1, \eta_1'\rangle)\\ +&=\int \eta_1 \left(\frac{\partial{f}}{\partial{y}} - \frac{d}{dx}\frac{\partial{f}}{\partial{y'}}\right) dx. +\end{align*} +$$ + + +The last line by integration by parts: +$\int u'(x) v(x) dx = (u \cdot v)(x)\mid_{x_0}^{x_1} - \int u(x) \frac{d}{dx} v(x) dx = - \int u(x) \frac{d}{dx} v(x) dx$. +The last lines, as $\eta_1 = 0$ at $x_0$ and $x_1$ by assumption. We get: + +$$ +0 = \int \eta_1\left(\frac{\partial{f}}{\partial{y}} - \frac{d}{dx}\frac{\partial{f}}{\partial{y'}}\right). +$$ + +Similarly were $G$ considered, we would find a similar statement. Setting $L(x, y, y') = f(x, y, y') - \lambda g(x, y, y')$, the combination of terms gives: + + +$$ +0 = \int \eta_1\left(\frac{\partial{L}}{\partial{y}} - \frac{d}{dx}\frac{\partial{L}}{\partial{y'}}\right) dx. +$$ + +Since $\eta_1$ is arbitrary save for its boundary conditions, under smoothness conditions on $L$ this will imply the rest of the integrand *must* be $0$. + + +That is, If $y(x)$ is a maximizer of $\int_{x_0}^{x_1} f(x, y, y')dx$ and sufficiently smooth over $[x_0, x_1]$ and $y(x)$ satisfies the constraint $\int_{x_0}^{x_1} g(x, y, y')dx = K$ then there exists a constant $\lambda$ such that $L = f -\lambda g$ will satisfy: + + +$$ +\frac{d}{dx}\frac{\partial{L}}{\partial{y'}} - \frac{\partial{L}}{\partial{y}} = 0. +$$ + +If $\partial{L}/\partial{x} = 0$, this simplifies to the [Beltrami](https://en.wikipedia.org/wiki/Beltrami_identity) identity: + + +$$ +L - y' \frac{\partial{L}}{\partial{y'}} = C.\quad(\text{Beltrami identity}) +$$ + +--- + + +For Dido's problem, $f(x,y,y') = y$ and $g(x, y, y') = \sqrt{1 + y'^2}$, so $L = y - \lambda\sqrt{1 + y'^2}$ will have $0$ partial derivative with respect to $x$. Using the Beltrami identify we have: + + +$$ +(y - \lambda\sqrt{1 + y'^2}) + \lambda y' \frac{2y'}{2\sqrt{1 + y'^2}} = C. +$$ + +by multiplying through by the denominator and squaring to remove the square root, a quadratic equation in $y'^2$ can be found. This can be solved to give: + + +$$ +y' = \frac{dy}{dx} = \sqrt{\frac{\lambda^2 -(y - C)^2}{(y-C)^2}}. +$$ + +Here is a snippet of `SymPy` code to verify the above: + + +```{julia} +#| hold: true +@syms y y′ λ C +ex = Eq(-λ*y′^2/sqrt(1 + y′^2) + λ*sqrt(1 + y′^2), y - C) +Δ = sqrt(1 + y′^2) / (y - C) +ex1 = Eq(simplify(ex.lhs()*Δ), simplify(ex.rhs() * Δ)) +ex2 = Eq(ex1.lhs()^2 - 1, simplify(ex1.rhs()^2) - 1) +``` + +Now $y'$ can be integrated using the substitution $y - C = \lambda \cos\theta$ to give: $-\lambda\int\cos\theta d\theta = x + D$, $D$ some constant. That is: + + +$$ +\begin{align*} +x + D &= - \lambda \sin\theta\\ +y - C &= \lambda\cos\theta. +\end{align*} +$$ + + +Squaring gives the equation of a circle: $(x +D)^2 + (y-C)^2 = \lambda^2$. + + +We center and *rescale* the problem so that $x_0 = -1, x_1 = 1$. Then $L > 2$ as otherwise the rope is too short. From here, we describe the radius and center of the circle. + + +We have $y=0$ at $x=1$ and $-1$ giving: + + +$$ +\begin{align*} +(-1 + D)^2 + (0 - C)^2 &= \lambda^2\\ +(+1 + D)^2 + (0 - C)^2 &= \lambda^2. +\end{align*} +$$ + + +Squaring out and solving gives $D=0$, $1 + C^2 = \lambda^2$. That is, an arc of circle with radius $\sqrt{1+C^2}$ and centered at $(0, C)$. + + +$$ +x^2 + (y - C)^2 = 1 + C^2. +$$ + +Now to identify $C$ in terms of $L$. $L$ is the length of arc of circle of radius $r =\sqrt{1 + C^2}$ and angle $2\theta$, so $L = 2r\theta$ But using the boundary conditions in the equations for $x$ and $y$ gives $\tan\theta = 1/C$, so $L = 2\sqrt{1 + C^2}\tan^{-1}(1/C)$ which can be solved for $C$ provided $L \geq 2$. + + +##### Example: more constraints + + +Consider now the case of maximizing $f(x,y,z)$ subject to $g(x,y,z)=c$ and $h(x,y,z) = d$. Can something similar be said to characterize potential values for this to occur? Trying to describe where $g(x,y,z) = c$ and $h(x,y,z)=d$ in general will prove difficult. The easy case would be it the two equations were linear, in which case they would describe planes. Two non-parallel planes would intersect in a line. If the general case, imagine the surfaces locally replaced by their tangent planes, then their intersection would be a line, and this line would point in along the curve given by the intersection of the surfaces formed by the constraints. This line is similar to the tangent line in the $2$-variable case. Now if $\nabla{f}$, which points in the direction of greatest increase of $f$, had a non-zero projection onto this line, then moving the point in that direction along the line would increase $f$ and still leave the point following the constraints. That is, if there is a non-zero directional derivative the point is not a maximum. + + +The tangent planes are *orthogonal* to the vectors $\nabla{g}$ and $\nabla{h}$, so in this case parallel to $\nabla{g} \times \nabla{h}$. The condition that $\nabla{f}$ be *orthogonal* to this vector, means that $\nabla{f}$ *must* sit in the plane described by $\nabla{g}$ and $\nabla{h}$ - the plane of orthogonal vectors to $\nabla{g} \times \nabla{h}$. That is, this condition is needed: + + +$$ +\nabla{f}(x,y,z) = \lambda_1 \nabla{g}(x,y,z) + \lambda_2 \nabla{h}(x,y,z). +$$ + +At a point satisfying the above, we would have the tangent "plane" of $f$ is contained in the intersection of the tangent "plane"s to $g$ and $h$. + + +--- + + +Consider a curve given through the intersection of two expressions: $g_1(x,y,z) = x^2 + y^2 - z^2 = 0$ and $g_2(x,y,z) = x - 2z = 3$. What is the minimum distance to the origin along this curve? + + +We have $f(x,y,z) = \text{distance}(\vec{x},\vec{0}) = \sqrt{x^2 + y^2 + z^2}$, subject to the two constraints. As the square root is increasing, we can actually just consider $f(x,y,z) = x^2 + y^2 + z^2$, ignoring the square root. The Lagrange multiplier technique instructs us to look for solutions to: + + +$$ +\langle 2x, 2y ,2z \rangle = \lambda_1\langle 2x, 2y, -2z\rangle + \lambda_2 \langle 1, 0, -2 \rangle. +$$ + +Here we use `SymPy`: + + +```{julia} +@syms z lambda1 lambda2 +g1(x, y, z) = x^2 + y^2 - z^2 +g2(x, y, z) = x - 2z - 3 +fₘ(x,y,z)= x^2 + y^2 + z^2 +Lₘ(x,y,z,lambda1, lambda2) = fₘ(x,y,z) - lambda1*(g1(x,y,z) - 0) - lambda2*(g2(x,y,z) - 0) + +∇Lₘ = diff.(Lₘ(x,y,z,lambda1, lambda2), [x, y, z,lambda1, lambda2]) +``` + +Before trying to solve for $\nabla{L} = \vec{0}$ we see from the second equation that *either* $\lambda_1 = 1$ or $y = 0$. First we solve with $\lambda_1 = 1$: + + +```{julia} +solve(subs.(∇Lₘ, lambda1 .=> 1)) +``` + +There are no real solutions. Next when $y = 0$ we get: + + +```{julia} +outₘ = solve(subs.(∇Lₘ, y .=> 0)) +``` + +The two solutions have values yielding the extrema: + + +```{julia} +[fₘ(d[x], 0, d[z]) for d in outₘ] +``` + +## Taylor's theorem + + +Taylor's theorem for a univariate function states that if $f$ has $k+1$ derivatives in an open interval around $a$, $f^{(k)}$ is continuous between the closed interval from $a$ to $x$ then: + + +$$ +f(x) = \sum_{j=0}^k \frac{f^{j}(a)}{j!} (x-a)^j + R_k(x), +$$ + +where $R_k(x) = f^{k+1}(\xi)/(k+1)!(x-a)^{k+1}$ for some $\xi$ between $a$ and $x$. + + +This theorem can be generalized to scalar functions, but the notation can be cumbersome. Following [Folland](https://sites.math.washington.edu/~folland/Math425/taylor2.pdf) we use *multi-index* notation. Suppose $f:R^n \rightarrow R$, and let $\alpha=(\alpha_1, \alpha_2, \dots, \alpha_n)$. Then define the following notation: + + +$$ +\begin{align*} +|\alpha| &= \alpha_1 + \cdots + \alpha_n, \\ +\alpha! &= \alpha_1!\alpha_2!\cdot\cdots\cdot\alpha_n!, \\ +\vec{x}^\alpha &= x_1^{\alpha_1}x_2^{\alpha_2}\cdots x_n^{\alpha^n}, \\ +\partial^\alpha f &= \partial_1^{\alpha_1}\partial_2^{\alpha_2}\cdots \partial_n^{\alpha_n} f \\ +& = \frac{\partial^{|\alpha|}f}{\partial x_1^{\alpha_1} \partial x_2^{\alpha_2} \cdots \partial x_n^{\alpha_n}}. +\end{align*} +$$ + + +This notation makes many formulas from one dimension carry over to higher dimensions. For example, the binomial theorem says: + + +$$ +(a+b)^n = \sum_{k=0}^n \frac{n!}{k!(n-k)!}a^kb^{n-k}, +$$ + +and this becomes: + + +$$ +(x_1 + x_2 + \cdots + x_n)^n = \sum_{|\alpha|=k} \frac{k!}{\alpha!} \vec{x}^\alpha. +$$ + +::: {.callout-note icon=false} +## Taylor's theorem using multi-index + +If $f: R^n \rightarrow R$ is sufficiently smooth ($C^{k+1}$) on an open convex set $S$ about $\vec{a}$ then if $\vec{a}$ and $\vec{a}+\vec{h}$ are in $S$, + + +$$ +f(\vec{a} + \vec{h}) = \sum_{|\alpha| \leq k}\frac{\partial^\alpha f(\vec{a})}{\alpha!}\vec{h}^\alpha + R_{\vec{a},k}(\vec{h}), +$$ + +where $R_{\vec{a},k} = \sum_{|\alpha|=k+1}\partial^\alpha \frac{f(\vec{a} + c\vec{h})}{\alpha!} \vec{h}^\alpha$ for some $c$ in $(0,1)$. + +::: + +##### Example + + +The elegant notation masks what can be complicated expressions. Consider the simple case $f:R^2 \rightarrow R$ and $k=2$. Then this says: + +$$ +\begin{align*} +f(x + dx, y+dy) &= f(x, y) + \frac{\partial f}{\partial x} dx + \frac{\partial f}{\partial y} dy \\ +&+ \frac{\partial^2 f}{\partial x^2} \frac{dx^2}{2} + 2\frac{\partial^2 f}{\partial x\partial y} \frac{dx dy}{2}\\ +&+ \frac{\partial^2 f}{\partial y^2} \frac{dy^2}{2} + R_{\langle x, y \rangle, k}(\langle dx, dy \rangle). +\end{align*} +$$ + +Using $\nabla$ and $H$ for the Hessian and $\vec{x} = \langle x, y \rangle$ and $d\vec{x} = \langle dx, dy \rangle$, this can be expressed as: + + +$$ +f(\vec{x} + d\vec{x}) = f(\vec{x}) + \nabla{f} \cdot d\vec{x} + d\vec{x} \cdot (H d\vec{x}) +R_{\vec{x}, k}d\vec{x}. +$$ + +As for $R$, the full term involves terms for $\alpha = (3,0), (2,1), (1,2)$, and $(0,3)$. Using $\vec{a} = \langle x, y\rangle$ and $\vec{h}=\langle dx, dy\rangle$: + + +$$ +\frac{\partial^3 f(\vec{a}+c\vec{h})}{\partial x^3} \frac{dx^3}{3!}+ +\frac{\partial^3 f(\vec{a}+c\vec{h})}{\partial x^2\partial y} \frac{dx^2 dy}{2!1!} + +\frac{\partial^3 f(\vec{a}+c\vec{h})}{\partial x\partial y^2} \frac{dxdy^2}{1!2!} + +\frac{\partial^3 f(\vec{a}+c\vec{h})}{\partial y^3} \frac{dy^3}{3!}. +$$ + +The exact answer is usually not as useful as the bound: $|R| \leq M/(k+1)! \|\vec{h}\|^{k+1}$, for some finite constant $M$. + + +##### Example + + +We can encode multiindices using `SymPy`. The basic definitions are fairly straightforward using `zip` to pair variables with components of $\alpha$. We define a new type so that we can overload the familiar notation: + + +```{julia} +struct MultiIndex + alpha::Vector{Int} + end +Base.show(io::IO, α::MultiIndex) = println(io, "α = ($(join(α.alpha, ", ")))") + +## |α| = α_1 + ... + α_m +Base.length(α::MultiIndex) = sum(α.alpha) + +## factorial(α) computes α! +Base.factorial(α::MultiIndex) = prod(factorial(Sym(a)) for a in α.alpha) + +## x^α = x_1^α_1 * x_2^α^2 * ... * x_n^α_n +import Base: ^ +^(x, α::MultiIndex) = prod(u^a for (u,a) in zip(x, α.alpha)) + +## ∂^α(ex) = ∂_1^α_1 ∘ ∂_2^α_2 ∘ ... ∘ ∂_n^α_n (ex) +partial(ex::SymPy.SymbolicObject, α::MultiIndex, vars=free_symbols(ex)) = diff(ex, zip(vars, α.alpha)...) +``` + +```{julia} +@syms w +alpha = MultiIndex([1,2,1,3]) +length(alpha) # 1 + 2 + 1 + 3=7 +[1,2,3,4]^alpha +exₜ = x^3 * cos(w*y*z) +partial(exₜ, alpha, [w,x,y,z]) +``` + +The remainder term needs to know information about sets like $|\alpha| =k$. This is a combinatoric problem, even to identify the length. Here we define an iterator to iterate over all possible MultiIndexes. This is low level, and likely could be done in a much better style, so shouldn't be parsed unless there is curiosity. It manually chains together iterators. + + +```{julia} +struct MultiIndices + n::Int + k::Int +end + +function Base.length(as::MultiIndices) + n,k = as.n, as.k + n == 1 && return 1 + sum(length(MultiIndices(n-1, j)) for j in 0:k) # recursively identify length +end + +function Base.iterate(alphas::MultiIndices) + k, n = alphas.k, alphas.n + n == 1 && return ([k],(0, MultiIndices(0,0), nothing)) + + m = zeros(Int, n) + m[1] = k + betas = MultiIndices(n-1, 0) + stb = iterate(betas) + st = (k, MultiIndices(n-1, 0), stb) + return (m, st) +end + +function Base.iterate(alphas::MultiIndices, st) + + st == nothing && return nothing + k,n = alphas.k, alphas.n + k == 0 && return nothing + n == 1 && return nothing + + # can we iterate the next on + bk, bs, stb = st + + if stb==nothing + bk = bk-1 + bk < 0 && return nothing + bs = MultiIndices(bs.n, bs.k+1) + val, stb = iterate(bs) + return (vcat(bk,val), (bk, bs, stb)) + end + + resp = iterate(bs, stb) + if resp == nothing + bk = bk-1 + bk < 0 && return nothing + bs = MultiIndices(bs.n, bs.k+1) + val, stb = iterate(bs) + return (vcat(bk, val), (bk, bs, stb)) + end + + val, stb = resp + return (vcat(bk, val), (bk, bs, stb)) + +end +``` + +This returns a vector, not a `MultiIndex`. Here we get all multiindices in two variables of size $3$ + + +```{julia} +collect(MultiIndices(2, 3)) +``` + +To get all of size $3$ or less, we could do something like this: + + +```{julia} +union((collect(MultiIndices(2, i)) for i in 0:3)...) +``` + +To see the computational complexity. Suppose we had $3$ variables and were interested in the error for order $4$: + + +```{julia} +k = 4 +length(MultiIndices(3, k+1)) +``` + +Finally, to see how compact the notation issue, suppose $f:R^3 \rightarrow R$, we have the third-order Taylor series expands to $20$ terms as follows: + + +```{julia} +#| hold: true +@syms 𝐅() a[1:3] dx[1:3] + +sum(partial(𝐅(a...), α, a) / factorial(α) * dx^α for k in 0:3 for α in MultiIndex.(MultiIndices(3, k))) # 3rd order +``` + +## Questions + + +###### Question + + +Let $f(x,y) = \sqrt{x + y}$. Find the tangent plane approximation for $f(2.1, 2.2)$? + + +```{julia} +#| hold: true +#| echo: false +f(x,y) = sqrt(x + y) +f(v) = f(v...) +pt = [2,2] +dxdy = [.1, .2] +val = f(pt) + dot(ForwardDiff.gradient(f, pt), dxdy) +numericq(val) +``` + +###### Question + + +Let $f(x,y,z) = xy + yz + zx$. Using a *linear approximation* estimate $f(1.1, 1.0, 0.9)$. + + +```{julia} +#| hold: true +#| echo: false +f(x,y,z) = x*y + y*z + z*x +f(v) = f(v...) +pt = [1,1,1] +dx = [0.1, 0.0, -0.1] +val = f(pt) + ∇(f)(pt) ⋅ dx +numericq(val) +``` + +###### Question + + +Let $f(x,y,z) = xy + yz + zx - 3$. What equation describes the tangent approximation at $(1,1,1)$? + + +```{julia} +#| hold: true +#| echo: false +f(x,y,z) = x*y + y*z + z*x - 8 +f(v) = f(v...) +pt = [1,1,1] +n = ∇(f)(pt) +d = dot(n, pt) +choices = [ + raw"`` x + y + z = 3``", + raw"`` 2x + y - 2z = 1``", + raw"`` x + 2y + 3z = 6``" +] +answ = 1 +radioq(choices, answ) +``` + +###### Question + + +([Knill](http://www.math.harvard.edu/~knill/teaching/summer2018/handouts/week4.pdf)) Let $f(x,y) = xy + x^2y + xy^2$. + + +Find the gradient of $f$: + + +```{julia} +#| hold: true +#| echo: false +choices = [ + raw"`` \langle 2xy + y^2 + y, 2xy + x^2 + x\rangle``", + raw"`` y^2 + y, x^2 + x``", + raw"`` \langle 2y + y^2, 2x + x^2``" +] +answ = 1 +radioq(choices, answ) +``` + +Is this the Hessian of $f$? + + +$$ +\left[\begin{matrix}2 y & 2 x + 2 y + 1\\2 x + 2 y + 1 & 2 x\end{matrix}\right] +$$ + +```{julia} +#| hold: true +#| echo: false +yesnoq(true) +``` + +The point $(-1/3, -1/3)$ is a solution to the $\nabla{f} = 0$. What is the *determinant*, $d$, of the Hessian at this point? + + +```{julia} +#| hold: true +#| echo: false +f(x,y) = x*y + x*y^2 + x^2 * y +f(v) = f(v...) +val = det(ForwardDiff.hessian(f, [-1/3, -1/3])) +numericq(val) +``` + +Which is true of $f$ at $(-1/3, -1/3)$: + + +```{julia} +#| hold: true +#| echo: false +choices = [ + L"The function $f$ has a local minimum, as $f_{xx} > 0$ and $d >0$", + L"The function $f$ has a local maximum, as $f_{xx} < 0$ and $d >0$", + L"The function $f$ has a saddle point, as $d < 0$", + L"Nothing can be said, as $d=0$" +] +answ = 2 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +([Knill](http://www.math.harvard.edu/~knill/teaching/summer2018/handouts/week4.pdf)) Let the Tutte polynomial be $f(x,y) = x + 2x^2 + x^3 + y + 2xy + y^2$. + + +Does this accurately find the gradient of $f$? + + +```{julia} +#| hold: true +#| results: "hidden" +f(x,y) = x + 2x^2 + x^3 + y + 2x*y + y^2 +@syms x::real y::real +gradf = gradient(f(x,y), [x,y]) +``` + +```{julia} +#| hold: true +#| echo: false +yesnoq(true) +``` + +How many answers does this find to $\nabla{f} = \vec{0}$? + + +```{julia} +#| hold: true +#| results: "hidden" +f(x,y) = x + 2x^2 + x^3 + y + 2x*y + y^2 +@syms x::real y::real +gradf = gradient(f(x,y), [x,y]) + +solve(gradf, [x,y]) +``` + +```{julia} +#| hold: true +#| echo: false +numericq(2) +``` + +The Hessian is found by + + +```{julia} +#| hold: true +f(x,y) = x + 2x^2 + x^3 + y + 2x*y + y^2 +@syms x::real y::real +gradf = gradient(f(x,y), [x,y]) + +sympy.hessian(f(x,y), [x,y]) +``` + +Which is true of $f$ at $(-2/3, 1/6)$: + + +```{julia} +#| hold: true +#| echo: false +choices = [ + L"The function $f$ has a local minimum, as $f_{xx} > 0$ and $d >0$", + L"The function $f$ has a local maximum, as $f_{xx} < 0$ and $d >0$", + L"The function $f$ has a saddle point, as $d < 0$", + L"Nothing can be said, as $d=0$", + L"The test does not apply, as $\nabla{f}$ is not $0$ at this point." +] +answ = 3 +radioq(choices, answ, keep_order=true) +``` + +Which is true of $f$ at $(0, -1/2)$: + + +```{julia} +#| hold: true +#| echo: false +choices = [ + L"The function $f$ has a local minimum, as $f_{xx} > 0$ and $d >0$", + L"The function $f$ has a local maximum, as $f_{xx} < 0$ and $d >0$", + L"The function $f$ has a saddle point, as $d < 0$", + L"Nothing can be said, as $d=0$", + L"The test does not apply, as $\nabla{f}$ is not $0$ at this point." +] +answ = 1 +radioq(choices, answ, keep_order=true) +``` + +Which is true of $f$ at $(1/2, 0)$: + + +```{julia} +#| hold: true +#| echo: false +choices = [ + L"The function $f$ has a local minimum, as $f_{xx} > 0$ and $d >0$", + L"The function $f$ has a local maximum, as $f_{xx} < 0$ and $d >0$", + L"The function $f$ has a saddle point, as $d < 0$", + L"Nothing can be said, as $d=0$", + L"The test does not apply, as $\nabla{f}$ is not $0$ at this point." +] +answ = 5 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +(Strang p509) Consider the quadratic function $f(x,y) = ax^2 + bxy +cy^2$. Since the second partial derivative test is essentially done by replacing the function at a critical point by a quadratic function, understanding this $f$ is of some interest. + + +Is this the Hessian of $f$? + + +$$ +\begin{bmatrix} +2a & b\\ +b & 2c +\end{bmatrix} +$$ + +```{julia} +#| hold: true +#| echo: false +yesnoq(true) +``` + +Or is this the Hessian of $f$? + + +$$ +\begin{bmatrix} +2ax & by\\ +bx & 2cy +\end{bmatrix} +$$ + +```{julia} +#| hold: true +#| echo: false +yesnoq(false) +``` + +Explain why $4ac - b^2$ is of any interest here: + + +```{julia} +#| hold: true +#| echo: false +choices =[ + "It is the determinant of the Hessian", + L"It isn't, $b^2-4ac$ is from the quadratic formula" +] +answ = 1 +radioq(choices, answ) +``` + +Which condition on $a$, $b$, and $c$ will ensure a *local maximum*: + + +```{julia} +#| hold: true +#| echo: false +choices = [ + L"That $a>0$ and $4ac-b^2 > 0$", + L"That $a<0$ and $4ac-b^2 > 0$", + L"That $4ac-b^2 < 0$" +] +answ = 2 +radioq(choices, answ, keep_order=true) +``` + +Which condition on $a$, $b$, and $c$ will ensure a saddle point? + + +```{julia} +#| hold: true +#| echo: false +choices = [ + L"That $a>0$ and $4ac-b^2 > 0$", + L"That $a<0$ and $4ac-b^2 > 0$", + L"That $4ac-b^2 < 0$" +] +answ = 3 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +Let $f(x,y) = e^{-x^2 - y^2} (2x^2 + y^2)$. Use Lagrange's method to find the absolute maximum and absolute minimum over $x^2 + y^2 = 3$. + + +Is $\nabla{f}$ given by the following? + + +$$ +\nabla{f} =2 e^{-x^2 - y^2} \langle x(2 - 2x^2 - y^2), y(1 - 2x^2 - y^2)\rangle. +$$ + +```{julia} +#| hold: true +#| echo: false +yesnoq(true) +``` + +Which vector is orthogonal to the contour line $x^2 + y^2 = 3$? + + +```{julia} +#| echo: false +choices = [ + raw"`` \langle 2x, 2y\rangle``", + raw"`` \langle 2x, y^2\rangle``", + raw"`` \langle x^2, 2y \rangle``" +] +answ = 1 +radioq(choices, answ) +``` + +Due to the form of the gradient of the constraint, finding when $\nabla{f} = \lambda \nabla{g}$ is the same as identifying when this ratio $|f_x/f_y|$ is $1$. The following solves for this by checking each point on the constraint: + + +```{julia} +f(x,y) = exp(-x^2-y^2) * (2x^2 + y^2) +f(v) = f(v...) +r(t) = sqrt(3)*[cos(t), sin(t)] +rat(x) = abs(x[1]/x[2]) - 1 +fn = rat ∘ ∇(f) ∘ r +ts = fzeros(fn, 0, 2pi) +``` + +Using these points, what is the largest value on the boundary? + + +```{julia} +#| eval: false +#| echo: false +f(x,y) = exp(-x^2-y^2) * (2x^2 + y^2) +r(t) = sqrt(3)*[cos(t), sin(t)] +rat(x) = abs(x[1]/x[2]) - 1 +fn = rat ∘ ∇(splat(f)) ∘ r +ts = fzeros(fn, 0, 2pi) + +val = maximum((splat(u)∘r).(ts)) +numericq(val) +``` diff --git a/quarto/differentiable_vector_calculus/scalar_functions_applications.qmd b/quarto/differentiable_vector_calculus/scalar_functions_applications.qmd index 2b74579..dec255f 100644 --- a/quarto/differentiable_vector_calculus/scalar_functions_applications.qmd +++ b/quarto/differentiable_vector_calculus/scalar_functions_applications.qmd @@ -9,7 +9,7 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia using Plots -plotly() +#plotly() using SymPy using Roots ``` @@ -33,7 +33,6 @@ Consider the case $f:R^2 \rightarrow R$. We visualize $z=f(x,y)$ through a surfa For the univariate case, the tangent line has many different uses. Here we see the tangent plane also does. - ### Equation of the tangent plane @@ -119,6 +118,7 @@ arrow!(pt, [1,0,0], linestyle=:dash) arrow!(pt, [0,1,0], linestyle=:dash) ``` + #### Alternate forms @@ -170,7 +170,6 @@ For clarity: * The scalar function $z = f(x,y)$ describes a surface, $(x,y,f(x,y))$; the gradient, $\nabla{f}$, is $2$ dimensional and points in the direction of greatest ascent for the surface. * The scalar function $f(x,y,z)$ *also* describes a surface, through level curves $f(x,y,z) = c$, for some *constant* $c$. The gradient $\nabla{f}$ is $3$ dimensional and *orthogonal* to the surface. - ##### Example @@ -231,6 +230,7 @@ t = 1/2 (n1(gamma(t)) × n2(gamma(t))) × gamma'(t) ``` + #### Plotting level curves of $F(x,y,z) = c$ @@ -250,7 +250,6 @@ f(x,y,z) = (x^2 + ((1+b) * y)^2 + z^2 - 1)^3 - x^2 * z^3 - a * y^2 * z^3 CalculusWithJulia.plot_implicit_surface(f, xlim=-2..2, ylim=-1..1, zlim=-1..2) ``` - ## Linearization @@ -329,7 +328,7 @@ Here, $\nabla{f}$ describes a *normal* to the tangent plane. The description of f(x,y,z) = x^4 -x^3 + y^2 + z^2 f(v) = f(v...) a, b,c = ∇(f)(2,2,2) -"$a x + $b y + $c z = $([a,b,c] ⋅ [2,2,2])" +println("$a x + $b y + $c z = $([a,b,c] ⋅ [2,2,2])"); ``` ### Newton's method to solve $f(x,y) = 0$ and $g(x,y)=0$. @@ -504,6 +503,9 @@ function nm(f, g, x, n=5) end ``` + + + ##### Example @@ -655,6 +657,7 @@ plot(F, 0.01, 5) # p > 0 This problem does not have a readily expressed value for $x^*$, but when $p \approx 0$ we should get similar behavior to the intersection of $y=px$ and $y=\pi/2 - x$ for $x^*$, or $x^* \approx \pi/(2(1+p))$ which has derivative of $-\pi/2$ at $p=0$, matching the above graph. For *large* $p$, the problem looks like the intersection of the line $y=1$ with $y=px$ or $x^* \approx 1/p$ which has derivative that goes to $0$ as $p$ goes to infinity, again matching this graph. + ## Optimization @@ -807,7 +810,6 @@ det(H_a) (The test is inconclusive, as it needs the function to "fall away" from the tangent plane in all directions, in this case, along a circular curve, the function touches the tangent plane, so it doesn't fall away.) - ##### Example @@ -853,6 +855,7 @@ end p ``` + ##### Example @@ -875,9 +878,8 @@ At $\vec{a}$ this has positive determinant and $f_{xx} > 0$, so $\vec{a}$ corres ```{julia} fₗ(x,y) = x^2 + 2y^2 - x -fₗ(v) = fₗ(v...) gammaₗ(t) = [cos(t), sin(t)] # traces out x^2 + y^2 = 1 over [0, 2pi] -gₗ = fₗ ∘ gammaₗ +gₗ = splat(fₗ) ∘ gammaₗ cpsₗ = find_zeros(gₗ', 0, 2pi) # critical points of g append!(cpsₗ, [0, 2pi]) @@ -907,6 +909,12 @@ So we have the maximum occurs at the angles $2\pi/3$ and $4\pi/3$. Here we visua hₗ(x,y) = fₗ(x,y) * (x^2 + y^2 <= 1 ? 1 : NaN) ``` + +```{julia} +#| echo: false +gr(); # scatter3d! and plotly() are failing!! +``` + ```{julia} #| hold: true xs = ys = range(-1,1, length=100) @@ -918,6 +926,11 @@ zs = fₗ.(xs, ys) scatter3d!(xs, ys, zs) ``` +```{julia} +#| echo: false +#plotly(); +``` + A contour plot also shows that some---and only one---extrema happens on the interior: @@ -930,6 +943,7 @@ contour(xs, ys, hₗ) The extrema are identified by the enclosing regions, in this case the one around the point $(1/2, 0)$. + ##### Example: Steiner's problem @@ -1189,6 +1203,10 @@ end We now visualize, using the `Contour` package to draw the contour lines in the $x-y$ plane: +```{julia} +#| echo: false +gr() +``` ```{julia} #| hold: true @@ -1217,6 +1235,11 @@ scatter3d!(unzip(pts)...) plot!(unzip(pts)..., linewidth=3) ``` +```{julia} +#| echo: false +#plotly() +``` + ### Newton's method for minimization @@ -2324,7 +2347,6 @@ Which vector is orthogonal to the contour line $x^2 + y^2 = 3$? ```{julia} -#| hold: true #| echo: false choices = [ raw"`` \langle 2x, 2y\rangle``", @@ -2339,7 +2361,6 @@ Due to the form of the gradient of the constraint, finding when $\nabla{f} = \la ```{julia} -#| hold: true f(x,y) = exp(-x^2-y^2) * (2x^2 + y^2) f(v) = f(v...) r(t) = sqrt(3)*[cos(t), sin(t)] @@ -2352,15 +2373,14 @@ Using these points, what is the largest value on the boundary? ```{julia} -#| hold: true +#| eval: false #| echo: false f(x,y) = exp(-x^2-y^2) * (2x^2 + y^2) -f(v) = f(v...) r(t) = sqrt(3)*[cos(t), sin(t)] rat(x) = abs(x[1]/x[2]) - 1 -fn = rat ∘ ∇(f) ∘ r +fn = rat ∘ ∇(splat(f)) ∘ r ts = fzeros(fn, 0, 2pi) -val = maximum((f∘r).(ts)) +val = maximum((splat(u)∘r).(ts)) numericq(val) ``` diff --git a/quarto/differentiable_vector_calculus/test.html b/quarto/differentiable_vector_calculus/test.html new file mode 100644 index 0000000..26d9d85 --- /dev/null +++ b/quarto/differentiable_vector_calculus/test.html @@ -0,0 +1,641 @@ + + + + + + + + + +test + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+ +
+
+

Applications with scalar functions

+
+ + + +
+ + + + +
+ + + +
+ + +

This section uses these add-on packages:

+
+
using CalculusWithJulia
+using Plots
+plotly()
+using SymPy
+using Roots
+
+
+
Example
+

Consider the function \(f(x,y) = x^2 + 3y^2 -x\) over the region \(x^2 + y^2 \leq 1\). This is a continuous function over a closed set, so will have both an absolute maximum and minimum. Find these from an investigation of the critical points and the boundary points.

+

The gradient is easily found: \(\nabla{f} = \langle 2x - 1, 6y \rangle\), and is \(\vec{0}\) only at \(\vec{a} = \langle 1/2, 0 \rangle\). The Hessian is:

+

\[ +H = +\begin{bmatrix} +2 & 0\\ +0 & 6 +\end{bmatrix}. +\]

+

At \(\vec{a}\) this has positive determinant and \(f_{xx} > 0\), so \(\vec{a}\) corresponds to a local minimum with values \(f(\vec{a}) = (1/2)^2 + 3(0) - 1/2 = -1/4\). The absolute maximum and minimum may occur here (well, not the maximum) or on the boundary, so that must be considered. In this case we can easily parameterize the boundary and turn this into the univariate case:

+
+
fₗ(x,y) = x^2 + 2y^2 - x
+gammaₗ(t) = [cos(t), sin(t)]  # traces out x^2 + y^2 = 1 over [0, 2pi]
+gₗ = splat(fₗ) ∘ gammaₗ
+
+cpsₗ = find_zeros(gₗ', 0, 2pi) # critical points of g
+append!(cpsₗ, [0, 2pi])
+unique!(cpsₗ)
+gₗ.(cpsₗ)
+
+
5-element Vector{Float64}:
+ 0.0
+ 2.25
+ 2.0
+ 2.25
+ 0.0
+
+
+

We see that maximum value is 2.25 and that the interior point, \(\vec{a}\), will be where the minimum value occurs. To see exactly where the maximum occurs, we look at the values of gamma:

+
+
inds = [2,4]
+cpsₗ[inds]
+
+
2-element Vector{Float64}:
+ 2.0943951023931953
+ 4.1887902047863905
+
+
+

These are multiples of \(\pi\):

+
+
cpsₗ[inds]/pi
+
+
2-element Vector{Float64}:
+ 0.6666666666666666
+ 1.3333333333333333
+
+
+

So we have the maximum occurs at the angles \(2\pi/3\) and \(4\pi/3\). Here we visualize, using a hacky trick of assigning NaN values to the function to avoid plotting outside the circle:

+
+
hₗ(x,y) = fₗ(x,y) * (x^2 + y^2 <= 1 ? 1 : NaN)
+
+
hₗ (generic function with 1 method)
+
+
+
+
gr()
+
+
Plots.GRBackend()
+
+
+
+
xs = ys = range(-1,1, length=100)
+plt = surface(xs, ys, hₗ)
+
+ts = cpsₗ  # 2pi/3 and 4pi/3 by above
+xs, ys = cos.(ts), sin.(ts)
+zs = fₗ.(xs, ys)
+scatter3d!(xs, ys, zs)
+#tuple.(xs, ys, zs)
+#plot!(plt, tuple.(xs, ys, zs); linetype=:scatter)
+#plt
+
+ +
+
+

A contour plot also shows that some—and only one—extrema happens on the interior:

+ +
+ +
+ + +
+ + + + + \ No newline at end of file diff --git a/quarto/differentiable_vector_calculus/test.jl b/quarto/differentiable_vector_calculus/test.jl new file mode 100644 index 0000000..fe58175 --- /dev/null +++ b/quarto/differentiable_vector_calculus/test.jl @@ -0,0 +1,827 @@ +@show 4 +using QuizQuestions +using LaTeXStrings +using CalculusWithJulia +using Plots +plotly() +using SymPy +using Roots +@show 6 +import Contour: contours, levels, level, lines, coordinates +@show 15 +@syms f_x f_y +n = [1, 0, f_x] × [0, 1, f_y] +@show 27 +#| hold: true +f(x,y) = 6 - x^2 -y^2 +f(x)= f(x...) + +a,b = 1, -1/2 + + +# draw surface +xr = 7/4 +xs = ys = range(-xr, xr, length=100) +surface(xs, ys, f, legend=false) + +# visualize tangent plane as 3d polygon +pt = [a,b] +tplane(x) = f(pt) + gradient(f)(pt) ⋅ (x - [a,b]) + +pts = [[a-1,b-1], [a+1, b-1], [a+1, b+1], [a-1, b+1], [a-1, b-1]] +plot!(unzip([[pt..., tplane(pt)] for pt in pts])...) + +# plot paths in x and y direction through (a,b) +γ_x(t) = pt + t*[1,0] +γ_y(t) = pt + t*[0,1] + +plot_parametric!((-xr-a)..(xr-a), t -> [γ_x(t)..., (f∘γ_x)(t)], linewidth=3) +plot_parametric!((-xr-b)..(xr-b), t -> [γ_y(t)..., (f∘γ_y)(t)], linewidth=3) + +# draw directional derivatives in 3d and normal +pt = [a, b, f(a,b)] +fx, fy = gradient(f)(a,b) +arrow!(pt, [1, 0, fx], linewidth=3) +arrow!(pt, [0, 1, fy], linewidth=3) +arrow!(pt, [-fx, -fy, 1], linewidth=3) # normal + +# draw point in base, x-y, plane +pt = [a, b, 0] +scatter!(unzip([pt])...) +arrow!(pt, [1,0,0], linestyle=:dash) +arrow!(pt, [0,1,0], linestyle=:dash) +@show 33 +function tangent_plane_1st_crack(f, pt) + fx, fy = ForwardDiff.gradient(f, pt) + x -> f(x...) + fx * (x[1]-pt[1]) + fy * (x[2]-pt[2]) +end +@show 35 +function tangent_plane(f, pt) + ∇f = ForwardDiff.gradient(f, pt) # using a variable ∇f + x -> f(pt) + ∇f ⋅ (x - pt) +end +@show 46 +@syms x, y +@show 47 +#| hold: true +f(x,y) = sin(x) * cos(x-y) +f(x) = f(x...) +vars = [x, y] + +gradf = diff.(f(x,y), vars) # or use gradient(f, vars) or ∇((f,vars)) + +pt = [PI/4, PI/3] +gradfa = subs.(gradf, x=>pt[1], y=>pt[2]) + +f(pt) + gradfa ⋅ (vars - pt) +@show 55 +#| hold: true +a = 1 +gamma(t) = a * [1 + cos(t), sin(t), 2sin(t/2) ] +P = gamma(1/2) +n1(x,y,z)= [2*(x-a), 2y, 0] +n2(x,y,z) = [2x,2y,2z] +n1(x) = n1(x...) +n2(x) = n2(x...) + +t = 1/2 +(n1(gamma(t)) × n2(gamma(t))) × gamma'(t) +@show 60 +#| hold: true +a, b = 1, 3 +f(x,y,z) = (x^2 + ((1+b) * y)^2 + z^2 - 1)^3 - x^2 * z^3 - a * y^2 * z^3 + +CalculusWithJulia.plot_implicit_surface(f, xlim=-2..2, ylim=-1..1, zlim=-1..2) +@show 71 +V(r, h) = pi * r^2 * h +V(v) = V(v...) +a₁ = [1,2] +dx₁ = [0.01, 0.01] +ForwardDiff.gradient(V, a₁) ⋅ dx₁ # or use ∇(V)(a) +@show 73 +V(a₁ + dx₁) - V(a₁) +@show 85 +#| hold: true +f(x,y,z) = x^4 -x^3 + y^2 + z^2 +f(v) = f(v...) +a, b,c = ∇(f)(2,2,2) +"$a x + $b y + $c z = $([a,b,c] ⋅ [2,2,2])" +#@show 92 +#| hold: true +@syms a b c d u v +M = [a b; c d] +B = [u, v] +M \ B .|> simplify +@show 96 +#| hold: true +#| echo: false +f(x,y) = 2 - x^2 - y^2 +g(x,y) = 3 - 2x^2 - (1/3)y^2 +xs = ys = range(-3, stop=3, length=100) +zfs = [f(x,y) for x in xs, y in ys] +zgs = [g(x,y) for x in xs, y in ys] + + +ps = Any[] +pf = surface(xs, ys, f, alpha=0.5, legend=false) + +for cl in levels(contours(xs, ys, zfs, [0.0])) + for line in lines(cl) + _xs, _ys = coordinates(line) + plot!(pf, _xs, _ys, 0*_xs, linewidth=3, color=:blue) + end +end + + +pg = surface(xs, ys, g, alpha=0.5, legend=false) +for cl in levels(contours(xs, ys, zgs, [0.0])) + for line in lines(cl) + _xs, _ys = coordinates(line) + plot!(pg, _xs, _ys, 0*_xs, linewidth=3, color=:red) + end +end + +pcnt = plot(legend=false) +for cl in levels(contours(xs, ys, zfs, [0.0])) + for line in lines(cl) + _xs, _ys = coordinates(line) + plot!(pcnt, _xs, _ys, linewidth=3, color=:blue) + end +end + +for cl in levels(contours(xs, ys, zgs, [0.0])) + for line in lines(cl) + _xs, _ys = coordinates(line) + plot!(pcnt, _xs, _ys, linewidth=3, color=:red) + end +end + +l = @layout([a b c]) +plot(pf, pg, pcnt, layout=l) +@show 106 +function newton_step(f, g, xn) + M = [ForwardDiff.gradient(f, xn)'; ForwardDiff.gradient(g, xn)'] + b = -[f(xn), g(xn)] + Delta = M \ b + xn + Delta +end +@show 108 +𝒇(x,y) = 2 - x^2 - y^2 +𝒈(x,y) = 3 - 2x^2 - (1/3)y^2 +𝒇(v) = 𝒇(v...); 𝒈(v) = 𝒈(v...) +𝒙₀ = [1,1] +𝒙₁ = newton_step(𝒇, 𝒈, 𝒙₀) +@show 110 +𝒇(𝒙₁), 𝒈(𝒙₁) +@show 112 +𝒙₂ = newton_step(𝒇, 𝒈, 𝒙₁) +𝒙₃ = newton_step(𝒇, 𝒈, 𝒙₂) +𝒙₄ = newton_step(𝒇, 𝒈, 𝒙₃) +𝒙₅ = newton_step(𝒇, 𝒈, 𝒙₄) +𝒙₅, 𝒇(𝒙₅), 𝒈(𝒙₅) +@show 116 +function nm(f, g, x, n=5) + for i in 1:n + x = newton_step(f, g, x) + end + x +end +@show 123 +#| hold: true +c = 1/2 +f(x,y) = 1 - y^2 - c^2 +g(x,y) = (1 - x^2) - c^2 +f(v) = f(v...); g(v) = g(v...) +nm(f, g, [1/2, 1/3]) +@show 148 +#| hold: true +@syms x, y, Z() +∂x = solve(diff(x^4 -x^3 + y^2 + Z(x,y)^2, x), diff(Z(x,y),x)) +∂y = solve(diff(x^4 -x^3 + y^2 + Z(x,y)^2, y), diff(Z(x,y),y)) +∂x, ∂y +@show 158 +f(x, p) = cos(x) - p*x +p = 2 +xᵅ = find_zero(f, (0, pi/2), p) +@show 160 +p = 2 +xᵅ = find_zero(f, (0, pi/2), p) +fₓ = ForwardDiff.derivative(x -> f(x,p), xᵅ) +fₚ = ForwardDiff.derivative(p -> f(xᵅ, p), p) +- fₚ / fₓ +@show 163 +function find_zero_derivative(f, x₀, p) + xᵅ = find_zero(f, x₀, p) + fₓ = ForwardDiff.derivative(x -> f(x,p), xᵅ) + fₚ = ForwardDiff.derivative(p -> f(xᵅ, p), p) + - fₚ / fₓ +end +F(p) = find_zero_derivative(f, (0, pi/2), p) +plot(F, 0.01, 5) # p > 0 +@show 183 +#| hold: true +f(x,y)= exp(-(x^2 + y^2)/5) * cos(x^2 + y^2) +xs = ys = range(-4, 4, length=100) +surface(xs, ys, f, legend=false) +@show 190 +#| hold: true +f(x,y) = x*y +xs = ys = range(-3, 3, length=100) +surface(xs, ys, f, legend=false) + +plot_parametric!(-4..4, t -> [t, 0, f(t, 0)], linewidth=5) +plot_parametric!(-4..4, t -> [0, t, f(0, t)], linewidth=5) +@show 203 +fₖ(x,y) = exp(-(x^2 + y^2)/5) * cos(x^2 + y^2) +Hₖ = sympy.hessian(fₖ(x,y), (x,y)) +@show 205 +H₀₀ = subs.(Hₖ, x=>0, y=>0) +@show 207 +H₀₀[1,1] < 0 && det(H₀₀) > 0 +@show 209 +#| hold: true +gradfₖ = diff.(fₖ(x,y), [x,y]) +a = [sqrt(2PI + atan(-Sym(1)//5)), 0] +subs.(gradfₖ, x => a[1], y => a[2]) +@show 211 +#| hold: true +a = [sqrt(PI + atan(-Sym(1)//5)), 0] +H_a = subs.(Hₖ, x => a[1], y => a[2]) +det(H_a) +@show 216 +fⱼ(x,y) = 4x*y - x^4 - y^4 +gradfⱼ = diff.(fⱼ(x,y), [x,y]) +@show 217 +all_ptsⱼ = solve(gradfⱼ, [x,y]) +ptsⱼ = filter(u -> all(isreal.(u)), all_ptsⱼ) +@show 219 +Hⱼ = sympy.hessian(fⱼ(x,y), (x,y)) +function classify(H, pt) + Ha = subs.(H, x => pt[1], y => pt[2]) + (det=det(Ha), f_xx=Ha[1,1]) +end +[classify(Hⱼ, pt) for pt in ptsⱼ] +@show 221 +#| hold: true +xs = ys = range(-3/2, 3/2, length=100) +p = surface(xs, ys, fⱼ, legend=false) +for pt ∈ ptsⱼ + scatter!(p, unzip([N.([pt...,fⱼ(pt...)])])..., + markercolor=:black, markersize=5) # add each pt on surface +end +p +@show 228 +fₗ(x,y) = x^2 + 2y^2 - x +fₗ(v) = fₗ(v...) +gammaₗ(t) = [cos(t), sin(t)] # traces out x^2 + y^2 = 1 over [0, 2pi] +gₗ = fₗ ∘ gammaₗ + +cpsₗ = find_zeros(gₗ', 0, 2pi) # critical points of g +append!(cpsₗ, [0, 2pi]) +unique!(cpsₗ) +gₗ.(cpsₗ) +@show 230 +inds = [2,4] +cpsₗ[inds] +@show 232 +cpsₗ[inds]/pi +@show 234 +hₗ(x,y) = fₗ(x,y) * (x^2 + y^2 <= 1 ? 1 : NaN) +@show 235 +#| hold: true +xs = ys = range(-1,1, length=100) +surface(xs, ys, hₗ) + +ts = cpsₗ # 2pi/3 and 4pi/3 by above +xs, ys = cos.(ts), sin.(ts) +zs = fₗ.(xs, ys) +scatter3d!(xs, ys, zs) +@show 237 +#| hold: true +xs = ys = range(-1,1, length=100) +contour(xs, ys, hₗ) +@show 243 +@syms x1 y1 x2 y2 x3 y3 +d2(p,x) = (p[1] - x[1])^2 + (p[2]-x[2])^2 +d2_1, d2_2, d2_3 = d2((x,y), (x1, y1)), d2((x,y), (x2, y2)), d2((x,y), (x3, y3)) +exₛ = d2_1 + d2_2 + d2_3 +@show 245 +gradfₛ = diff.(exₛ, [x,y]) +xstarₛ = solve(gradfₛ, [x,y]) +@show 248 +Hₛ = subs.(hessian(exₛ, [x,y]), x=>xstarₛ[x], y=>xstarₛ[y]) +@show 259 +usₛ = [[cos(t), sin(t)] for t in (0, 2pi/3, 4pi/3)] +polygon(ps) = unzip(vcat(ps, ps[1:1])) # easier way to plot a polygon + +pₛ = scatter([0],[0], markersize=2, legend=false, aspect_ratio=:equal) + +asₛ = (1,2,3) +plot!(polygon([a*u for (a,u) in zip(asₛ, usₛ)])...) +[arrow!([0,0], a*u, alpha=0.5) for (a,u) in zip(asₛ, usₛ)] +pₛ +@show 261 +asₛ₁ = (1, -1, 3) +scatter([0],[0], markersize=2, legend=false) +psₛₗ = [a*u for (a,u) in zip(asₛ₁, usₛ)] +plot!(polygon(psₛₗ)...) +@show 263 +euclid_dist(x; ps=psₛₗ) = sum(norm(x-p) for p in ps) +euclid_dist(x,y; ps=psₛₗ) = euclid_dist([x,y]; ps=ps) +@show 264 +#| hold: true +xs = range(-1.5, 1.5, length=100) +ys = range(-3, 1.0, length=100) + +p = plot(polygon(psₛₗ)..., linewidth=3, legend=false) +scatter!(p, unzip(psₛₗ)..., markersize=3) +contour!(p, xs, ys, euclid_dist) + +# add some gradients along boundary +li(t, p1, p2) = p1 + t*(p2-p1) # t in [0,1] +for t in range(1/100, 1/2, length=3) + pt = li(t, psₛₗ[2], psₛₗ[3]) + arrow!(pt, ForwardDiff.gradient(euclid_dist, pt)) + pt = li(t, psₛₗ[2], psₛₗ[1]) + arrow!(pt, ForwardDiff.gradient(euclid_dist, pt)) +end + +p +@show 266 +#| hold : true +li(t, p1, p2) = p1 + t*(p2-p1) +p = plot(legend=false) +for i in 1:2, j in (i+1):3 + plot!(p, t -> euclid_dist(li(t, psₛₗ[i], psₛₗ[j]); ps=psₛₗ), 0, 1) +end +p +@show 280 +@syms xₗₛ[1:3] yₗₛ[1:3] α β +li(x, alpha, beta) = alpha + beta * x +d₂(alpha, beta) = sum((y - li(x, alpha, beta))^2 for (y,x) in zip(yₗₛ, xₗₛ)) +d₂(α, β) +@show 282 +grad_d₂ = diff.(d₂(α, β), [α, β]) +@show 283 +outₗₛ = solve(grad_d₂, [α, β]) +@show 285 +subs(outₗₛ[β], sum(xₗₛ) => 0) +@show 292 +[k => subs(v, xₗₛ[1]=>1, yₗₛ[1]=>1, xₗₛ[2]=>2, yₗₛ[2]=>3, + xₗₛ[3]=>5, yₗₛ[3]=>8) for (k,v) in outₗₛ] +@show 302 +f₂(x,y) = -exp(-((x-1)^2 + 2(y-1/2)^2)) +f₂(x) = f₂(x...) + +xs₂ = [[0.0, 0.0]] # we store a vector +gammas₂ = [1.0] + +for n in 1:5 + xn = xs₂[end] + gamma₀ = gammas₂[end] + xn1 = xn - gamma₀ * gradient(f₂)(xn) + dx, dy = xn1 - xn, gradient(f₂)(xn1) - gradient(f₂)(xn) + gamman1 = abs( (dx ⋅ dy) / (dy ⋅ dy) ) + + push!(xs₂, xn1) + push!(gammas₂, gamman1) +end + +[(x, f₂(x)) for x in xs₂] +@show 304 +#| hold: true +function surface_contour(xs, ys, f; offset=0) + p = surface(xs, ys, f, legend=false, fillalpha=0.5) + + ## we add to the graphic p, then plot + zs = [f(x,y) for x in xs, y in ys] # reverse order for use with Contour package + for cl in levels(contours(xs, ys, zs)) + lvl = level(cl) # the z-value of this contour level + for line in lines(cl) + _xs, _ys = coordinates(line) # coordinates of this line segment + _zs = offset * _xs + plot!(p, _xs, _ys, _zs, alpha=0.5) # add curve on x-y plane + end + end + p +end + + +offset = 0 +us = vs = range(-1, 2, length=100) +surface_contour(us, vs, f₂, offset=offset) +pts = [[pt..., offset] for pt in xs₂] +scatter3d!(unzip(pts)...) +plot!(unzip(pts)..., linewidth=3) +@show 314 +function peaks(x, y) + z = 3 * (1 - x)^2 * exp(-x^2 - (y + 1)^2) + z += -10 * (x / 5 - x^3 - y^5) * exp(-x^2 - y^2) + z += -1/3 * exp(-(x+1)^2 - y^2) + return z +end +peaks(v) = peaks(v...) +@show 315 +#| hold: true +xs = range(-3, stop=3, length=100) +ys = range(-2, stop=2, length=100) +Ps = surface(xs, ys, peaks, legend=false) +Pc = contour(xs, ys, peaks, legend=false) +plot(Ps, Pc, layout=2) # combine plots +@show 319 +function newton_stepₚ(f, x) + M = ForwardDiff.hessian(f, x) + b = ForwardDiff.gradient(f, x) + x - M \ b +end +@show 321 +xₚ = [0, 1.5] +xₚ = newton_stepₚ(peaks, xₚ) +xₚ = newton_stepₚ(peaks, xₚ) +xₚ = newton_stepₚ(peaks, xₚ) +xₚ, ForwardDiff.gradient(peaks, xₚ) +@show 323 +Hₚ = ForwardDiff.hessian(peaks, xₚ) +@show 325 +#| hold: true +fxx = Hₚ[1,1] +d = det(Hₚ) +fxx, d +@show 335 +#| hold: true +g(x,y) = x^2 + 2y^2 -1 +g(v) = g(v...) + +xs = range(-3, 3, length=100) +ys = range(-1, 4, length=100) + +p = plot(aspect_ratio=:equal, legend=false) +contour!(xs, ys, g, levels=[0]) + +gi(x) = sqrt(1/2*(1-x^2)) # solve for y in terms of x +pts = [[x, gi(x)] for x in (-3/4, -1/4, 1/4, 3/4)] + +for pt in pts + arrow!(pt, ForwardDiff.gradient(g, pt) ) +end + +p +@show 338 +#| hold: true +#| echo: false +r(t) = [cos(t), sin(t)/2] +plot_parametric(pi/12..pi/3, r, legend=false, aspect_ratio=true, linewidth=3) +T(t) = -r'(t) / norm(r'(t)) +No(t) = T'(t) / norm(T'(t)) +t = pi/4 +lambda=1/10 +scatter!(unzip([r(t)])...) +arrow!(r(t), T(t)*lambda) +arrow!(r(t), No(t)* lambda) + +f(x,y)= x^2 + y^2 +f(v) = f(v...) +arrow!(r(t), lambda*ForwardDiff.gradient(f, r(t))) + +xs = range(0.5,1, length=100) +ys = range(0.1, 0.5, length=100) +contour!(xs, ys, f) +@show 344 +#| hold: true +#| echo: false +r(t) = [cos(t), sin(t)/2] +plot_parametric(-pi/6..pi/6,r, legend=false, aspect_ratio=true, linewidth=3) +T(t) = -r'(t) / norm(r'(t)) +No(t) = T'(t) / norm(T'(t)) +t = 0 +lambda=1/10 +scatter!(unzip([r(t)])...) +arrow!(r(t), T(t)*lambda) +arrow!(r(t), No(t)* lambda) + +f(x,y)= x^2 + y^2 +f(v) = f(v...) +arrow!(r(t), lambda*ForwardDiff.gradient(f, r(t))) + +xs = range(0.5,1.5, length=100) +ys = range(-0.5, 0.5, length=100) +contour!(xs, ys, f, levels = [.7, .85, 1, 1.15, 1.3]) +@show 381 +@syms lambda +fₗₐ(x, y) = x^2 - y^2 +gₗₐ(x, y) = x^2 + y^2 +Lₗₐ(x, y, lambda) = fₗₐ(x,y) - lambda * (gₗₐ(x,y) - 1) +dsₗₐ = solve(diff.(Lₗₐ(x, y, lambda), [x, y, lambda])) +@show 383 +[fₗₐ(d[x], d[y]) for d in dsₗₐ] +@show 432 +#| hold: true +@syms y y′ λ C +ex = Eq(-λ*y′^2/sqrt(1 + y′^2) + λ*sqrt(1 + y′^2), y - C) +Δ = sqrt(1 + y′^2) / (y - C) +ex1 = Eq(simplify(ex.lhs()*Δ), simplify(ex.rhs() * Δ)) +ex2 = Eq(ex1.lhs()^2 - 1, simplify(ex1.rhs()^2) - 1) +@show 457 +@syms z lambda1 lambda2 +g1(x, y, z) = x^2 + y^2 - z^2 +g2(x, y, z) = x - 2z - 3 +fₘ(x,y,z)= x^2 + y^2 + z^2 +Lₘ(x,y,z,lambda1, lambda2) = fₘ(x,y,z) - lambda1*(g1(x,y,z) - 0) - lambda2*(g2(x,y,z) - 0) + +∇Lₘ = diff.(Lₘ(x,y,z,lambda1, lambda2), [x, y, z,lambda1, lambda2]) +@show 459 +solve(subs.(∇Lₘ, lambda1 .=> 1)) +@show 461 +outₘ = solve(subs.(∇Lₘ, y .=> 0)) +@show 463 +[fₘ(d[x], 0, d[z]) for d in outₘ] +@show 498 +struct MultiIndex + alpha::Vector{Int} + end +Base.show(io::IO, α::MultiIndex) = println(io, "α = ($(join(α.alpha, ", ")))") + +## |α| = α_1 + ... + α_m +Base.length(α::MultiIndex) = sum(α.alpha) + +## factorial(α) computes α! +Base.factorial(α::MultiIndex) = prod(factorial(Sym(a)) for a in α.alpha) + +## x^α = x_1^α_1 * x_2^α^2 * ... * x_n^α_n +import Base: ^ +^(x, α::MultiIndex) = prod(u^a for (u,a) in zip(x, α.alpha)) + +## ∂^α(ex) = ∂_1^α_1 ∘ ∂_2^α_2 ∘ ... ∘ ∂_n^α_n (ex) +partial(ex::SymPy.SymbolicObject, α::MultiIndex, vars=free_symbols(ex)) = diff(ex, zip(vars, α.alpha)...) +@show 499 +@syms w +alpha = MultiIndex([1,2,1,3]) +length(alpha) # 1 + 2 + 1 + 3=7 +[1,2,3,4]^alpha +exₜ = x^3 * cos(w*y*z) +partial(exₜ, alpha, [w,x,y,z]) +@show 501 +struct MultiIndices + n::Int + k::Int +end + +function Base.length(as::MultiIndices) + n,k = as.n, as.k + n == 1 && return 1 + sum(length(MultiIndices(n-1, j)) for j in 0:k) # recursively identify length +end + +function Base.iterate(alphas::MultiIndices) + k, n = alphas.k, alphas.n + n == 1 && return ([k],(0, MultiIndices(0,0), nothing)) + + m = zeros(Int, n) + m[1] = k + betas = MultiIndices(n-1, 0) + stb = iterate(betas) + st = (k, MultiIndices(n-1, 0), stb) + return (m, st) +end + +function Base.iterate(alphas::MultiIndices, st) + + st == nothing && return nothing + k,n = alphas.k, alphas.n + k == 0 && return nothing + n == 1 && return nothing + + # can we iterate the next on + bk, bs, stb = st + + if stb==nothing + bk = bk-1 + bk < 0 && return nothing + bs = MultiIndices(bs.n, bs.k+1) + val, stb = iterate(bs) + return (vcat(bk,val), (bk, bs, stb)) + end + + resp = iterate(bs, stb) + if resp == nothing + bk = bk-1 + bk < 0 && return nothing + bs = MultiIndices(bs.n, bs.k+1) + val, stb = iterate(bs) + return (vcat(bk, val), (bk, bs, stb)) + end + + val, stb = resp + return (vcat(bk, val), (bk, bs, stb)) + +end +@show 503 +collect(MultiIndices(2, 3)) +@show 505 +union((collect(MultiIndices(2, i)) for i in 0:3)...) +@show 507 +k = 4 +length(MultiIndices(3, k+1)) +@show 509 +#| hold: true +@syms 𝐅() a[1:3] dx[1:3] + +sum(partial(𝐅(a...), α, a) / factorial(α) * dx^α for k in 0:3 for α in MultiIndex.(MultiIndices(3, k))) # 3rd order +@show 513 +#| hold: true +#| echo: false +f(x,y) = sqrt(x + y) +f(v) = f(v...) +pt = [2,2] +dxdy = [.1, .2] +val = f(pt) + dot(ForwardDiff.gradient(f, pt), dxdy) +numericq(val) +@show 516 +#| hold: true +#| echo: false +f(x,y,z) = x*y + y*z + z*x +f(v) = f(v...) +pt = [1,1,1] +dx = [0.1, 0.0, -0.1] +val = f(pt) + ∇(f)(pt) ⋅ dx +numericq(val) +@show 519 +#| hold: true +#| echo: false +f(x,y,z) = x*y + y*z + z*x - 8 +f(v) = f(v...) +pt = [1,1,1] +n = ∇(f)(pt) +d = dot(n, pt) +choices = [ + raw"`` x + y + z = 3``", + raw"`` 2x + y - 2z = 1``", + raw"`` x + 2y + 3z = 6``" +] +answ = 1 +radioq(choices, answ) +@show 523 +#| hold: true +#| echo: false +choices = [ + raw"`` \langle 2xy + y^2 + y, 2xy + x^2 + x\rangle``", + raw"`` y^2 + y, x^2 + x``", + raw"`` \langle 2y + y^2, 2x + x^2``" +] +answ = 1 +radioq(choices, answ) +@show 527 +#| hold: true +#| echo: false +yesnoq(true) +@show 529 +#| hold: true +#| echo: false +f(x,y) = x*y + x*y^2 + x^2 * y +f(v) = f(v...) +val = det(ForwardDiff.hessian(f, [-1/3, -1/3])) +numericq(val) +@show 531 +#| hold: true +#| echo: false +choices = [ + L"The function $f$ has a local minimum, as $f_{xx} > 0$ and $d >0$", + L"The function $f$ has a local maximum, as $f_{xx} < 0$ and $d >0$", + L"The function $f$ has a saddle point, as $d < 0$", + L"Nothing can be said, as $d=0$" +] +answ = 2 +radioq(choices, answ, keep_order=true) +@show 535 +#| hold: true +#| results: "hidden" +f(x,y) = x + 2x^2 + x^3 + y + 2x*y + y^2 +@syms x::real y::real +gradf = gradient(f(x,y), [x,y]) +@show 536 +#| hold: true +#| echo: false +yesnoq(true) +@show 538 +#| hold: true +#| results: "hidden" +f(x,y) = x + 2x^2 + x^3 + y + 2x*y + y^2 +@syms x::real y::real +gradf = gradient(f(x,y), [x,y]) + +solve(gradf, [x,y]) +@show 539 +#| hold: true +#| echo: false +numericq(2) +@show 541 +#| hold: true +f(x,y) = x + 2x^2 + x^3 + y + 2x*y + y^2 +@syms x::real y::real +gradf = gradient(f(x,y), [x,y]) + +sympy.hessian(f(x,y), [x,y]) +@show 543 +#| hold: true +#| echo: false +choices = [ + L"The function $f$ has a local minimum, as $f_{xx} > 0$ and $d >0$", + L"The function $f$ has a local maximum, as $f_{xx} < 0$ and $d >0$", + L"The function $f$ has a saddle point, as $d < 0$", + L"Nothing can be said, as $d=0$", + L"The test does not apply, as $\nabla{f}$ is not $0$ at this point." +] +answ = 3 +radioq(choices, answ, keep_order=true) +@show 545 +#| hold: true +#| echo: false +choices = [ + L"The function $f$ has a local minimum, as $f_{xx} > 0$ and $d >0$", + L"The function $f$ has a local maximum, as $f_{xx} < 0$ and $d >0$", + L"The function $f$ has a saddle point, as $d < 0$", + L"Nothing can be said, as $d=0$", + L"The test does not apply, as $\nabla{f}$ is not $0$ at this point." +] +answ = 1 +radioq(choices, answ, keep_order=true) +@show 547 +#| hold: true +#| echo: false +choices = [ + L"The function $f$ has a local minimum, as $f_{xx} > 0$ and $d >0$", + L"The function $f$ has a local maximum, as $f_{xx} < 0$ and $d >0$", + L"The function $f$ has a saddle point, as $d < 0$", + L"Nothing can be said, as $d=0$", + L"The test does not apply, as $\nabla{f}$ is not $0$ at this point." +] +answ = 5 +radioq(choices, answ, keep_order=true) +@show 553 +#| hold: true +#| echo: false +yesnoq(true) +@show 557 +#| hold: true +#| echo: false +yesnoq(false) +@show 559 +#| hold: true +#| echo: false +choices =[ + "It is the determinant of the Hessian", + L"It isn't, $b^2-4ac$ is from the quadratic formula" +] +answ = 1 +radioq(choices, answ) +@show 561 +#| hold: true +#| echo: false +choices = [ + L"That $a>0$ and $4ac-b^2 > 0$", + L"That $a<0$ and $4ac-b^2 > 0$", + L"That $4ac-b^2 < 0$" +] +answ = 2 +radioq(choices, answ, keep_order=true) +@show 563 +#| hold: true +#| echo: false +choices = [ + L"That $a>0$ and $4ac-b^2 > 0$", + L"That $a<0$ and $4ac-b^2 > 0$", + L"That $4ac-b^2 < 0$" +] +answ = 3 +radioq(choices, answ, keep_order=true) +@show 569 +#| hold: true +#| echo: false +yesnoq(true) +@show 571 +#| echo: false +choices = [ + raw"`` \langle 2x, 2y\rangle``", + raw"`` \langle 2x, y^2\rangle``", + raw"`` \langle x^2, 2y \rangle``" +] +answ = 1 +radioq(choices, answ) +@show 573 +f(x,y) = exp(-x^2-y^2) * (2x^2 + y^2) +f(v) = f(v...) +r(t) = sqrt(3)*[cos(t), sin(t)] +rat(x) = abs(x[1]/x[2]) - 1 +fn = rat ∘ ∇(f) ∘ r +ts = fzeros(fn, 0, 2pi) +@show 575 +#| eval: false +#| echo: false +f(x,y) = exp(-x^2-y^2) * (2x^2 + y^2) +r(t) = sqrt(3)*[cos(t), sin(t)] +rat(x) = abs(x[1]/x[2]) - 1 +fn = rat ∘ ∇(splat(f)) ∘ r +ts = fzeros(fn, 0, 2pi) + +val = maximum((splat(u)∘r).(ts)) +numericq(val) diff --git a/quarto/differentiable_vector_calculus/test.qmd b/quarto/differentiable_vector_calculus/test.qmd new file mode 100644 index 0000000..37ab0eb --- /dev/null +++ b/quarto/differentiable_vector_calculus/test.qmd @@ -0,0 +1,98 @@ +# Applications with scalar functions + + +{{< include ../_common_code.qmd >}} + +This section uses these add-on packages: + + +```{julia} +using CalculusWithJulia +using Plots +plotly() +using SymPy +using Roots +``` + +##### Example + + +Consider the function $f(x,y) = x^2 + 3y^2 -x$ over the region $x^2 + y^2 \leq 1$. This is a continuous function over a closed set, so will have both an absolute maximum and minimum. Find these from an investigation of the critical points and the boundary points. + + +The gradient is easily found: $\nabla{f} = \langle 2x - 1, 6y \rangle$, and is $\vec{0}$ only at $\vec{a} = \langle 1/2, 0 \rangle$. The Hessian is: + + +$$ +H = +\begin{bmatrix} +2 & 0\\ +0 & 6 +\end{bmatrix}. +$$ + +At $\vec{a}$ this has positive determinant and $f_{xx} > 0$, so $\vec{a}$ corresponds to a *local* minimum with values $f(\vec{a}) = (1/2)^2 + 3(0) - 1/2 = -1/4$. The absolute maximum and minimum may occur here (well, not the maximum) or on the boundary, so that must be considered. In this case we can easily parameterize the boundary and turn this into the univariate case: + + +```{julia} +fₗ(x,y) = x^2 + 2y^2 - x +gammaₗ(t) = [cos(t), sin(t)] # traces out x^2 + y^2 = 1 over [0, 2pi] +gₗ = splat(fₗ) ∘ gammaₗ + +cpsₗ = find_zeros(gₗ', 0, 2pi) # critical points of g +append!(cpsₗ, [0, 2pi]) +unique!(cpsₗ) +gₗ.(cpsₗ) +``` + +We see that maximum value is `2.25` and that the interior point, $\vec{a}$, will be where the minimum value occurs. To see exactly where the maximum occurs, we look at the values of gamma: + +```{julia} +inds = [2,4] +cpsₗ[inds] +``` + +These are multiples of $\pi$: + + +```{julia} +cpsₗ[inds]/pi +``` + +So we have the maximum occurs at the angles $2\pi/3$ and $4\pi/3$. Here we visualize, using a hacky trick of assigning `NaN` values to the function to avoid plotting outside the circle: + + +```{julia} +hₗ(x,y) = fₗ(x,y) * (x^2 + y^2 <= 1 ? 1 : NaN) +``` + +```{julia} +gr() +``` + +```{julia} +#| hold: true +xs = ys = range(-1,1, length=100) +plt = surface(xs, ys, hₗ) + +ts = cpsₗ # 2pi/3 and 4pi/3 by above +xs, ys = cos.(ts), sin.(ts) +zs = fₗ.(xs, ys) +scatter3d!(xs, ys, zs) +#tuple.(xs, ys, zs) +#plot!(plt, tuple.(xs, ys, zs); linetype=:scatter) +#plt +``` + +A contour plot also shows that some---and only one---extrema happens on the interior: + + diff --git a/quarto/differentiable_vector_calculus/vector_fields.qmd b/quarto/differentiable_vector_calculus/vector_fields.qmd index 70136e0..c315d8a 100644 --- a/quarto/differentiable_vector_calculus/vector_fields.qmd +++ b/quarto/differentiable_vector_calculus/vector_fields.qmd @@ -9,7 +9,7 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia using Plots -plotly() +#plotly() using SymPy using ForwardDiff using LinearAlgebra @@ -454,7 +454,7 @@ surface(unzip(surf.(ts, θs'))...; legend=false) ```{julia} #| echo: false -plotly() +#plotly() nothing ``` From 253295ff6e05e91ff0dc61a8bc94edf56a9040af Mon Sep 17 00:00:00 2001 From: jverzani Date: Tue, 11 Aug 2026 17:17:08 -0400 Subject: [PATCH 4/7] lots of cleanup --- .gitignore | 9 + quarto/.gitignore | 1 + quarto/ODEs/Project.toml | 1 + quarto/ODEs/differential_equations.qmd | 178 +- quarto/ODEs/euler.qmd | 225 +- quarto/ODEs/odes.qmd | 28 +- quarto/ODEs/solve.qmd | 186 +- quarto/Project.toml | 1 + quarto/_common_code.qmd | 21 +- .../custom-callout/_extension.yml | 9 + .../custom-callout/assets/css/fontawesome.css | 7 + .../assets/webfonts/fa-solid-900.woff2 | Bin 0 -> 114740 bytes .../custom-callout/customcallout.lua | 299 ++ .../coatless-quarto/custom-callout/fa.lua | 2594 +++++++++++++++ quarto/_quarto.yml | 69 +- quarto/alternatives.qmd | 2 + quarto/alternatives/Project.toml | 3 + quarto/alternatives/SciML.qmd | 116 +- quarto/alternatives/makie_plotting.qmd | 668 ++-- quarto/alternatives/symbolics.qmd | 192 +- quarto/basics/Project.toml | 1 + quarto/basics/calculator.qmd | 166 +- quarto/basics/logical_expressions.qmd | 91 +- quarto/basics/numbers-types-orig.qmd | 685 ++++ quarto/basics/numbers_types-II.html | 2804 +++++++++++++++++ quarto/basics/numbers_types-II.qmd | 675 ++++ quarto/basics/numbers_types.qmd | 644 ++-- quarto/basics/ranges.qmd | 160 +- quarto/basics/variables.qmd | 51 +- quarto/basics/vectors.qmd | 1169 ++++--- quarto/curves/early-curvature.qmd | 161 + quarto/derivatives/curve_sketching.qmd | 274 +- quarto/derivatives/derivatives.qmd | 505 +-- .../derivatives/first_second_derivatives.qmd | 713 +++-- .../derivatives/implicit_differentiation.qmd | 156 +- quarto/derivatives/jsxgraph-newton.qmd | 95 + quarto/derivatives/lhospitals_rule.qmd | 63 +- quarto/derivatives/linearization.qmd | 305 +- quarto/derivatives/mean_value_theorem.qmd | 393 ++- quarto/derivatives/more_zeros.qmd | 772 +++-- quarto/derivatives/newtons_method.qmd | 925 +++--- quarto/derivatives/numeric_derivatives.qmd | 229 +- quarto/derivatives/optimization.qmd | 330 +- quarto/derivatives/related_rates.qmd | 144 +- quarto/derivatives/symbolic_derivatives.qmd | 19 +- .../derivatives/taylor_series_polynomials.qmd | 300 +- quarto/index.qmd | 10 +- quarto/integrals/arc_length.qmd | 291 +- quarto/integrals/area.qmd | 1224 ++----- quarto/integrals/area_between_curves.qmd | 889 ++++-- quarto/integrals/center_of_mass.qmd | 350 +- quarto/integrals/figures/ice-cream.jpg | Bin 0 -> 344901 bytes quarto/integrals/figures/johns-catenary-3.jpg | Bin 0 -> 30906 bytes .../integrals/figures/johns-catenary-orig.jpg | Bin 0 -> 66901 bytes quarto/integrals/figures/johns-catenary.jpg | Bin 18119 -> 30906 bytes quarto/integrals/ftc.qmd | 369 ++- quarto/integrals/improper_integrals.qmd | 194 +- quarto/integrals/integration_by_parts.qmd | 278 +- quarto/integrals/mean_value_theorem.qmd | 76 +- quarto/integrals/numeric_integrals.qmd | 861 +++++ quarto/integrals/orthogonal_polynomials.qmd | 91 +- quarto/integrals/partial_fractions.qmd | 32 +- quarto/integrals/substitution.qmd | 288 +- quarto/integrals/surface_area.qmd | 163 +- quarto/integrals/twelve-qs.qmd | 79 +- quarto/integrals/volumes_slice.qmd | 228 +- quarto/limits.qmd | 5 +- quarto/limits/Project.toml | 3 + quarto/limits/continuity.qmd | 149 +- quarto/limits/intermediate_value_theorem.qmd | 563 ++-- quarto/limits/limits.qmd | 457 +-- quarto/limits/limits_extensions.qmd | 284 +- quarto/limits/sequences_series.qmd | 214 +- quarto/misc/Project.toml | 2 + quarto/misc/getting_started_with_julia.qmd | 27 +- quarto/misc/julia_interfaces.qmd | 4 +- quarto/misc/quick_notes.qmd | 14 +- quarto/precalc/Project.toml | 1 + quarto/precalc/exp_log_functions.qmd | 128 +- quarto/precalc/functions.qmd | 197 +- quarto/precalc/inversefunctions.qmd | 159 +- quarto/precalc/julia_overview.qmd | 39 +- quarto/precalc/plotting.qmd | 477 +-- quarto/precalc/polynomial.qmd | 186 +- quarto/precalc/polynomial_roots.qmd | 264 +- quarto/precalc/polynomials_package.qmd | 178 +- quarto/precalc/rational_functions.qmd | 289 +- quarto/precalc/transformations.qmd | 394 ++- quarto/precalc/trig_functions.qmd | 249 +- quarto/references.bib | 11 + quarto/styles.css | 0 91 files changed, 18284 insertions(+), 7872 deletions(-) create mode 100644 quarto/_extensions/coatless-quarto/custom-callout/_extension.yml create mode 100644 quarto/_extensions/coatless-quarto/custom-callout/assets/css/fontawesome.css create mode 100644 quarto/_extensions/coatless-quarto/custom-callout/assets/webfonts/fa-solid-900.woff2 create mode 100644 quarto/_extensions/coatless-quarto/custom-callout/customcallout.lua create mode 100644 quarto/_extensions/coatless-quarto/custom-callout/fa.lua create mode 100644 quarto/basics/numbers-types-orig.qmd create mode 100644 quarto/basics/numbers_types-II.html create mode 100644 quarto/basics/numbers_types-II.qmd create mode 100644 quarto/curves/early-curvature.qmd create mode 100644 quarto/derivatives/jsxgraph-newton.qmd create mode 100644 quarto/integrals/figures/ice-cream.jpg create mode 100644 quarto/integrals/figures/johns-catenary-3.jpg create mode 100644 quarto/integrals/figures/johns-catenary-orig.jpg create mode 100644 quarto/integrals/numeric_integrals.qmd create mode 100644 quarto/styles.css diff --git a/.gitignore b/.gitignore index 761b201..28c53ce 100644 --- a/.gitignore +++ b/.gitignore @@ -17,3 +17,12 @@ default.profraw /*/bonepile.qmd /*/*/bonepile.qmd /*/*_files +/.cache +/*/.cache +/*/*/.cache +/*.log +/*/*.log +/*/*/*.log +/*.tex +/*/*.tex +/*/*/*.tex diff --git a/quarto/.gitignore b/quarto/.gitignore index ea9ed1f..3bef8ee 100644 --- a/quarto/.gitignore +++ b/quarto/.gitignore @@ -7,3 +7,4 @@ /*/references.bib weave_support.jl **/*.quarto_ipynb +/style-sheet* diff --git a/quarto/ODEs/Project.toml b/quarto/ODEs/Project.toml index b8ebcfe..c52df32 100644 --- a/quarto/ODEs/Project.toml +++ b/quarto/ODEs/Project.toml @@ -1,5 +1,6 @@ [deps] CalculusWithJulia = "a2e0e22d-7d4c-5312-9169-8b992201a882" +DiffEqBase = "2b5f629d-d688-5b77-993f-72d75c75574e" IJulia = "7073ff75-c697-5162-941a-fcdaad2a7d2a" LaTeXStrings = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f" ModelingToolkit = "961ee093-0014-501f-94e3-6117800e7a78" diff --git a/quarto/ODEs/differential_equations.qmd b/quarto/ODEs/differential_equations.qmd index faf815d..5b5ff45 100644 --- a/quarto/ODEs/differential_equations.qmd +++ b/quarto/ODEs/differential_equations.qmd @@ -8,15 +8,15 @@ This section uses these add-on packages: ```{julia} using OrdinaryDiffEq +using DiffEqBase: terminate! using Plots -using ModelingToolkit ``` --- -The [`DifferentialEquations`](https://github.com/SciML/DifferentialEquations.jl) suite of packages contains solvers for a wide range of various differential equations. This section just briefly touches on ordinary differential equations (ODEs), and so relies only on `OrdinaryDiffEq`, a small part of the suite. For more detail on this type and many others covered by the suite of packages, there are many other resources, including the [documentation](https://diffeq.sciml.ai/stable/) and accompanying [tutorials](https://github.com/SciML/SciMLTutorials.jl). +The [`DifferentialEquations`](https://github.com/SciML/DifferentialEquations.jl) suite of packages contains solvers for a wide range of various differential equations. This section just briefly touches on ordinary differential equations (ODEs), and so relies only on `OrdinaryDiffEq`, a small part of the suite. For more detail on this type and many others covered by the suite of packages, there are many other resources, including the [documentation](https://docs.sciml.ai/DiffEqDocs/stable/) and accompanying tutorials. ## SIR Model @@ -25,7 +25,7 @@ The [`DifferentialEquations`](https://github.com/SciML/DifferentialEquations.jl) We follow along with an introduction to the SIR model for the spread of disease by [Smith and Moore](https://www.maa.org/press/periodicals/loci/joma/the-sir-model-for-spread-of-disease-introduction). This model received a workout due to the COVID-19 pandemic. -The basic model breaks a population into three cohorts: The **susceptible** individuals, the **infected** individuals, and the **recovered** individuals. These add to the population size, $N$, which is fixed, but the cohort sizes vary in time. We name these cohort sizes $S(t)$, $I(t)$, and $R(t)$ and define $s(t)=S(t)/N$, $i(t) = I(t)/N$ and $r(t) = R(t)/N$ to be the respective proportions. +The basic model breaks a population into three cohorts: The **susceptible** individuals, the **infected** individuals, and the **recovered** individuals. These add up to the population size, $N$, which is fixed, but the cohort sizes vary in time. We name these cohort sizes $S(t)$, $I(t)$, and $R(t)$ and define $s(t)=S(t)/N$, $i(t) = I(t)/N$ and $r(t) = R(t)/N$ to be the respective proportions. The following *assumptions* are made about these cohorts by Smith and Moore: @@ -59,7 +59,7 @@ $$ which can also be expressed in proportions as $r'(t) = k \cdot i(t)$. -Finally, from $S(t) + I(T) + R(t) = N$ we have $S'(T) + I'(t) + R'(t) = 0$ or $s'(t) + i'(t) + r'(t) = 0$. +Finally, from $S(t) + I(T) + R(t) = N$ (a constant) we have $S'(T) + I'(t) + R'(t) = 0$ or $s'(t) + i'(t) + r'(t) = 0$. Combining, it is possible to express the rate of change of the infected population through: @@ -83,36 +83,40 @@ In `Julia` we define these parameter values and `N` to model the total populatio ```{julia} -S0, I0, R0 = 7_900_000, 10, 0 -N = S0 + I0 + R0 -u0 = [S0, I0, R0]/N # initial proportions +S₀, I₀, R₀ = 7_900_000, 10, 0 +N = S₀ + I₀ + R₀ +u0 = [S₀, I₀, R₀]/N # initial proportions ``` -An *estimated* set of values for $k$ and $b$ are $k=1/3$, coming from the average period of infectiousness being estimated at three days and $b=1/2$, which seems low in normal times, but not for an infected person who may be feeling quite ill and staying at home. (The model for COVID would certainly have a larger $b$ value). +An *estimated* set of values for $k$ and $b$ are $k=1/3$, coming from the average period of infectiousness being estimated at three days and $b=1/2$, which seems low in normal times, but not for an infected person who may be feeling quite ill and staying at home. (The model applied to COVID infections would certainly have a larger $b$ value). Okay, the mathematical modeling is done; now we try to solve for the unknown functions using `DifferentialEquations`. +### Warm up: no interaction To warm up, if $b=0$ then $i'(t) = -k \cdot i(t)$ describes the infected. (There is no circulation of people in this case.) This is a single ODE. The solution would be achieved through: - +::: {#fig-SIR-model-HK-data-b-0} ```{julia} #| hold: true k = 1/3 f(u,p,t) = -k * u # solving u′(t) = - k u(t) -uᵢ0= I0/N +uᵢ₀= I₀/N time_span = (0.0, 20.0) -prob = ODEProblem(f, uᵢ0, time_span) +prob = ODEProblem(f, uᵢ₀, time_span) sol = solve(prob, Tsit5(); reltol=1e-8, abstol=1e-8) plot(sol) ``` -The `sol` object is a set of numbers with a convenient `plot` method. As may have been expected, this graph shows exponential decay. +Plot of SIR model for the Hong Kong parameters when $b=0$ +::: + +The `sol` object is a set of numbers with a convenient `plot` method. As may have been expected, @fig-SIR-model-HK-data-b-0 shows steady exponential decay, as there is no mixing of infected with others. A few comments are in order. The problem we want to solve is @@ -125,20 +129,21 @@ $$ where $F$ depends on the current value ($i$), a parameter ($k$), and the time ($t$). We did not utilize $p$ above for the parameter, as it was easy not to, but could have, and will in the following. The time variable $t$ does not appear by itself in our equation, so only `f(u, p, t) = -k * u` was used, `u` the generic name for a solution which in this case was labeled with an $i$. -The problem we set up needs an initial value (the $u0$) and a time span to solve over. Here we want time to model real time, so use floating point values. +The problem we set up needs an initial value (the $u_0$) and a time span to solve over. Here we want time to model real time, so use floating point values. +Relating the key commands to the last section, `solve` generic function dispatches on the problem type (`ODEProblem`) and the method for approximation (declared with `Tsit5()`). Convergence parameters are passed through as keyword arguments. -The plot shows steady decay, as there is no mixing of infected with others. - +### Adding in interaction Adding in the interaction requires a bit more work. We now have what is known as a *system* of equations: + $$ -\begin{align*} -\frac{ds}{dt} &= -b \cdot s(t) \cdot i(t)\\ -\frac{di}{dt} &= b \cdot s(t) \cdot i(t) - k \cdot i(t)\\ -\frac{dr}{dt} &= k \cdot i(t)\\ -\end{align*} +\begin{flalign*} +\frac{ds}{dt} &=-b \cdot s(t) \cdot i(t) \\ +\frac{di}{dt} &= b \cdot s(t) \cdot i(t) - k \cdot i(t)\\ +\frac{dr}{dt} &= k \cdot i(t)\\ +\end{flalign*} $$ Systems of equations can be solved in a similar manner as a single ordinary differential equation, though adjustments are made to accommodate the multiple functions. @@ -165,13 +170,13 @@ The notation `du` is suggestive of both the derivative and a small increment. Th :::{.callout-note} ## Mutation not re-binding -The `sir!` function has the trailing `!` indicating – by convention – it *mutates* its first value, `du`. In this case, through an assignment, as in `du[1]=ds`. This could use some explanation. The *binding* `du` refers to the *container* holding the $3$ values, whereas `du[1]` refers to the first value in that container. So `du[1]=ds` changes the first value, but not the *binding* of `du` to the container. That is, `du` mutates. This would be quite different were the call `du = [ds,di,dr]` which would create a new *binding* to a new container and not mutate the values in the original container. +The `sir!` function has the trailing `!` indicating---by convention---it *mutates* its first value, `du`. In this case, through an assignment, as in `du[1]=ds`. This could use some explanation. The *binding* `du` refers to the *container* holding the $3$ values, whereas `du[1]` refers to the first value in that container. So `du[1]=ds` changes the first value, but not the *binding* of `du` to the container. That is, `du` mutates. This would be quite different were the call `du = [ds,di,dr]` which would create a new *binding* to a new container and not mutate the values in the original container. ::: With the update function defined, the problem is setup and a solution is found using the same manner as before: - +::: {#fig-sir-k-1-third-b-1-half} ```{julia} p = (k=1/3, b=1/2) # parameters time_span = (0.0, 150.0) # time span to solve over, 5 months @@ -180,15 +185,18 @@ prob = ODEProblem(sir!, u0, time_span, p) sol = solve(prob, Tsit5()) plot(sol) -plot!(x -> 0.5, linewidth=2) # mark 50% line +plot!(x -> 0.5, line=(2, :dot), label="50%") # mark 50% line ``` +Plot of SIR model when $k=1/3$ and $b=1/2$ showing over half the population is infected +::: + The lower graph shows the number of infected at each day over the five-month period displayed. The peak is around 6-7% of the population at any one time. However, over time the recovered part of the population reaches over 50%, meaning more than half the population is modeled as getting sick. Now we change the parameter $b$ and observe the difference. We passed in a value `p` holding our two parameters, so we just need to change that and run the model again: - +::: {#fig-sir-k-1-third-b-2} ```{julia} #| hold: true p = (k=1/2, b=2) # change b from 1/2 to 2 -- more daily contact @@ -198,12 +206,15 @@ sol = solve(prob, Tsit5()) plot(sol; legend=:right) ``` +Plot of SIR model when $k=1/3$ and $b=2$. With more interaction, nearly everyone is eventually infected. +::: + The graphs are somewhat similar, but the steady state is reached much more quickly and nearly everyone became infected. What about if $k$ were bigger? - +::: {#fig-sir-k-2-thirds-b-1-half} ```{julia} #| hold: true p = (k=2/3, b=1/2) @@ -213,6 +224,10 @@ sol = solve(prob, Tsit5()) plot(sol) ``` +Plot of SIR model when $k=2/3$ and $b=1/2$. The larger recovery rate ($k$) inhibits the proportion who are eventually infected. +::: + + The graphs show that under these conditions the infections never take off; we have $i' = (b\cdot s-k)i = k\cdot((b/k) s - 1) i$ which is always negative, since `(b/k)s < 1`, so infections will only decay. @@ -221,27 +236,31 @@ The solution object is indexed by time, then has the `s`, `i`, `r` estimates. We ```{julia} function recovered(k,b) - prob = ODEProblem(sir!, u0, time_span, (k,b)); - sol = solve(prob, Tsit5()); - s,i,r = last(sol) + prob = ODEProblem(sir!, u0, time_span, (; k,b)) + sol = solve(prob, Tsit5()) + s,i,r = last(sol.u) r end ``` This function makes it easy to see the impact of changing the parameters. For example, fixing $k=1/3$ we have: - +::: {#fig-SIR-model-recovered-function} ```{julia} f(b) = recovered(1/3, b) -plot(f, 0, 2) +plot(f, 0, 2; label="Recovered proportion") ``` +Plot of recovery proportion for $k=1/3$ and $b$ in $[0,2]$. As can be seen if $b$ is large enough, nearly every one in the proportion is infected and then recovered. +::: + + This very clearly shows the sharp dependence on the value of $b$; below some level, the proportion of people who are ever infected (the recovered cohort) remains near $0$; above that level it can climb quickly towards $1$. The function `recovered` is of two variables returning a single value. In subsequent sections we will see a few $3$-dimensional plots that are common for such functions, here we skip ahead and show how to visualize multiple function plots at once using "`z`" values in a graph. - +::: {#fig-recoverd-over-various-k-values} ```{julia} #| hold: true k, ks = 0.1, 0.2:0.1:0.9 # first `k` and then the rest @@ -254,6 +273,9 @@ end p ``` +Plot of recovery proportion for different values of $k$. When the recovery rate $k$ is larger, the value of $b$---number of daily contacts---must also be larger to see a significant proportion get infected and subsequently recovered. +::: + (A 3-dimensional graph with `plotly` or `Makie` can have its viewing angle adjusted with the mouse. When looking down on the $x-y$ plane, which code `b` and `k`, we can see the rapid growth along a line related to $b/k$.) @@ -279,16 +301,24 @@ This equation does not depend on $t$; $s$ is the dependent variable. It could be We now solve numerically the problem of a trajectory with a drag force from air resistance. -The general model is: +The general model is (with $W = W(t,x(t), x'(t), y(t), y'(t))$ ): $$ \begin{align*} -x''(t) &= - W(t,x(t), x'(t), y(t), y'(t)) \cdot x'(t)\\ -y''(t) &= -g - W(t,x(t), x'(t), y(t), y'(t)) \cdot y'(t)\\ +x''(t) &= - W \cdot x'(t)\\ +y''(t) &= -g - W \cdot y'(t)\\ \end{align*} $$ -with initial conditions: $x(0) = y(0) = 0$ and $x'(0) = v_0 \cos(\theta), y'(0) = v_0 \sin(\theta)$. +with initial conditions: + +$$ +\begin{align*} +x(0) &= y(0) = 0, \\ +x'(0) &= v_0 \cos(\theta),\\ +y'(0) &= v_0 \sin(\theta). +\end{align*} +$$ This is turned into an ODE by a standard trick. Here we define our function for updating a step. As can be seen the vector `u` contains both $\langle x,y \rangle$ and $\langle x',y' \rangle$ @@ -296,7 +326,7 @@ This is turned into an ODE by a standard trick. Here we define our function for ```{julia} function xy!(du, u, p, t) - g, γ = p.g, p.k + (; g, γ) = p x, y = u[1], u[2] x′, y′ = u[3], u[4] # unicode \prime[tab] @@ -304,7 +334,7 @@ function xy!(du, u, p, t) du[1] = x′ du[2] = y′ - du[3] = 0 - W * x′ + du[3] = - W * x′ du[4] = -g - W * y′ end ``` @@ -344,74 +374,50 @@ This allows us to define an `ODEProblem`: trajectory_problem = ODEProblem(xy!, INITIAL, TSPAN) ``` -When $\gamma = 0$ there should be no drag and we expect to see a parabola: - +When $\gamma = 0$ there should be no drag and we expect to see a parabola, as we can in @fig-windy-day-gamma-0. +::: {#fig-windy-day-gamma-0} ```{julia} #| hold: true -ps = (g=9.8, k=0) -SOL = solve(trajectory_problem, Tsit5(); p = ps) +p = (g=9.8, γ=0) +SOL = solve(trajectory_problem, Tsit5(); p) -plot(t -> SOL(t)[1], t -> SOL(t)[2], TSPAN...; legend=false) +plot(t -> SOL(t)[1], t -> SOL(t)[2], TSPAN...; label="γ=0") ``` +Parametric plot of trajectory when $\gamma=0$ (no drag). The resulting shape is a parabola. +::: -The plot is a parametric plot of the $x$ and $y$ parts of the solution over the time span. We can see the expected parabolic shape. + +The plot is a *parametric plot* of the $x$ and $y$ parts of the solution over the time span. We can see the expected parabolic shape. On a *windy* day, the value of $k$ would be positive. Repeating the above with $k=1/4$ gives: - +::: {#fig-windy-day-gamma-1-fourth} ```{julia} #| hold: true -ps = (g=9.8, k=1/4) -SOL = solve(trajectory_problem, Tsit5(); p = ps) +p = (g=9.8, γ=1/4) +SOL = solve(trajectory_problem, Tsit5(); p) -plot(t -> SOL(t)[1], t -> SOL(t)[2], TSPAN...; legend=false) +plot(t -> SOL(t)[1], t -> SOL(t)[2], TSPAN...; label="γ=1/4") ``` - -We see that the $y$ values have gone negative. The `DifferentialEquations` package can adjust for that with a *callback* which terminates the problem once $y$ has gone negative. This can be implemented as follows: +Parametric plot of trajectory when $\gamma=1/4$ (some drag). The resulting shape is skewed. +::: +We see that the $y$ values have gone negative. In the `DiffEqBase` package that can be adjusted with a *callback* that terminates the problem once $y$ has gone negative. For example: + +::: {#fig-windy-day-gamma-1-fourth-callback} ```{julia} #| hold: true condition(u,t,integrator) = u[2] # called when `u[2]` is negative -affect!(integrator) = terminate!(integrator) # stop the process -cb = ContinuousCallback(condition, affect!) +affect!(integrator) = terminate!(integrator) # stop the process DiffEqBase.terminate! +callback = ContinuousCallback(condition, affect!) -ps = (g=9.8, k = 1/4) -SOL = solve(trajectory_problem, Tsit5(); p = ps, callback=cb) +p = (g=9.8, γ=1/4) +SOL = solve(trajectory_problem, Tsit5(); p, callback) -plot(t -> SOL(t)[1], t -> SOL(t)[2], TSPAN...; legend=false) +plot(t -> SOL(t)[1], t -> SOL(t)[2], TSPAN...; label="γ=1/4") ``` - -Finally, we note that the `ModelingToolkit` package provides symbolic-numeric computing. This allows the equations to be set up symbolically, as has been illustrated with `SymPy`, before being passed off to `DifferentialEquations` to solve numerically. The above example with no wind resistance could be translated into the following: - - -```{julia} -#| hold: true -@parameters γ g -@variables t x(t) y(t) -D = Differential(t) - -eqs = [D(D(x)) ~ -γ * D(x), - D(D(y)) ~ -g - γ * D(y)] - -@named sys = ODESystem(eqs, t, [x,y], [γ,g]) -sys = ode_order_lowering(sys) # turn 2nd order into 1st -sys = structural_simplify(sys) - -u0 = [D(x) => vxy₀[1], - D(y) => vxy₀[2], - x => 0.0, - y => 0.0] - -p = [γ => 0.0, - g => 9.8] - -prob = ODEProblem(sys, u0, TSPAN, p, jac=true) -sol = solve(prob,Tsit5()) - -plot(t -> sol(t)[1], t -> sol(t)[3], TSPAN..., legend=false) -``` - -The toolkit will automatically generate fast functions and can perform transformations (such as is done by `ode_order_lowering`) before passing along to the numeric solves. +Parametric plot of trajectory_problem when $\gamma=1/4$ using a callback to avoid $y$ values going negative. +::: diff --git a/quarto/ODEs/euler.qmd b/quarto/ODEs/euler.qmd index 356aab4..682ec9e 100644 --- a/quarto/ODEs/euler.qmd +++ b/quarto/ODEs/euler.qmd @@ -1,5 +1,12 @@ # Euler's method +::: {#fig-euler-publication} +![](./figures/euler.png) + +Figure from first publication of Euler's method. From [Gander and Wanner](http://www.unige.ch/~gander/Preprints/Ritz.pdf). +::: + + {{< include ../_common_code.qmd >}} @@ -8,7 +15,7 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia -using Plots +using Plots; plotly() using SymPy using Roots ``` @@ -17,7 +24,7 @@ using Roots --- -The following section takes up the task of numerically approximating solutions to differential equations. `Julia` has a huge set of state-of-the-art tools for this task starting with the [DifferentialEquations](https://github.com/SciML/DifferentialEquations.jl) package. We don't use that package in this section, focusing on simpler methods and implementations for pedagogical purposes, but any further exploration should utilize the tools provided therein. A brief introduction to the package follows in an upcoming [section](./differential_equations.html). +The following section takes up the task of numerically approximating solutions to some ordinary differential equations. `Julia` has a huge set of state-of-the-art tools for this task starting with the [DifferentialEquations](https://github.com/SciML/DifferentialEquations.jl) package. We don't use that package in this section, focusing on simpler methods and implementations for pedagogical purposes, but any further exploration should utilize the tools provided therein. A brief introduction to the package follows in an upcoming [section](./differential_equations.html). --- @@ -49,24 +56,24 @@ With the given initial condition, the solution becomes: out = dsolve(D(u)(x) - F(u(x),x), u(x), ics=Dict(u(x0) => y0)) ``` -Plotting this solution over the slope field +Plotting this solution over the slope field, as in @fig-plot-soln-to-ode-with-vectorfield-plot-tangent-to-integral-curve we see that the vectors that are drawn seem to be tangent to the graph of the solution. This is no coincidence, the tangent lines to integral curves are in the direction of the slope field. +::: {#fig-plot-soln-to-ode-with-vectorfield-plot-tangent-to-integral-curve} ```{julia} p = plot(legend=false) vectorfieldplot!((x,y) -> [1, F(x,y)], xlims=(0, 2.5), ylims=(0, 10)) plot!(rhs(out), linewidth=5) ``` +Plot of a vector field for $F(y,x)$ and a solution to $y'=F(y,x)$. +::: -we see that the vectors that are drawn seem to be tangent to the graph of the solution. This is no coincidence, the tangent lines to integral curves are in the direction of the slope field. - - -What if the graph of the solution were not there, could we use this fact to *approximately* reconstruct the solution? +What if the graph of the solution were not in @fig-plot-soln-to-ode-with-vectorfield-plot-tangent-to-integral-curve, could we use this fact about tangency to *approximately* reconstruct the solution? That is, if we stitched together pieces of the slope field, would we get a curve that was close to the actual answer? - +::: {#fig-animation-of-stitching-together-tangents} ```{julia} #| hold: true #| echo: false @@ -112,15 +119,16 @@ imgfile = tempname() * ".gif" gif(anim, imgfile, fps = 1) -caption = """ -Illustration of a function stitching together slope field lines to -approximate the answer to an initial-value problem. The other function drawn is the actual solution. -""" - +caption = "" ImageFile(imgfile, caption) ``` -The illustration suggests the answer is yes, let's see. The solution is drawn over $x$ values $1$ to $2$. Let's try piecing together $5$ pieces between $1$ and $2$ and see what we have. +Illustration of a function stitching together slope field lines to +approximate the answer to an initial-value problem. The other function drawn is the actual solution. +::: + + +The animation in @fig-animation-of-stitching-together-tangents The suggests the answer is yes, let's see. The solution is drawn over $x$ values $1$ to $2$. Let's try piecing together $5$ pieces between $1$ and $2$ and see what we have. The slope-field vectors are *scaled* versions of the vector `[1, F(y,x)]`. The `1` is the part in the direction of the $x$ axis, so here we would like that to be $0.2$ (which is $(2-1)/5$. So our vectors would be `0.2 * [1, F(y,x)]`. To allow for generality, we use `h` in place of the specific value $0.2$. @@ -147,10 +155,10 @@ $$ x_2 = x_1 + h, \quad y_2 = y_1 + h F(y_1, x_1). $$ -We just shifted the indices forward by $1$. But graphically what is this? It takes the tip of the first part of our "stitched" together solution, finds the slope filed there (`[1, F(y,x)]`) and then uses this direction to stitch together one more piece. +We just shifted the indices forward by $1$. But graphically what is this? It takes the tip of the first part of our "stitched" together solution, finds the slope field there (`[1, F(y,x)]`) and then uses this direction to stitch together one more piece. -Clearly, we can repeat. The $n$th piece will end at: +Clearly, we can repeat. The $n+1$st piece will end at: $$ @@ -176,14 +184,18 @@ for i in 1:n end ``` -So how did we do? Let's look graphically: +So how did we do? @fig-euler-yp-yx-n-5 shows the graph of the exact answer and the stiched-together answer. +::: {#fig-euler-yp-yx-n-5} ```{julia} -plot(exp(-1/2)*exp(x^2/2), x0, 2) -plot!(xs, ys) +plot(exp(-1/2)*exp(x^2/2), x0, 2; label="Exact") +plot!(xs, ys; label="euler, n=5") ``` +Plot of an exact solution and an approximate solution using $5$ steps to the differential equationss $y'=y\cdot x$. +::: + Not bad. We wouldn't expect this to be exact---due to the concavity of the solution, each step is an underestimate. However, we see it is an okay approximation and would likely be better with a smaller $h$. A topic we pursue in just a bit. @@ -231,19 +243,20 @@ With `euler`, it becomes easy to explore different values. For example, we thought the solution would look better with a smaller $h$ (or larger $n$). Instead of $n=5$, let's try $n=50$: +::: {#fig-euler-yp-yx-n-50} ```{julia} u12 = euler(F, 1, 2, 1, 50) -plot(exp(-1/2)*exp(x^2/2), x0, 2) -plot!(u12, x0, 2) +plot(exp(-1/2)*exp(x^2/2), x0, 2; label="Exact") +plot!(u12, x0, 2; label="euler, n=50") ``` +Plot of an exact solution and an approximate solution using $50$ steps to the differential equation $y'=y\cdot x$. +::: It is more work for the computer, but not for us, and clearly a much better approximation to the actual answer is found. ## The Euler method -![Figure from first publication of Euler's method. From [Gander and Wanner](http://www.unige.ch/~gander/Preprints/Ritz.pdf).](./figures/euler.png) - The name of our function reflects the [mathematician](https://en.wikipedia.org/wiki/Leonhard_Euler) associated with the iteration: @@ -266,11 +279,7 @@ The total error, or more commonly, *global truncation error*, is the error betwe Other, somewhat more complicated, methods have global truncation errors that involve higher powers of $h$---that is for the same size $h$, the error is smaller. In analogy is the fact that Riemann sums have error that depends on $h$, whereas other methods of approximating the integral have smaller errors. For example, Simpson's rule had error related to $h^4$. So, the Euler method may not be employed if there is concern about total resources (time, computer, ...), it is important for theoretical purposes in a manner similar to the role of the Riemann integral. -In the examples, we will see that for many problems the simple Euler method is satisfactory, but not always so. The task of numerically solving differential equations is not a one-size-fits-all one. In the following, a few different modifications are presented to the basic Euler method, but this just scratches the surface of the topic. - - -#### Examples - +In the examples, we will see that for many problems the simple Euler method is satisfactory, but not always so. The task of numerically solving differential equations is not a one-size-fits-all one. In the following, a few different modifications are presented to the basic Euler method, but what is presented just scratches the surface of the topic. ##### Example @@ -287,14 +296,17 @@ f(xn) We graphically compare our approximate answer with the exact one: - +::: {#fig-euler-yp-x-plus-y} ```{julia} -𝒐ut = dsolve(D(u)(x) - F(u(x),x), u(x), ics = Dict(u(x0) => y0)) -plot(rhs(𝒐ut), x0, xn) -plot!(f, x0, xn) +out = dsolve(D(u)(x) - F(u(x),x), u(x), ics = Dict(u(x0) => y0)) +plot(rhs(out), x0, xn; label="Exact") +plot!(f, x0, xn; label="euler, n=25") ``` -From the graph it appears our value for `f(xn)` will underestimate the actual value of the solution slightly. +Plot of an exact solution to $y'=x + y$ and an approximate one using `euler` with $n=25$ +::: + +From @fig-euler-yp-x-plus-y it appears our value for `f(xn)` will underestimate the actual value of the solution slightly. ##### Example: the power series method and Euler @@ -321,23 +333,25 @@ out1 = dsolve(eqn, u(x), ics=Dict(u(0) => 1), hint="1st_power_series") The approximate value given by the Euler method is - +::: {#fig-approx-soln-to-yp-sin-x-y} ```{julia} x0, xn, y0 = 0, 2, 1 -plot(legend=false) +plot() vectorfieldplot!((x,y) -> [1, F(y,x)], xlims=(x0, xn), ylims=(0,5)) -plot!(rhs(out1).removeO(), linewidth=5) +plot!(rhs(out1).removeO(); line=(5, :dash), label="SymPy power series") u = euler(F, x0, xn, y0, 10) -plot!(u, linewidth=5) +plot!(u; line=(5, :dot), label="euler, n=10") ``` +Plot of a power-series solution to $y'=\sin(y,x)$ returned by `SymPy` and an approximate one using `euler` with $n=25$ +::: We see that the answer found from using a polynomial series matches that of Euler's method for a bit, but as time evolves, the approximate solution given by Euler's method more closely tracks the slope field. ---- -The [power series method](https://en.wikipedia.org/wiki/Power_series_solution_of_differential_equations) to solve a differential equation starts with an assumption that the solution can be represented as a power series with some positive radius of convergence. This is formally substituted into the differential equation and derivatives may be taken term by term. The resulting coefficients are equated for like powers giving a system of equations to be solved. +The [power series method](https://en.wikipedia.org/wiki/Power_series_solution_of_differential_equations) to solve a differential equation starts with an assumption that the solution can be represented as a power series with some positive radius of convergence. This is formally substituted into the differential equation and derivatives are taken term by term. The resulting coefficients are equated for like powers giving a system of equations to be solved. An example of the method applied to the ODE $f''(x) - 2xf'(x) + \lambda f(x)=0$ is given in the reference above and follows below. Assume $f(x) = \sum_{n=0}^\infty a_n x^n$. Then @@ -391,9 +405,9 @@ We can see these terms in the `SymPy` solution which uses the power series metho ```{julia} @syms x::real u() -∂ = Differential(x) -eqn = (∂∘∂)(u(x)) - 2*x*∂(u(x)) + λ*u(x) ~ 0 -inits = Dict(u(0) => a_0, ∂(u(x))(0) => a_1) +D = Differential(x) +eqn = (D∘D)(u(x)) - 2*x*D(u(x)) + λ*u(x) ~ 0 +inits = Dict(u(0) => a_0, D(u(x))(0) => a_1) dsolve(eqn, u(x); ics=inits) ``` @@ -406,22 +420,11 @@ dsolve(eqn, u(x); ics=inits) The [Brachistochrone problem](http://www.unige.ch/~gander/Preprints/Ritz.pdf) was posed by Johann Bernoulli in 1696. It asked for the curve between two points for which an object will fall faster along that curve than any other. For an example, a bead sliding on a wire will take a certain amount of time to get from point $A$ to point $B$, the time depending on the shape of the wire. Which shape will take the least amount of time? - -```{julia} -#| hold: true -#| echo: false -imgfile = "figures/bead-game.jpg" -caption = """ +::: {#fig-bead-game} +![](./figures/bead-game.jpg) A child's bead game. What shape wire will produce the shortest time for a bead to slide from a top to the bottom? - -""" -#ImageFile(:ODEs, imgfile, caption) -nothing -``` - -![A child's bead game. What shape wire will produce the shortest time for a bed to slide from a top to the bottom?](./figures/bead-game.jpg) - +::: Restrict our attention to the $x$-$y$ plane, and consider a path, between the point $(0,A)$ and $(B,0)$. Let $y(x)$ be the distance from $A$, so $y(0)=0$ and at the end $y$ will be $A$. @@ -429,32 +432,24 @@ Restrict our attention to the $x$-$y$ plane, and consider a path, between the po [Galileo](http://www-history.mcs.st-and.ac.uk/HistTopics/Brachistochrone.html) knew the straight line was not the curve, but incorrectly thought the answer was a part of a circle. -```{julia} -#| hold: true -#| echo: false -imgfile = "figures/galileo.gif" -caption = """ -As early as 1638, Galileo showed that an object falling along `AC` and then `CB` will fall faster than one traveling along `AB`, where `C` is on the arc of a circle. -From the [History of Math Archive](http://www-history.mcs.st-and.ac.uk/HistTopics/Brachistochrone.html). -""" -#ImageFile(:ODEs, imgfile, caption) -nothing -``` +::: {#fig-galileo-result-history-of-math} +![](./figures/galileo.png) -![As early as 1638, Galileo showed that an object falling along `AC` +As early as 1638, Galileo showed that an object falling along `AC` and then `CB` will fall faster than one traveling along `AB`, where -`C` is on the arc of a circle. From the [History of Math +`C` is on the arc of a circle. Low-res image from the [History of Math Archive](http://www-history.mcs.st-and.ac.uk/HistTopics/Brachistochrone.html). -](./figures/galileo.png) - -This simulation also suggests that a curved path is better than the shorter straight one: +::: +This simulation in @fig-brachistochrone-animation also suggests that a curved path is better than the shorter straight one: +::: {#fig-brachistochrone-animation} ```{julia} #| hold: true #| echo: false ##{{{brach_graph}}} - +let + gr() function brach(f, x0, vx0, y0, vy0, dt, n) m = 1 g = 9.8 @@ -514,21 +509,23 @@ end n = 4 -anim = @animate for i=[1,5,10,15,20,25,30,35,40,45,50,55,60] +anim = @animate for i in 1:2:60 #[1,5,10,15,20,25,30,35,40,45,50,55,60] make_brach_graph(i) end imgfile = tempname() * ".gif" -gif(anim, imgfile, fps = 1) +gif(anim, imgfile, fps = 8) -caption = """ -The race is on. An illustration of beads falling along a path, as can be seen, some paths are faster than others. The fastest path would follow a cycloid. See [Bensky and Moelter](https://pdfs.semanticscholar.org/66c1/4d8da6f2f5f2b93faf4deb77aafc7febb43a.pdf) for details on simulating a bead on a wire. -""" - -ImageFile(imgfile, caption) + caption = "" + plotly() + ImageFile(imgfile, caption) +end ``` +The race is on. An illustration of beads falling along a path, as can be seen, some paths are faster than others. The fastest path would follow a cycloid. See [Bensky and Moelter](https://pdfs.semanticscholar.org/66c1/4d8da6f2f5f2b93faf4deb77aafc7febb43a.pdf) for details on simulating a bead on a wire. +::: + Now, the natural question is which path is best? The solution can be [reduced](http://mathworld.wolfram.com/BrachistochroneProblem.html) to solving this equation for a positive $C$: @@ -576,7 +573,7 @@ $$ y_{n+1} = y_n + h \cdot F(y_{n+1}, x_{n+1}). $$ -Seems innocuous, but the value we are trying to find, $y_{n+1}$, is now on both sides of the equation, so is only *implicitly* defined. In this code, we use the `find_zero` function from the `Roots` package. The caveat is, this function needs a good initial guess, and the one we use below need not be widely applicable. +Seems innocuous, but the value we are trying to find, $y_{n+1}$, is now on both sides of the equation, so is only *implicitly* defined. In this code, we lazily use the `find_zero` function from the `Roots` package. The caveat is, this is non-performant and this function needs a good initial guess; the one we use below need not be widely applicable. ```{julia} @@ -595,56 +592,67 @@ function back_euler(F, x0, xn, y0, n) end ``` -We then have with $C=1$ over the interval $[0,1.2]$ the following: +We then have with $C=1$ over the interval $[0,1.2]$ the graph in @fig-back-euler-solution-to-brachistochrone. +::: {#fig-back-euler-solution-to-brachistochrone} ```{julia} F(y, x; C=1) = sqrt(C/y - 1) x0, xn, y0 = 0, 1.2, 0 -cyc = back_euler(F, x0, xn, y0, 50) -plot(x -> 1 - cyc(x), x0, xn) +cycloid = back_euler(F, x0, xn, y0, 50) +plot(x -> 1 - cycloid(x), x0, xn; legend=false) ``` -Remember, $y$ is the displacement from the top, so it is non-negative. Above we flipped the graph to make it look more like expectation. In general, the trajectory may actually dip below the ending point and come back up. The above won't see this, for as written $dy/dx \geq 0$, which need not be the case, as the defining equation is in terms of $(dy/dx)^2$, so the derivative could have any sign. - +Remember, $y$ is the displacement from the top, so it is non-negative. #fig-back-euler-solution-to-brachistochrone flips the graph to make it look more like expectation. In general, the trajectory may actually dip below the ending point and come back up. The above won't see this, for as written $dy/dx \geq 0$, which need not be the case, as the defining equation is in terms of $(dy/dx)^2$, so the derivative could have any sign. +::: ##### Example: stiff equations -The Euler method is *convergent*, in that as $h$ goes to $0$, the approximate solution will converge to the actual answer. However, this does not say that for a fixed size $h$, the approximate value will be good. For example, consider the differential equation $y'(x) = -5y$. This has solution $y(x)=y_0 e^{-5x}$. However, if we try the Euler method to get an answer over $[0,2]$ with $h=0.5$ we don't see this: - +The Euler method is *convergent*, in that as $h$ goes to $0$, the approximate solution will converge to the actual answer. However, this does not say that for a fixed size $h$, the approximate value will be good. For example, consider the differential equation $y'(x) = -5y$. This has solution $y(x)=y_0 e^{-5x}$. However, if we try the Euler method to get an answer over $[0,2]$ with $h=0.5$ we don't see this, as seen in @fig-too-stiff-this-equation. +::: {#fig-too-stiff-this-equation} ```{julia} F(y,x) = -5y x0, xn, y0 = 0, 2, 1 u = euler(F, x0, xn, y0, 4) # n =4 => h = 2/4 vectorfieldplot((x,y) -> [1, F(y,x)], xlims=(0, 2), ylims=(-5, 5)) -plot!(x -> y0 * exp(-5x), 0, 2, linewidth=5) -plot!(u, 0, 2, linewidth=5) +plot!(x -> y0 * exp(-5x), 0, 2; linewidth=5, label="Exact") +plot!(u, 0, 2; linewidth=5, label="euler, n=4") ``` +For this rapidly decaying vector field, the Euler method overshoots with big step sizes +::: + + What we see is that the value of $h$ is too big to capture the decay scale of the solution. A smaller $h$, can do much better: - +::: {#fig-too-stiff-this-equation-but-n-50} ```{julia} u₁ = euler(F, x0, xn, y0, 50) # n=50 => h = 2/50 -plot(x -> y0 * exp(-5x), 0, 2) -plot!(u₁, 0, 2) +plot(x -> y0 * exp(-5x), 0, 2; label="Exact") +plot!(u₁, 0, 2; label="euler, n=50") ``` +For the same rapidly decaying vector field, the Euler method tracks better with smaller step sizes +::: + This is an example of a [stiff equation](https://en.wikipedia.org/wiki/Stiff_equation). Such equations cause explicit methods like the Euler one problems, as small $h$s are needed to good results. The implicit, backward Euler method does not have this issue, as we can see here: - +::: {#fig-too-stiff-this-equation-but-back-euler-n-4} ```{julia} u₂ = back_euler(F, x0, xn, y0, 4) # n =4 => h = 2/4 vectorfieldplot((x,y) -> [1, F(y,x)], xlims=(0, 2), ylims=(-1, 1)) -plot!(x -> y0 * exp(-5x), 0, 2, linewidth=5) -plot!(u₂, 0, 2, linewidth=5) +plot!(x -> y0 * exp(-5x), 0, 2; linewidth=5, label="Exact") +plot!(u₂, 0, 2; linewidth=5, label="back_euler, n=4") ``` +For the same rapidly decaying vector field, the backwards Euler method tracks better even with larger step sizes +::: + ##### Example: The pendulum @@ -678,11 +686,11 @@ $$ Here we need *two* initial conditions: one for the initial value $u(t_0)$ and the initial value of $u'(t_0)$. We have seen if we start at an angle $a$ and release the bob from rest, so $u'(0)=0$ we get a sinusoidal answer to the linearized model. What happens here? We let $a=1$, $l=5$ and $g=9.8$: -We write a function to solve this starting from $(x_0, y_0)$ and ending at $x_n$: +We write a function to solve this equation for the pendulum starting from $(x_0, y_0)$ and ending at $x_n$: ```{julia} -function euler2(x0, xn, y0, yp0, n; g=9.8, l = 5) +function euler_p(x0, xn, y0, yp0, n; g=9.8, l = 5) xs, us, vs = zeros(n+1), zeros(n+1), zeros(n+1) xs[1], us[1], vs[1] = x0, y0, yp0 h = (xn - x0)/n @@ -697,23 +705,30 @@ end Let's take $a = \pi/4$ as the initial angle, then the approximate solution should be $\pi/4\cos(\sqrt{g/l}x)$ with period $T = 2\pi\sqrt{l/g}$. We try first to plot them over 4 periods: - +::: {#fig-euler-2-but-amiss} ```{julia} l, g = 5, 9.8 T = 2pi * sqrt(l/g) x0, xn, y0, yp0 = 0, 4T, pi/4, 0 -plot(euler2(x0, xn, y0, yp0, 20), 0, 4T) +plot(euler_p(x0, xn, y0, yp0, 20), 0, 4T) ``` -Something looks terribly amiss. The issue is the step size, $h$, is too large to capture the oscillations. There are basically only $5$ steps to capture a full up and down motion. Instead, we try to get $20$ steps per period so $n$ must be not $20$, but $4 \cdot 20 \cdot T \approx 360$. To this graph, we add the approximate one: +Possible solution to pendulum, but doesn't capture behavior at all +::: +Something looks terribly amiss in @fig-euler-2-but-amiss. The issue is the step size, $h$, is too large to capture the oscillations. There are basically only $5$ steps to capture a full up and down motion. Instead, we try to get $20$ steps per period so $n$ must be not $20$, but $4 \cdot 20 \cdot T \approx 360$. To the graph in @fig-euler-2-more-steps, we add the solution to the linearized equations +::: {#fig-euler-2-more-steps} ```{julia} -plot(euler2(x0, xn, y0, yp0, 360), 0, 4T) -plot!(x -> pi/4*cos(sqrt(g/l)*x), 0, 4T) +plot(euler_p(x0, xn, y0, yp0, 360), 0, 4T; label="euler_p, n=360") +plot!(x -> pi/4*cos(sqrt(g/l)*x), 0, 4T; label="Approximate") ``` -Even now, we still see that something seems amiss, though the issue is not as dramatic as before. The oscillatory nature of the pendulum is seen, but in the Euler solution, the amplitude grows, which would necessarily mean energy is being put into the system. A familiar instance of a pendulum would be a child on a swing. Without pumping the legs---putting energy in the system---the height of the swing's arc will not grow. Though we now have oscillatory motion, this growth indicates the solution is still not quite right. The issue is likely due to each step mildly overcorrecting and resulting in an overall growth. One of the questions pursues this a bit further. +Plot of possible solution to the linearized pendulum problem. The approximation is better than @fig-euler-2-but-amiss, as it captures the oscillatory behavior, but the approximation could still be improved.. + +::: + +Even now, @fig-euler-2-more-steps shows something still seems amiss, though the issue is not as dramatic as before. The oscillatory nature of the pendulum is seen, but in the Euler solution, the amplitude grows, which would necessarily mean energy is being put into the system. A familiar instance of a pendulum would be a child on a swing. Without pumping the legs---putting energy in the system---the height of the swing's arc will not grow. Though we now have oscillatory motion, this growth indicates the solution is still not quite right. The issue is likely due to each step mildly overcorrecting and resulting in an overall growth. One of the questions pursues this a bit further. ## Questions @@ -849,10 +864,10 @@ numericq(u(3/2)) ##### Question: The pendulum revisited. -The issue with the pendulum's solution growing in amplitude can be addressed using a modification to the Euler method attributed to [Cromer](http://astro.physics.ncsu.edu/urca/course_files/Lesson14/index.html). The fix is to replace the term `sin(us[i])` in the line `vs[i+1] = vs[i] + h * (-g / l) * sin(us[i])` of the `euler2` function with `sin(us[i+1])`, which uses the updated angular velocity in the $2$nd step in place of the value before the step. +The issue with the pendulum's solution growing in amplitude can be addressed using a modification to the Euler method attributed to [Cromer](http://astro.physics.ncsu.edu/urca/course_files/Lesson14/index.html). The fix is to replace the term `sin(us[i])` in the line `vs[i+1] = vs[i] + h * (-g / l) * sin(us[i])` of the `euler_p` function with `sin(us[i+1])`, which uses the updated angular velocity in the $2$nd step in place of the value before the step. -Modify the `euler2` function to implement the Euler-Cromer method. What do you see? +Modify the `euler_p` function to implement the Euler-Cromer method. What do you see? ```{julia} diff --git a/quarto/ODEs/odes.qmd b/quarto/ODEs/odes.qmd index d9e4652..651b71d 100644 --- a/quarto/ODEs/odes.qmd +++ b/quarto/ODEs/odes.qmd @@ -21,14 +21,14 @@ Some relationships are easiest to describe in terms of rates or derivatives. For * Knowing the speed of a car and how long it has been driving can summarize the car's location. * One of Newton's famous laws, $F=ma$, describes the force on an object of mass $m$ in terms of the acceleration. The acceleration is the derivative of velocity, which in turn is the derivative of position. So if we know the rates of change of $v(t)$ or $x(t)$, we can differentiate to find $F$. - * Newton's law of [cooling](http://tinyurl.com/z4lmetp). This describes the temperature change in an object due to a difference in temperature with the object's surroundings. The formula being, $T'(t) = -r \left(T(t) - T_a \right)$, where $T(t)$ is temperature at time $t$ and $T_a$ the ambient temperature. + * Newton's law of [cooling](http://tinyurl.com/z4lmetp) describes the temperature change in an object due to a difference in temperature with the object's surroundings. The formula being, $T'(t) = -r \left(T(t) - T_a \right)$, where $T(t)$ is temperature at time $t$ and $T_a$ the ambient temperature. * [Hooke's law](http://tinyurl.com/kbz7r8l) relates force on an object to the position on the object, through $F = k x$. This is appropriate for many systems involving springs. Combined with Newton's law $F=ma$, this leads to an equation that $x$ must satisfy: $m x''(t) = k x(t)$. ## Motion with constant acceleration -Let's consider the case of constant acceleration. This describes how nearby objects fall to earth, as the force due to gravity is assumed to be a constant, so the acceleration is the constant force divided by the constant mass. +Let's consider the case of motion under a constant acceleration. This scenario describes how nearby objects fall to earth, as the force due to gravity is assumed to be a constant, so the acceleration is the constant force divided by the constant mass. With constant acceleration, what is the velocity? @@ -189,7 +189,7 @@ let T0, Ta, r = 200, 72, 1/2 f(u, t) = -r*(u - Ta) v(t) = Ta + (T0 - Ta) * exp(-r*t) - p = plot(v, 0, 6, linewidth=4, legend=false) + p = plot(v, 0, 6; ylims=(50, 200), linewidth=4, legend=false) [plot!(p, x -> v(a) + f(v(a), a) * (x-a), 0, 6) for a in 1:2:5] p end @@ -352,8 +352,7 @@ Special cases include: * *separable* if $F(y,x) = G(y)H(x)$. -As seen, separable equations are approached by moving the "$y$" terms to one side, the "$x$" terms to the other and integrating. This also applies to autonomous equations then. There are other families of equation types that have exact solutions, and techniques for solution, summarized at this [Wikipedia page](http://tinyurl.com/zywzz4q). - +As seen, separable equations are approached by moving the "$y$" terms to one side, the "$x$" terms to the other and integrating. This also applies to autonomous equations then. There are other families of equation types that have exact solutions, and techniques for solution, summarized at the wikipedia page on [ordinary differential equations](https://en.wikipedia.org/wiki/Ordinary_differential_equation). Rather than go over these various families, we demonstrate that `SymPy` can solve many of these equations symbolically. @@ -368,7 +367,7 @@ Symbolic functions are defined by the `@syms` macro (also see `?symbols`) using @syms x u() # a symbolic variable and a symbolic function ``` -We will solve the following, known as the *logistic equation*: +We will solve the following differential equation, known as the *logistic equation*: $$ @@ -387,15 +386,12 @@ To specify a derivative of `u` in our equation we can use `diff(u(x),x)` but her ```{julia} D = Differential(x) -eqn = D(u)(x) ~ a * u(x) * (1 - u(x)) # use l \Equal[tab] r, Eq(l,r), or just l - r +eqn = D(u)(x) ~ a * u(x) * (1 - u(x)) ``` In the above, we evaluate the symbolic function at the variable `x` through the use of `u(x)` in the expression. The equation above uses `~` to combine the left- and right-hand sides as an equation in `SymPy`. (A unicode equals is also available for this task). This is a shortcut for `Eq(l,r)`, but even just using `l - r` would suffice, as the default assumption for an equation is that it is set to `0`. -The `Differential` operation is borrowed from the `ModelingToolkit` package, which will be introduced later. - - To finish, we call `dsolve` to find a solution (if possible): @@ -420,16 +416,16 @@ We can confirm that the solution is always increasing, hence trapped within $[0, diff(rhs(out),x) ``` -Suppose that $u(0) = 1/2$. Can we solve for $C_1$ symbolically? We can use `solve`, but first we will need to get the symbol for `C₁`: +Suppose that $u(0) = 1/2$. Can we solve for $C_1$ symbolically? We can use `solve`, but first we will need to get the symbol for `C₁` which is the only unknown one within the set of all free symbols: ```{julia} eq = rhs(out) # just the right hand side -C1 = first(setdiff(free_symbols(eq), (x,a))) # fish out constant, it is not x or a +C1 = only(setdiff(free_symbols(eq), (x,a))) # fish out constant, it is not x or a c1 = solve(eq(x=>0) - 1//2, C1) ``` -And we plug in with: +We can now plug in with: ```{julia} @@ -444,7 +440,7 @@ x0, y0 = 0, Sym(1//2) dsolve(eqn, u(x), ics=Dict(u(x0) => y0)) ``` -(The one subtlety is the need to write the rational value as a symbolic expression, as otherwise it will get converted to a floating point value prior to being passed along.) +(The one subtlety is the need to write the rational value as a symbolic expression, as otherwise it will get converted to a floating point value prior to being passed along. This could also be achieved with `one(x)/2`.) ##### Example: Hooke's law @@ -476,7 +472,7 @@ Suppose the spring were started by pulling it down to a bottom and releasing. Th dsolve(eqnh, u(x), ics = Dict(u(0) => -a, D(u)(0) => 0)) ``` -We get that the motion will follow $u(x) = -a \cos(\sqrt{k/m}x)$. This is simple oscillatory behavior. As the spring stretches, the force gets large enough to pull it back, and as it compresses the force gets large enough to push it back. The amplitude of this oscillation is $a$ and the period $2\pi/\sqrt{k/m}$. Larger $k$ values mean shorter periods; larger $m$ values mean longer periods. +The motion will follow $u(x) = -a \cos(\sqrt{k/m}x)$; simple oscillatory behavior. As the spring stretches, the force gets large enough to pull it back, and as it compresses the force gets large enough to push it back. The amplitude of this oscillation is $a$ and the period $2\pi/\sqrt{k/m}$. Larger $k$ values mean shorter periods; larger $m$ values mean longer periods. ##### Example: the pendulum @@ -503,7 +499,7 @@ Trying to do so, can cause `SymPy` to hang or simply give up and repeat its inpu In general, for the first-order initial value problem characterized by $y'(x) = F(y,x)$, there are conditions ([Peano](http://tinyurl.com/h663wba) and [Picard-Lindelof](http://tinyurl.com/3rbde5e)) that can guarantee the existence (and uniqueness) of equation locally, but there may not be an accompanying method to actually find it. This particular problem has a solution, but it can not be written in terms of elementary functions. -However, as [Huygens](https://en.wikipedia.org/wiki/Christiaan_Huygens) first noted, if the angles involved are small, then we approximate the solution through the linearization $\sin(\theta(t)) \approx \theta(t)$. The resulting equation for an approximate answer is just that of Hooke: +However, as [Huygens](https://en.wikipedia.org/wiki/Christiaan_Huygens) first noted, if the angles involved are small, then we can approximate the solution through the linearization $\sin(\theta(t)) \approx \theta(t)$. The resulting equation for an approximate answer is just that of Hooke: $$ diff --git a/quarto/ODEs/solve.qmd b/quarto/ODEs/solve.qmd index d9fc00d..1cad965 100644 --- a/quarto/ODEs/solve.qmd +++ b/quarto/ODEs/solve.qmd @@ -15,10 +15,12 @@ using MonteCarloMeasurements --- -The [DifferentialEquations.jl](https://github.com/SciML) package is an entry point to a suite of `Julia` packages for numerically solving differential equations in `Julia` and other languages. A common interface is implemented that flexibly adjusts to the many different problems and algorithms covered by this suite of packages. In this section, we review a very informative [post](https://discourse.julialang.org/t/function-depending-on-the-global-variable-inside-module/64322/10) by discourse user `@genkuroki` which very nicely demonstrates the usefulness of the problem-algorithm-solve approach used with `DifferentialEquations.jl`. We slightly modify the presentation below for our needs, but suggest a perusal of the original post. +The [DifferentialEquations.jl](https://github.com/SciML) package is an entry point to a suite of `Julia` packages for numerically solving differential equations in `Julia` and other languages. A common interface is implemented that flexibly adjusts to the many different problems and algorithms covered by this suite of packages. + +In this section, we review a very informative [post](https://discourse.julialang.org/t/function-depending-on-the-global-variable-inside-module/64322/10) by discourse user `@genkuroki` which very nicely demonstrates the usefulness of the problem-algorithm-solve approach used with `DifferentialEquations.jl`. We slightly modify the presentation below for our needs, but suggest a perusal of the original post. -##### Example: FreeFall +##### Example: Free fall The motion of an object under a uniform gravitational field is of interest. @@ -43,8 +45,9 @@ Problem(;g=9.80665, y0=0.0, v0=30.0, tspan=(0.0,8.0)) = Problem(g, y0, v0, tspan The above creates a type, `Problem`, *and* a default constructor with default values. (The original uses a more sophisticated setup that allows the two things above to be combined.) +Types, as used above, serve two purposes: they bundle together the parameters for later reference and they can be used to dispatch varying methods to solve problems. The `solve` generic in the `Julia` ecosystem dispatches on the type of problem it is given. -Just calling `Problem()` will create a problem suitable for the earth, passing different values for `g` would be possible for other planets. +In the above code, just calling `Problem()` will create a problem suitable for the earth, passing different values for `g` would be possible for other planets. To solve differential equations there are many different possible algorithms. Here is the construction of two types to indicate two algorithms: @@ -62,7 +65,7 @@ end ExactFormula(; dt=0.1) = ExactFormula(dt) ``` -The above just specifies a type for dispatch –- the directions indicating what code to use to solve the problem. As seen, each specifies a size for a time step with default of `0.1`. +The above just specifies a type for dispatch-–-the directions indicating what code to use to solve the problem and default constructors. As seen, each constructor specifies a default size for a time step of `0.1`. A type for solutions is useful for different `show` methods or other methods. One can be created through: @@ -86,172 +89,209 @@ solve(prob::Problem) = solve(prob, default_algorithm(prob)) default_algorithm(prob::Problem) = EulerMethod() function solve(prob::Problem, alg::ExactFormula) - g, y0, v0, tspan = prob.g, prob.y0, prob.v0, prob.tspan - dt = alg.dt + + (; g, y0, v0, tspan) = prob # property destructuring + dt = alg.dt # direct property access t0, t1 = tspan - t = range(t0, t1 + dt/2; step = dt) + + ts = range(t0, t1 + dt/2; step = dt) y(t) = y0 + v0*(t - t0) - g*(t - t0)^2/2 v(t) = v0 - g*(t - t0) - Solution(y.(t), v.(t), t, prob, alg) -end - -function solve(prob::Problem, alg::EulerMethod) - g, y0, v0, tspan = prob.g, prob.y0, prob.v0, prob.tspan - dt = alg.dt - t0, t1 = tspan - t = range(t0, t1 + dt/2; step = dt) - - n = length(t) - y = Vector{typeof(y0)}(undef, n) - v = Vector{typeof(v0)}(undef, n) - y[1] = y0 - v[1] = v0 - - for i in 1:n-1 - v[i+1] = v[i] - g*dt # F*h step of Euler - y[i+1] = y[i] + v[i]*dt # F*h step of Euler - end - - Solution(y, v, t, prob, alg) + Solution(y.(ts), v.(ts), ts, prob, alg) end ``` -The post has a more elegant means to unpack the parameters from the structures, but for each of the above, the parameters are unpacked using the dot notation for `getproperty`, and then the corresponding algorithm employed. As of version `v1.7` of `Julia`, the syntax `(;g,y0,v0,tspan) = prob` could also have been employed. +The exact formulas: +$$ +\begin{align*} +y(t) &= y_0 + v_0\cdot(t - t_0) - g\cdot(t - t_0)^2/2\\ +v(t) &= v_0 - g\cdot(t - t_0), +\end{align*} +$$ -The exact answers, `y(t) = y0 + v0*(t - t0) - g*(t - t0)^2/2` and `v(t) = v0 - g*(t - t0)`, follow from well-known physics formulas for constant-acceleration motion. Each answer is wrapped in a `Solution` type so that the answers found can be easily extracted in a uniform manner. +are well-known physics formulas, discussed previously, for motion under a constant acceleration. The `ExactFormula` code broadcasts these functions over a range of values in `ts` and then wraps the output up in a `Solution` object so that the answers found can be easily extracted in a uniform manner. +For the Euler method, a `for` loop is utilized to step through the algorithm, in preparation, the new command `fill(y0, n)` is technical. It sets up a storage vector of length `n` which is initially filled with `y0` but for which the second through last are overwritten. There are many other means to do a similar task, including creating an uninitialized vector with `Vector{typeof(y0)}(undef, n)` for which all entries would be subsequently filled in. -For example, plots of each can be obtained through: +```{julia} +function solve(prob::Problem, alg::EulerMethod) + (; g, y0, v0, tspan) = prob + dt = alg.dt + t0, t1 = tspan + ts = range(t0, t1 + dt/2; step = dt) + n = length(ts) + + ys = fill(y0, n) + vs = fill(v0, n) + + for i in 1:n-1 + vs[i+1] = vs[i] - g*dt # F*h step of Euler + ys[i+1] = ys[i] + vs[i]*dt # F*h step of Euler + end + + Solution(ys, vs, ts, prob, alg) +end +``` + +Plots of solutions generated by the default values for each method are produced in @fig-projectile-motion-on-the-earth-dt-default. + +::: {#fig-projectile-motion-on-the-earth-dt-default} ```{julia} earth = Problem() sol_euler = solve(earth) sol_exact = solve(earth, ExactFormula()) plot(sol_euler.t, sol_euler.y; - label="Euler's method (dt = $(sol_euler.alg.dt))", ls=:auto) -plot!(sol_exact.t, sol_exact.y; label="exact solution", ls=:auto) + label="Euler's method (dt = $(sol_euler.alg.dt))", linestyle=:auto) +plot!(sol_exact.t, sol_exact.y; + label="exact solution", linestyle=:auto) + title!("On the Earth"; xlabel="t", legend=:bottomleft) ``` -Following the post, since the time step `dt = 0.1` is not small enough, the error of the Euler method is readily identified. +Plot of exact and approximate solutions, the latter using the default time step size +::: -Next we change the algorithm parameter, `dt`, to be smaller: +Following the post, since the time step `dt = 0.1` is not small enough, the error of the Euler method is readily identified in @fig-projectile-motion-on-the-earth-dt-default. +Next we change the algorithm's default parameter for `dt` to be smaller. @fig-projectile-motion-on-the-earth-dt-modified shows a much improved agreement between the exact answer and the approximate one found with `EulerMethod`. +::: {#fig-projectile-motion-on-the-earth-dt-modified} ```{julia} earth₂ = Problem() sol_euler₂ = solve(earth₂, EulerMethod(dt = 0.01)) sol_exact₂ = solve(earth₂, ExactFormula()) plot(sol_euler₂.t, sol_euler₂.y; - label="Euler's method (dt = $(sol_euler₂.alg.dt))", ls=:auto) -plot!(sol_exact₂.t, sol_exact₂.y; label="exact solution", ls=:auto) + label="Euler's method (dt = $(sol_euler₂.alg.dt))", linestyle=:auto) +plot!(sol_exact₂.t, sol_exact₂.y; + label="exact solution", linestyle=:auto) + title!("On the Earth"; xlabel="t", legend=:bottomleft) ``` -It is worth noting that only the first line is modified, and only the method requires modification. +Model of projectile motion on the earth along with exact solution. This approximation used a modification from the default for the step size. +::: + +The code is mostly a template. It is worth noting that only one line of code was modified, and in that line only the method required a modification. Were the moon to be considered, the gravitational constant would need adjustment. This parameter is a property of the problem, not the solution algorithm, as `dt` is. -Such adjustments are made by passing different values to the `Problem` constructor: - +Such adjustments are made by passing different values to the `Problem` constructor. Again, just the one line needs modification. +::: {#fig-projectile-motion-on-the-moon} ```{julia} moon = Problem(g = 1.62, tspan = (0.0, 40.0)) + sol_eulerₘ = solve(moon) sol_exactₘ = solve(moon, ExactFormula(dt = sol_euler.alg.dt)) plot(sol_eulerₘ.t, sol_eulerₘ.y; - label="Euler's method (dt = $(sol_eulerₘ.alg.dt))", ls=:auto) -plot!(sol_exactₘ.t, sol_exactₘ.y; label="exact solution", ls=:auto) + label="Euler's method (dt = $(sol_eulerₘ.alg.dt))", linestyle=:auto) +plot!(sol_exactₘ.t, sol_exactₘ.y; + label="exact solution", linestyle=:auto) + title!("On the Moon"; xlabel="t", legend=:bottomleft) ``` -The code above also adjusts the time span in addition to the graviational constant. The algorithm for exact formula is set to use the `dt` value used in the `euler` formula, for easier comparison. Otherwise, outside of the labels, the patterns are the same. Only those things that need changing are changed, the rest comes from defaults. +Model for projectile motion on the moon. Only modest changes needed to be introduced from the model for the motion on oearth illustrated in @fig-projectile-motion-on-the-earth-dt-default. +::: + +The code above also adjusts the time span in addition to the graviational constant. The algorithm for the exact formula is set to use the `dt` value used in the `euler` formula, for easier comparison. Otherwise, outside of the labels, the patterns are the same. Only those things that need changing are changed, the rest comes from defaults. -The above shows the benefits of using a common interface. +The above shows the benefits of using a common interface---new problems can be approached through only minor adjustments to the parameters, yet the calling pattern remains the same. -Next, the post illustrates how *other* authors could extend this code, simply by adding a *new* `solve` method. For example, a sympletic method conserves a quantity, so can track long-term evolution without drift. +Next, the post illustrates how *other* authors *could* extend this code. The `solve` method dispatches on the problem type and the method type. Adding a new method to `solve` requires defining new method type and the algorithm for that type in the extension of `solve`. + +For example, the following adds a sympletic method which conserves a quantity, allowing the approximate solutions to track long-term evolution without drift. ```{julia} struct Symplectic2ndOrder{T} dt::T end -Symplectic2ndOrder(;dt=0.1) = Symplectic2ndOrder(dt) +Symplectic2ndOrder(; dt=0.1) = Symplectic2ndOrder(dt) function solve(prob::Problem, alg::Symplectic2ndOrder) + g, y0, v0, tspan = prob.g, prob.y0, prob.v0, prob.tspan dt = alg.dt t0, t1 = tspan - t = range(t0, t1 + dt/2; step = dt) - n = length(t) - y = Vector{typeof(y0)}(undef, n) - v = Vector{typeof(v0)}(undef, n) - y[1] = y0 - v[1] = v0 + ts = range(t0, t1 + dt/2; step = dt) + n = length(ts) + + ys = fill(y0, n) + vs = fill(v0, n) for i in 1:n-1 - ytmp = y[i] + v[i]*dt/2 - v[i+1] = v[i] - g*dt - y[i+1] = ytmp + v[i+1]*dt/2 + vs[i+1] = vs[i] - g*dt + ys[i+1] = ys[i] + (vs[i] + vs[i+1])/2 * dt end - Solution(y, v, t, prob, alg) + Solution(ys, vs, ts, prob, alg) end ``` -Had the two prior methods been in a package, the other user could still extend the interface, as above, with just a slight standard modification. - - -The same approach works for this new type: +Had the two prior methods been in a package, the other user could still extend the interface, as above, with just a slight standard modification. +The exact same approach to solving a problem works for this new type: +::: {#fig-projectile-motion-symplectic-2-order} ```{julia} - earth₃ = Problem() sol_sympl₃ = solve(earth₃, Symplectic2ndOrder(dt = 2.0)) sol_exact₃ = solve(earth₃, ExactFormula()) -plot(sol_sympl₃.t, sol_sympl₃.y; label="2nd order symplectic (dt = $(sol_sympl₃.alg.dt))", ls=:auto) -plot!(sol_exact₃.t, sol_exact₃.y; label="exact solution", ls=:auto) +plot(sol_sympl₃.t, sol_sympl₃.y; + label="2nd order symplectic (dt = $(sol_sympl₃.alg.dt))", linestyle=:auto) +plot!(sol_exact₃.t, sol_exact₃.y; + label="exact solution", linestyle=:auto) + title!("On the Earth"; xlabel="t", legend=:bottomleft) ``` -Finally, the author of the post shows how the interface can compose with other packages in the `Julia` package ecosystem. This example uses the external package `MonteCarloMeasurements` which plots the behavior of the system for perturbations of the initial value: +Plot of exact solution and approximate solution using a $2$nd-order symplectic method +::: +Finally, the author of the post shows how the interface can compose with other packages in the `Julia` package ecosystem. This example uses the external package `MonteCarloMeasurements` which plots the behavior of the system for perturbations of the initial value, as seen in @fig-illustration-with-monte-carlo-measurements. +::: {#fig-illustration-with-monte-carlo-measurements} ```{julia} +using MonteCarloMeasurements # introduces ± operation + earth₄ = Problem(y0 = 0.0 ± 0.0, v0 = 30.0 ± 1.0) sol_euler₄ = solve(earth₄) sol_sympl₄ = solve(earth₄, Symplectic2ndOrder(dt = 2.0)) sol_exact₄ = solve(earth₄, ExactFormula()) -ylim = (-100, 60) + P = plot(sol_euler₄.t, sol_euler₄.y; - label="Euler's method (dt = $(sol_euler₄.alg.dt))", ls=:auto) -title!("On the Earth"; xlabel="t", legend=:bottomleft, ylim) + label="Euler's method (dt = $(sol_euler₄.alg.dt))", linestyle=:auto) Q = plot(sol_sympl₄.t, sol_sympl₄.y; - label="2nd order symplectic (dt = $(sol_sympl₄.alg.dt))", ls=:auto) -title!("On the Earth"; xlabel="t", legend=:bottomleft, ylim) + label="2nd order symplectic (dt = $(sol_sympl₄.alg.dt))", linestyle=:auto) -R = plot(sol_exact₄.t, sol_exact₄.y; label="exact solution", ls=:auto) -title!("On the Earth"; xlabel="t", legend=:bottomleft, ylim) +R = plot(sol_exact₄.t, sol_exact₄.y; + label="exact solution", linestyle=:auto) -plot(P, Q, R; size=(720, 600)) +title!.((P,Q,R), "On the Earth"; xlabel="t", legend=:bottomleft, ylims=(-100, 60)) + +plot(P, Q, R) ``` +Figures showing solutions to a differential equation where the initial values have some specified uncertainty +::: + The only change was in the problem, `Problem(y0 = 0.0 ± 0.0, v0 = 30.0 ± 1.0)`, where a different number type is used which accounts for uncertainty. The rest follows the same pattern. -This example, shows the flexibility of the problem-algorithm-solver pattern while maintaining a consistent pattern for execution. +This example, shows the flexibility of the problem-algorithm-solve pattern while maintaining a consistent pattern for execution. diff --git a/quarto/Project.toml b/quarto/Project.toml index 7525162..cd66591 100644 --- a/quarto/Project.toml +++ b/quarto/Project.toml @@ -1,4 +1,5 @@ [deps] +AbbreviatedStackTraces = "ac637c84-cc71-43bf-9c33-c1b4316be3d4" BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf" CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b" CalculusWithJulia = "a2e0e22d-7d4c-5312-9169-8b992201a882" diff --git a/quarto/_common_code.qmd b/quarto/_common_code.qmd index db01bfd..e5cc0f3 100644 --- a/quarto/_common_code.qmd +++ b/quarto/_common_code.qmd @@ -1,3 +1,10 @@ +```{julia} +#| echo: false +# comment this out for plotly graphics!!! +using Plots +plotly(); # gr() +``` + ```{julia} #| output: false #| echo: false @@ -32,7 +39,7 @@ nothing ```{julia} #| output: false #| echo: false -fig_size=(800, 600) +fig_size= (800, 600) nothing ``` @@ -46,6 +53,18 @@ Logging.disable_logging(Logging.Warn) nothing ``` + +```{julia} +#| echo: false +#| eval: false +#| message: false +# This is too noisy to keep +using AbbreviatedStackTraces +ENV["JULIA_STACKTRACE_ABBREVIATED"] = true +ENV["JULIA_STACKTRACE_MINIMAL"] = true +nothing +``` + ```{julia} #| eval: false #| echo: false diff --git a/quarto/_extensions/coatless-quarto/custom-callout/_extension.yml b/quarto/_extensions/coatless-quarto/custom-callout/_extension.yml new file mode 100644 index 0000000..1d2c191 --- /dev/null +++ b/quarto/_extensions/coatless-quarto/custom-callout/_extension.yml @@ -0,0 +1,9 @@ +title: custom-callout +author: James Joseph Balamuta +version: 0.0.1-dev.3 +quarto-required: ">=1.5.0" +contributes: + filters: + - customcallout.lua + - fa.lua + diff --git a/quarto/_extensions/coatless-quarto/custom-callout/assets/css/fontawesome.css b/quarto/_extensions/coatless-quarto/custom-callout/assets/css/fontawesome.css new file mode 100644 index 0000000..6578990 --- /dev/null +++ b/quarto/_extensions/coatless-quarto/custom-callout/assets/css/fontawesome.css @@ -0,0 +1,7 @@ +@font-face { + font-family: 'Font Awesome 7 Free'; + font-style: normal; + font-weight: 900; + font-display: block; + src: url('fa-solid-900.woff2') format('woff2'); +} diff --git a/quarto/_extensions/coatless-quarto/custom-callout/assets/webfonts/fa-solid-900.woff2 b/quarto/_extensions/coatless-quarto/custom-callout/assets/webfonts/fa-solid-900.woff2 new file mode 100644 index 0000000000000000000000000000000000000000..ea09c935a97afecc71d07f2128116878233bf9f5 GIT binary patch literal 114740 zcmV)2K+L~)Pew9NR8&s@0l+i>2><{91lvde0l(`5f&l;k00000000000000000000 z00001HUcCB1_odQtULvX0sw)PHU}ULk?M4%qivE$&tHt*1Z^K>73Xvt0#t3P){&HF z#sRiXgNAn7FzdJhikJO0%59g#?cmUl*NrCs|NsC0{~MDp(!0H+cS)19z3%SzV;c?- zrih@T5`jP=$fj_FMyXI#2pXNTcQkBT=EUa08x|jE#a0q`{C@ZYdlcNI%J8Su^Q_xT zFnK%^f6whpz9)O?{3+d1zreVQhSCYy$&sqCLWRogm!Ot>MdKdOsH2>t?HrH9$kSJ< zM3rhp(6gBtlLwJ!R)iH{MM&?l`U`PbHA|V{D^rng`tAEA zbXGWn$UJSSr3Wvme+xa5n)|v=)JFH*0vmebdwL&7<6ov_h>sENr=`T6ypQxBKK~0< zoN=X^MfBQMC+>c8N3R+%#d2V@vy#@1q>ws6;l$w9jM&EAoj4+%j7&r3d1lXyq8p>o zjzTt#ZPb!p0&JHJ1ckIrVHz`tMf4?4i}>rU8uKA0*+4unB3>wOF6Y47{o)_{xy|!1 zeG}J)Y5F3bDTT_oPUx!@iqhIpC6XTw>`TbJI{oJu{R0)!vN zBHCTy`R9J#_qB=|uot{Ds`8zo21q2P+ZyBsXYK%9^8&Q2F#sRP1@Qy1NGu8dziiq^ zr|JgD3$o}#kn|$iULsjMVj77}f(4*n3fbMa5mJ=Y{{Y|wLjZ(+`|aEw$WP%6Hgh%f zU?~Rp!kq3%1xx``02r{Ead+|;PVS;qt(U6yg`fYsmNUIha;DcovMpPXEL&8D5=D_B z31%=Mq=^6;m<(h_FoQu9NhK&#(U9dJXRf`r<*+uq>-B1VjjF~)2xPb+$>ohil&-;v z8EHTq9Gr0S3XmdFybeafiKYGQB372!OlR%Ai zx}_}w{C{#<|3BJm?ZL#8(Hj$eAnuD*)mW4`F_0|D_OZ|cv}AM030GCRsObU1_zgeR z2|9>6$f_pxk(!K-AU2KR5%khSG$#ADuz}YE)w}3GFbIOdU`UqM4E&If@R*PKtmjFH zd&1Zurk+wLfe-#xkmQEF8q6$8*y$xCd<*k=YPwEyIC3_dim#;RgQvI#4_;Z5zv49k zFRm>3YaHu94kV=22syLJ zAY(@ok^o5q#S z-+>5t#~WgWdBuU)jd@{B;}!9>-&n`5_xG;o(iBkIqE$LXD|3r(M3sYxf5iMTEPi(8 z=Kt?Ywc6efP(V;w?F`bi4EGp4Glz4Ed3vz{gOpB|hdNH#<*sSfv(2Y=~;mgI#H znGX%0 zhEdA#!vI`du6@KJo^ZYd)$UTTJNX$Irq2I#<8Q?O)mBdSd0``(D~l^}R;9(PTwBpu zb=S9j>|4{?&zkM@ui4GaSTA)Z_IB&lqwjX)9=9o%>%jHe*tfLe@eUW&J{Q{WIdlDh zRqb!ZA&PpzM{@N!c#c;>`rTWIeoCb(S_Niy8Xi%9ZPC{*VFB5)uzcBg#B!yEurGD= z6CmAqoV^JsC9jmHT3O(`q?EQM8t*LnhNQjBCaJ^$DDF|;XFo_5gSoQ?wwU!8r=9d> zwKYL+YmL9+Jz<0HXo)fTWMXe)wl3_eHGwrp>%xh0DOaw8x*N_2@{OOB1%AUGf;oXa z+5zwyHsKBc+CyA2dh(RNL-Xu0wdB@6p?-VLYe5qti6ml*=b#g5eQi8@+G@tAMoR+i*-6K8+@XYEUd5rJ_4H;BLzbGCVB!K_MJ2AbztC#L1+_<2ivTkFe@K=y+9 zu_hjq2ZrgxY18gzcD^ILo8O@wSy!^_U2E^I{K1))$EXpkNG8af8sWXdnfVsDBRDD# zFw{Us(V&`*(8=h9qMTrw_nu%{y0Mi2_xv&tzmbVQ7b`Vcd`S-aU?Hl}aoA zFM~Tzz`g+baZKgy+|EB34|q!heau2%@a~>^wwYxd`orS;t$!!DM}U|NsoJMH`z$YH zV*xWDoernfEDeJpV)?!9j{tcSA)Ey?Vc}za>FU`QcnGWYG2O$OXlPkOx}&Cr^}AipYNi`CLQsTGno z>i*__bxgqVx8Ql%Pi@`@l&dr7 zlOt9q|AxG-hIjToV~_wF36n(!h~yOdW#fgiU11WSw4Gw%FH56WMSZ7^8lz^<#{}4qTAW)gFdzUfC6!A_~vQho|O=UM_eep<`0_N1KjC$@uVrz z=MxHpBao=={xlYcClE;u(NuR2{aPP>P6`)~o5(1!v1DpjUj zr3Nibow-~4S2f*;#!uxhJ%lb>7mKK|&TI4dA~9`+>>Zq3IheVWxy!GmlT*&_P%cH) zbVDrLaXnt2AIjzN0fA7iusAupdFbj<^rV?_dODn8XknDhx0}d$D{( zjg_1Wq0Esbk()Vo?xqKo?P}F&`j`fXJ9nx20_m6@FN$V(K?F&jmLj5JAT4bjT~jl2 z3tJz*RLCsns`gFWZT5%9*Z*mk*#f1C*LtJ3dZ(|<=y-m6|NJ2+8YEda&~d$}?4~^> z>V^~Khxfp=Cw>sd`R?NK`tx6CO{TLsQ{h%<>k-wfxnEYpsy(ry47&Z{bbia&Fdf%V zvgvYr_u|$4=da(qegF60>JJvu@grll=3uT1PcO96Nf(lKdeWPI9*ELsPq)j}yDz9n zRcce0mSmHph)#5&+pUz#H=W5+sB{Li6%2vG5y><*Ae1Qd78^%bcP~0&@`R>l&Ro0q z)|U!OTKeWJf^t!g@`EHTt0nSpIr)5ZdGqqkH)>jX#%2~Q9C8F=muzhB-~UeLE45Z{ zvOZkJQ`u6b(H=||+x_w8?(yaQ=WsARd-Uv!pMU%P!pipU{^9XyR#e?lvpsIlv>C`e zhOn}tyM8<_F-p)(FRZ(1y*~Scu@_|Je0{RJyMOckuUKn!dNUO!M1dFO^So~RaS{wC zdGU1SWi3ei63xmC4X<1o40Yct8RSFYE%6KN{7IvQH?4B_{@Z6I2cV?+G-sg+!6b3{V0~$6 zZyr&pt(b;uAg>3m)&wf2Fv=NPG&AA&d9|%w|TOLF0n&&RP@eO`u*`3gN!pbc^F9g)W z7D=!c{MNi_?TE~t7CvHGWx`&6%QO89BX$Dg%e={ZG!3J$Kw93O2$7)%`l)s&ExPP+ zuY1?~zU$Am+0MlLUS`(Ldb2yS?4bdqWNUeNme0CATHXZTS^G~|hWM~f+uoQ^3QkHC zhaQS_2_D}vvT)&ng7%0BilR%7IOCRAcxNn7rI9cOg~1aF12jR~M&o%sSKrF}zvVc~ z+}Nq)yHLdUS8^*rKqUufE)|bql7v^4yh6 z7n%;$?YM8K;oi@dwjsgTeYdTaY_`WEB=ORaTY}u5>{gy1w<4l=^GwwHHc%D_e(4xs2 zSUua-s8XRsaVQ1y@6+Uf$l~++#wjLmwljF#Mk|W%a@Z_&WhmbY`9t4%YzFs(I}Dl_ z0cXLS;hyJA&+te7Ti5wI7gsG*)|Jmde%4$Ha$JBpR$a_uM5<7<|9{6%3<`f&7V`Hq3wXteqY-FIXr>mo_ zsiC5zsNmnPZ?D}ub8IV@3OH;gG0@X?cC@!On~Zw3LN1kv0X~n*W-%~mILM%oibVo0 zgHEGT$V40ljqvTocV?0O>#fHgx<0z-xT6kQZMntfnry6*2J5e_)|zXoy2>glucW+k z%P6hXQc5bZn4*d(ys$zGDzJe3^Uf)|Y_iHSi_9`e?>Oh2G}A~u)l^bQB9YiCD8HPt zN-Z%<(|!Nn^*y|sH}bk(!>d1D_KW?R@pWeB`W}rR!7W#8aL-LC+F>1$OI>7>C%pI*CK1@+5k-Ih59vQWzb6ib-;-Z==*1=^D4K%RUPNRKy zm+(UUsm1G~gSH)^VZGL4P1ktq)^cySXNW#(SzwN0i1yBNRr*Gp-$D3u)kJ~&h1*dz z^L$qAw_`dYZdi~~ZQG@eX*LRb=@u0Vj7*oX5sMjt<|X3Xg}9B+&;&&nnkZ(hoz%&k zm6(IMVpKn-O{Y9w2a(KpHdi(PBVy-2a8DBgnu@==enul6Y%#A-$ zVnfV;c(rq%{}91pS8<>#-*u-(B+Pl(-Riu0oNhp5`$do#@zwU%e#VOO$=~msNMpgL z>Qcqq43hp;&est!CB7*vEWP)uJ{dn<+&H(6TLIG-$*VaZ7Jqc5CMryq0;CKCW{R@TpnK_6p4-gAWa`WdK`%-(wS_j_E&E-mmp9o z$Mu6S(dy0mGa|dPH~Vlfqd8SGY(+^iE!z*Wyr`SDo91^(w(D+t(^nO0H;MDCer=sh;f|(!2$* zb$A zkt9i+q)6e}Heore_E0C*Kv)$IX6PO8kVMe2p~hA~gJxYq0VOtF*bSS0bPfXs7iARr zjh!}{VA`}Zr%oN8GavR{x^&{omA&2=oVj(&z@0l(eKtDe#fu|v-Zb`8xDCI~pcN-K z%!AVuI>4C@JL1fYHk~)un2!3pZ}?XT@Szn12yBHiMiFSxh>IaO15gMjj0m6*aWEaQ zC`2{}5egKxf!;WahFK(Rx1?>zUpk0UcIs>l0?4ZaEphGdZ==QOmiX5V6!y8YY$o5s_ z0{w^};|0PTcp-l==m#_yFft%O04k!$3*Zp8sUwBZXo@kJdek;L5r@O7Yv>@}!{hTE zI)O_gqZi3!UN#pnfUOtOhR@djIRjU&j)b)0v&|1`I15*7X^GC-rCUL}z_tsx0JeQl z4s3@|SD;80G8u~0FER<{SyT+=jT1tIiVu+sN|fwMCD;*AO#Vp~`#H>l9h3$Iin1_a zDuyUAuOj7Df&5d?nBx z@O=BAwtL??2y_Iz_%Y}lfBew}Cmvl15YUq%MZFj?(whYfeb}(kmnTpC1PRt(tXKo3 z%P>f(QbW|LHB^TV!weWO+=vk)%$qmTqD7-DTQ<6v4npq)#sXeE2F3widVHmFf+n|k$jXf$D`DO)bsapa~G=N`Cp>B+frm0s8T4b*x%Z{RHIL2}S8 z)=ijxl0!&=I!>DG2>Q#38W*9ogo|Y(PC6%l@4#Lt9`GLSO#(zn65+y?j1aLDWXPlwCBHJB4q!VVl@!V_ z28gMcGX$5&rj3dk&PI%w4B`v4X@@*5C$wq1rbEXKUApe+Gxbn22ivFSg5CiR zizm!>Sl+gs5BAGvGi3b?V-P3=rcj~&2@|GHxNx{9!qpm)cjSO4dc|zU2Nvg>1WAY_ zOUBcbVQ8FGsYqq6?9F6y<)M@>pSA+ksMV_TLysxH&6p)@$rjPGQ)gJHCh zLxSJY(DYm(YdF;PqAk&F8X>q z2dGx8#e9_51u?tH1PItE|DT}oG4ciEgGkoM+~qyT#a|Io=x%s1(1~fhNqc^ zZpo63SFS?*N|X}Opoy4f%_OvHC3)Ihv3BjGPKS$5oup5fOM47e(!2V$$mrKk*03oc zGv=vUupr*L4Vn%e(st~ajx%R;UAm;_)_?jQJTmm+g>gUqAtr!i$Qu?@2#{I9gvkyO zBKAm-_=^r5hyFA)3H##M;s^x9nK~^lbm;J)PoF14M!f28Sd9Py)P=0jEk3}&{F*DM z3XoiMB+uPtbsy1QAwhuT!H{4;^5~KL-}Shst5CCMMcA|{(vBTb4jqbi=~9eWZ({oq zLhAv^Um|f(PH!gOEe%^ic#6Jjh{SMeX3NMP_N#k7Hy_*5<8L1P=aG959aD0aBSnK&i}p zA4LR^`a=Yk`jckvKOIzT*!W{70Fxj=N=OW%#v1AaERKbQm9^DMj(jc(mGDsRA0PD^ zglN(tOq))Tx($joY)rCAQ_{_tlWEzq9BbC(+OQ$dmTmcV?J9KOK#^m|ik&-G;>wj$ z?><)9|1=0MS^-Fdep3k@x+++(R3kv31}Rdt$dUVp3bi`)7&Kt7hVDFC{Aas8cschpr)p42>{mYLo>l6KvR+X2;G9Cr)O$a52Y? zyLq0xEb`@RnZF<_LWSFqEZMeH8Fu8#bD%_-BUP%LYSQdXyLK14bh*@{$CZ8qu8kOR zYudE`X3e@YZ{EE{iykan_Gs0rC!4lB+q3W0kt6R;ocM6z(x;pCe;c5$#z67z9rR#d zO9z6Hj7-RtHBw)oQ3ln$%RSw2fx! zRBLt9Guo(6?iyfF9W~0>p_3+<8r?KA`e;u5wanUq&;dtdfG)-ox|X;8b2paK!&pvF zT2ZgOQTbve2l=j6g8@e_qIgemNU;xr5BayKC}x(RE%ZGrm@sYNz_o=BpN|-^04)X~ zR;*Hd_++F=$;psWkS9-2qNJotSH*}C$%YNZojc8wx0=5IjUb^qapD}LNO9Do*U5wl z?=4&AtXlO&O&F>ZBG|2r7x!n34F_CX9V@={t znloQ!%qTtcF6uuEz6RPZnR{tx)s;HWx+{6kj!WxiFMIW{!yMMS=fst^I?s7SdG1|W z-}B^2BXke_lv|*C_@`XbJ;GBSfbLPB2B3SKr`!YG=a}@TQY09`G6Y$&{s950qY#D4 z7nP_~nHk5pRJkVR%qdTvUf0hA&4!!k-xw4A5|dcNCoO5Ea_OIR8J`T9pUjiz;@Y;r z#J-Z0l(tTpOX}Q86BBAvTVty(b%m%;eVJUlb~ZY6_^C@5Zr$mweCYY~=uK}KMV~&X z1`H@OXi)eLc^S`m#bGB*2j8SgzfGCKb*5d~-m~C|#(?gZer`OT4VUSY-83D0TiGg; z*v@v1l^r{V?Aj%1&mJmAIjU5+apT0TTVWpaSZeZ=r!wYw^K9Hi(gX0k3`-9XUw8u@ z=%Nok@Kp&i03is~V~7w9LJCp^=ODY_G$>G%Ertm!m@xG$mVpd*u*=$T;5dgH+|oEa zc>XNDsk%_32B1ZY6xU2vXqt2PPyNw=ikFWm#JYJOowutyyG=% zHgsD2K85c?tMPBn(>{FhqU|;nie9}?jT=|CQ{M3wvsh8Hl%2C*6e1qxm0K`*;3 zp6S>wzDX1I1O#@3ghoU}Cd9E#E3M zh!GQCN^?ek=}j0ABuJQ0glael6UIp-A{A{QAjzv_LSt1-{GdsbA=kK8E`b9smdQv) z^-FPzYhX2`p_-#vvs9CrEVt~AiM-wQZVN|_Y&*?qIoO#qLwjMW`n@z~a`xIpgqz${ znfAX4cz5n-dC5!V*<0RfPy~O^d$0fb^b}%xb^oF~JmOJnZk&lL8OdmbPw&?6P=7+(Bz66UAnF3zF1%HSanC(*GTt&PU11TP( zgv2X}k;;@>Rys$SNM-AlvtD_ge?x_R73WmSS4CS@Ce;dGbqx8g23Z?wrmdD`GiqzD zj%tVM_N<;`bLubq?yv8|{ov`xNIyC3XS9O-BJ5Y&{r0<-{f-`Z4Qcwn#X;h|;0TFKOa=*mlCv#? z=)`hEE2^!+vf9#`N$dEn547RgrVm@RY`w7^Waoli-gYUE{-)i?&t*ePC_>6RH@T?J0n-Cvobr6)wv5${=jfC(x18h`n$hbdiGC&e?9!C z;D5ATBJ9iMb609z^>Iye*9E(Dv)Ju9cS_uia?ktjdtmC}l1KU;yX=Wn2cAjnIYRw= zq1&35@m@9VwQ%oyOVP7;w%*tJFzFMK&xgJ$&EflB%@@RflfEq5_y1GOz6dAnLk5{l z30b_df6Bp@EAsNl((_;WP{4fYEvo_J~p-;jv zi-`cUJ(jM0$E@Cu@cp#u)~_SKgZ;_J7Ceq$6^&D;K3p!iWyXWtC*ERwr1%jC*bxK~ z8YBFN2Z@@$@${3Z0tJ83@9#qd>`A8$4D$1QUK zVvPt?Ul3343)ZS{Awq=B}XV?_+7+_>V}l`yrfG+<@Qtti)1F{*NMRk#<|^s14YT8(c5t7Wfp z|43U{om(FoCf6uY+L}1)(d@hy?54H4uMPc-+h4nQi#iO}30Z)yN!<&2*7WY_J6*qV zTN_Ys&S0z|xD8{SVQ)qjjrJSs+&D_$O=vd1Nyk1m#ag-PKr_voRqT9omW`SZ+k)d& zceX|9GpKD#^k;BseOeCO3SRbBO}8e~!PZTBwGqEfhOTV2Z=1gsJ4|-V?Ty)QbP%ya z{Z4me-|3Ex+S-ZwCAYX!>{G0IXAq~-rSp(oSe3a;$(~%rxt4cpxc@QTxeM7nZvlHC z?qg5r`Tah>xqlo#jic9jZ%*%B%!T*ib z(PLGC;S^);n8Z85oQ%a2tJA&{wYZ;9XV#QovHLAqkv~CWL#zh72M(u?^9)zy1Wq4o@3X@pQPSi&Hy2gXzOnF$1z(8FDfDHDge@ zOps=ZS0gj4nZq^6!fKXemCp*dRo1<12H4uo4kpLh)Ai3^vpK+?QtfjzYi&;SmCqTe zYAy*}-MIO3SNl9pcuMjr<$c0ukDvVW|Ecf==rk&@VL_hV2##HdVRH)=s&-){xe6Z= z0bXQK8;gQ2PBfmFz=3g6T^waa;{BH()xN}#CDBzOxmHT-QfZ4`8tM7BuymSpZ*mzp z4aqE97C}|A1()-?+T}7;vOJ2`m#@{0g1i+Xx2x#yit)~;qm?L@zf!ZNRz_R0@{v_& zwzx{E3Ri`ECe5!}t^aDG)&B60ZguPptAErGv_``|G^MW@wPh_9$4Gt?kn7l6PazZu<^R zIZ)prZAFgwc1&8GliyAQcZO88^Bot#yQJ*dRq3t?I^9ir?{3Td=Ww^^e#67Q$68N( zo~`x*pX**3TK9JAJ#!z@ZS0Hg&x--}ln>%&kp4Xk%CP=Hi}wI#8f^aH68Z!`55ath z1O-FtfZPaWXlR6GLa%|rGfbkgV6nhvhT}3^;+lr1(dP($m$GjxZ6cfw)eUdWbQg6g@`%sxY~ZS-w@Q@cRzgn;%Dh&iDoP z+j)P`YQiRh9TSHTPWo}7RDxRz4=3IOe2(Ln>4Jb3K@cHV!ka{rC(70`u@MsRlO*eb zGz}Rlvc5oOFb-Flh(@UWrGXv(%8QRZ? zx$GI+bu<%=u4V>V(9BUfWKlWGUrn%DXWhvroNY3@gxRaK=dU-1>N#?^#i@z27#C)) zB6G8Dm;3QN99o~JM(@1a`LOeiogY!-{KEw#2sB?XUN;Lxsa;sCaQQ_LG%3=4QCQ_J zn!N!r`eMZgW-Ch^lK2pbOiNO2UrL75P-$*UhigPev#eOz7RzyNU0%}iT{=|Yr!Z7e zwc=W(G|FyQfz+ZZZ`Gt~@crvlr>wp~Q`?$FTF~OPR_fZdJEil(K2+&1iOOZueVYG2kunYR+oC zwF>L~Hr;IbZ5yYx?N~QscglXR1N|NHwYa0oR(8Twz0+G~Y0iJSsB#(O3fpy|n<=;5 z{#Uve@(}Ft-IL#*IlA;pyf@_Dyg&MA@oBm*%%Axd7!E=WBrYhYK?}42)?#oJ)eN4m z4hY&1-yxeoA%prC+7IY~!w@feScs=x&9G&g94>t)@Jxn}Ukm~x-%t=DAd(+3OpPNk z>Ju3Ra;Q;|wt!0PqrOKIgtiAA)1!yh1BMHXf5!w|cMq8P$AVZlRx{tx_5(f}KmY6( zK_z}O?GJYyW20^ddjt;daROcA8po|P9?1UTEgv6kNBAuW+!7omlt-9xBI#WbwIwz} z+>(SmNp?~slZNY&%n{iKAfL&h9JoXBMHJX5;^)WtJ^mVEt5#9*m;N%NL*F6LrwWhM@keW%Y(f>UZKN;(KfF0`CtpR;~zh-;|#t=>xIY2KM`bh!J z$56ijAYjzM8ZJOn*l3|$(p1xQj07|*UJz=q+{|B_lBrg8fT+5J(3zF96Mih=!dJ06d*$A@o4ga0ghRb@Siuj)}zxY`1(47(Sb3CLb$fn>M|7KQnOk zYVkL!7!WLwNl2_oZ{`Q*!tF5nz?~De-;Lg@&Leow-x*|KQm2o>r8h z!vW&;I1w5qyoKvplwiXe*U=_w`-7$$5Isf8*rskq!>N{yj)ZpZz|JL1{jB#57+3Y6 zC$6W_A$Qpo-P}((* zO$wmeUsK zA_^2VPwfYDptiP(GmhG_waf{x z{@b&kXsH{PLAw)8Jl~Q6_1j>FoGw&&J7tmFO0q6`pp;fgM2H~}=3tfsQFXc#fPAg` z6y2Hb2ZWdZ<+C3+l4uWzi!H$29ly!4@cominhT}24Pe$-)qJ$90lQ)LU3W>aVmNi_ zZWwu>6%3}kPYhIgJRhsE0A9fKU(%HU6oG%dH-74L8<3QsbjA7dMZv|6!_yOg_Gcac zH|KYQ+YieJ?v8B*_7f%!KB)sJeqL%Vhz=0OSv_cvE@QuS18~40$V-eo4qBCdIJY?& zUGJ-K*4ZWX@ox4FnmUWQ8yml#JCh4!Fy3{u@hm;N+|D^P+~4ve2nMR7+l@bx)q?s@ zgBE5$Jp@GXUSB`9Co2@h`%5!d7*Wk38|tWDy2Ic^6YT!(>{4@eZ(DH(-|Qb4zvrIT zuXqN?3a+ksd%VQ{hD6d850)4TPOW}=LYbd_pZV48RJ5#L)(6tB*$Mn5JCWwh^S&hK zJQ&o3Wo&|185-^YanrUMrO%U=);6&@qe&XA7Atg1kj7=@YUV&hHLqz;$*H zlIUT0CpSEIC3R;$Ocq2Jn-3aZrS4B zJaH3(ZSajLOAsl_s)8h|sxEP(Huqcq*WAvY$5B4AD)l@?>6}DhA(b!kslZTHgeg*=lqN*a^8cf0VDiHx@(`uRQe-5RxJj#WD=b zknv7eAi11m03%rCxa40&kNy&#>!pt%dn9RoW6Xr9&%ninsh7|k(_yTU(aiA!h9`Ia zzu^{^j>2$q-W``i#Vs}@(9KL?W0L7ph9p68;FX;G@Mh7vf2g_UTMXNr1Pg>AN>h@S zX0cBVSYe5UKTs1j(6*1L_x+M`-+D@O|2a2qa)0lwLv?-Vd+oiYM10Hbk_nH9mf@3e zE~Q@lVYDVtK4aap`Yzy8>W3Fh!Yf6~((4hRNUmYvN=7=``nIFXvYm;sruFv|)mcrUYk(A{bZ)g_jxIdYk! zO-|d&)f{Y!1|Rf!|D_v}yjz6y#tX46k|nVR=A1GF?6mvj|Bw5@buc-EsH-^j9;s-x z4SGp3Sn7Nm zPFtS$>4j&4bj9#Npc98OI3>_&4Xo3jbZxK6vHCMZrH@yIdf^6pN8SK%+Nn5yX1W#I zI?^N5R_8%?_)UPOOD{>-wqpmmN*``U8mwW@B()eD)N&bs#M_AsNTEa(&1qc_)nE|_ z#MZIETNA+~wuzu!coT4G4;|Khi!WJj`xc4VHf!hnD1m{o_JN-g+fy9xPp>uR|6)C5ehXrrp=epUeZsI2_b!htLxTJQ{(E^NqMh0^SvCW6PX}qtTS+V$q#_3kJmeskCw7gzkaWMYB6GhjCqZ=Zf&`as1JV7s3dU*RN$Jn zjV@BMqFTvc($k;#>--DSzpvj2tKri_CE{Xd01>?H zK9R4P9NRK<88_>nYM9Vx7V8LaqWbhC!*iBz_ylsYb3G#oU8v^#Mo)m@K5Kk|9g{*^ zfUe_dclDU@5m@io;d!3nd7VM@wA)(f2^sIusiD1m+Sd6YgyN3z--@+m+eJusObl(4 zl$r`M@O8XNVVF*@#CtD-63k4U1@~Q#&tr(#GU3~nZ3onsR|3d&;-2EZZ+uC-+^#GR zyaJt6hxY~ic;~35?U|{&bo)=wd`&FMC_wq0R-R8A{DDMMay$3wii$xyC#9qZ4^~U_ zl6E4uJxbbGuU_RH8D-G-1%!U-Cp1i;MPCKYR?t2PrPc=wa_=z8(gn=9=Y{YU>05Nf?=N>TE9nC7ggM2ZwSY?tk$+ z2c(v<)2J%7S@D8U33!60W2}G#G!1uv6(F%{7Q#wkLg{wP+NA(OHGyL>+fO-*=Z^eh zO;QiQ1txL>K(9=#fh#^G^TEtIMpW8`Leik!T(h z?bEr82$j=4pY)gPaVs3>*!tDY1pwz!R7gIV&BjjWnMastm)8w#*H3#3Y>ayDB0~Kp zICsF)j?p*%NJb0>qAICjF5pSHZPy_l-|>eb>UU%7)|-LhW)Reh5heR!j+o78zAlbz z^=V|=s^@E93=28D-on|C{wOt&ehYFZV<&1EKkY}YbQf$@U^f)a8k05t4dk_mFU_g4 z6QiK)V7fH~IAaO#g+68-E*2MA;$IC}X&`=yATG$XT1(QPco~bGDHmM7~7V+`u9LoWs zE^Ox3%IS%#t0IQ7#PaDFp?%1J9DPBi)QeM2PErxJ@-VvWlq~@Q%2Bk8Y^cZdgLLP=~%lW274gqt@?vw|Z7#yFGS2;?)bL&nj6 ztRjM#opczVFIuQII{|f*xE?+Y6tUY?py=}vSp3wE>>Oe-O8eI7Is-T-ce`Oqs-10` z^68lNhQVX#;K=kHtB86#W+!bCi`lO3F{~l!2Al!GzI44Bn@Pt|r;qRRkuz4=l|e`o zO(ul%*t!wLB66+yn=8|w-?x`7FOtxPYQer8kUe47Gp>O_7(eY7+(r2iE82=TAPfwf z9ixhV68BGTKIJZk5vlYAk278!1w^>u5`ck5 z6v+>YqxaMl6qR6zAY?AUb!<0=7?Oeop!qvA zCG!+kRHYdTC|RUPT2P`e}gYt8)%=1$Mo{>;)nZmz?Q#@#FTjKC5wMOZ|bV=ulh za{`~od5V$VS7 z!q)&n$T?>?LP}|ttlETz85}=NhX78f^}5doUUKkZQFkL)oCJn@w(Sig4a8}e<%hd? z5C+f??CXt3Ijz$$MV1YC?0D<0#4yG|SQ_Q6=hy`xCYspd@G%4t2b+axLU{lyK-9khGv)NtsOgjsNyjnZa*vD< zl=^{G(+%auSXh1{xccWS9WHF7fl?M3qyXU-Z>5|y+%H??BH*2{+dK0PUpe7uBcg-0AF_Sg~ z@du089XsqAoQ22CWhyyKD$`uB{MIT$dXXMy)XCd>e7rkwudLRKgw2 zV8kXQ{}f?2QRgb+@@{I!R6HyyASyRi!-#{IUI4J*#P40K7#1m7B85^i7W8?;I0x_E z@wWht03(h^!-4C}fs3%Qg=2&^C=Xmppmf4P{wIXQSo}8Kf5M?|d)tmU=HGlEJ3{k8 z6t$#n-lpc(qAW@P&H=K|id5(UtMvieTJT~*zKbX_jkF1{<97UB6@{koCD?x6*)`gt z<=WipeszR(#ASN+)9s>i9a|x!*H4~5h_L&1e zj}45)5DeHe@aM!&>}9AQ*6%{m?AwplR(G=rx$H&`R8mR6qv|ity7}zK7}m#3 z;f{a;sC)bR+TWB@D=tC`p9`OD2R>_yM%{g1-wn?rWr@z&jo&-4d`_KA)G%!XWfreWuP|rnVDR|vxXh_ zv7GP@|2V(}u9?UrZ7ZHKz z#M-pkV4WP5*EV5eA4k+PtYQBjIwPvFzJth~5Pn<260PJHdvfJ|&K+mi&s6r## z2S14uAq5K`1Tsirz5(8!`(@(_pg1VbaZFORJOhAVigqKo^&{XCA|6g4m6pvAxh&!c z&|2o}dsiQsgQduhh!XRmZ(k-X7n+p`*yIt+Q2P^3R}mF}H_;5BG&_*wN|g$+u7zGX zjKN`O_lR}P|1!c!xc_W%$IidQWj{$<440M^Aj#7lZ(sSv7?Fb-{x9x=26ztqySrBo zeK4A?1s=*_IcNwK74RNxD9$5>3yf(j0u^))o{2@5mVCI2#o_D!Mqk$av0|;-;mNcU zvKg6`bK6BANpK}u-xENMo7Od;Ysm=5$|8Y{@50axQs>8t=!RJHj{%JYW`@66UC4|j zJT|BW1yI>vPkTW*tZD|x++=r6j6S@Qq)ca0>c|C^ zsvPJ(u{O0RYH8R2aztv@u6_z z=0r>&OWhUI?yw&4sJ{b3G;6p4qzTUV6z^ZcD_O$`OFlzI`nRcp?MH5UUP=NO(8TE$ z!=}@B|9{MP|Ax+VBC1ZO(>&F5bX=t{?;>$EnV|6}(V zNQfuck_4XH#DFv3*G~k9#6=f6HfgBrO~B|+*nSo<#=^2gu4!e8_GuWguOo*M2fJ~! zAjJxtvt|mu?M@Qx7_sR>&;Bvu@hx{t#yt{SJ78D$Ul&JxcAaKW-~Y$CuFyNV9|!^(Ev8_2`AG>i9TJSR8SJ|F3*SNp7LuH8 zNx<>)F-%_E3^ZaS3Fm4gRwDGyKY_LukXhvlTX8*o-c|F&n1hRE-C`@PCnU090iaq5 z8+S)@d`#g2ASzvIVY!8%OUDlIFJSc3StJYqg@BnEA;_5-y`Z~o2tWJ0wQ+2OGHNaS z#>_8;;4Ak4@JBg-L3mU$%^w698Kw~|>maZ) z6ulzzW`zt6pjZe{nEGJ|yeZ0V7N@3Z*mlkoH3q-2hYzNsWT^?_+E|4e@~>3biop=} zyRlHrT{PLfakDfXhB&CWi}S+iuU19!$FZ&GRiK_Dog(Fe%ZVSZBa4=`svFak7QliM zg{|P6kP_fJf)cSvVRXSo&<2%N64fAnY3n-UVz%D5|@{Np%Y<|Kxkddb- z6__0VG0S>MS^9%};4i!n%eHR#^qDW01Bcoj&bW@;JU?rKYgv0Os4jxx;O-sZ9g`}? z_kv$rGh*^H&|8>%3C%g}#~NwPu9FGf=RjJ3>a4waTwB5O$J=f3y7NF4#Xz3Wno8^> z4(^9sgI56EAZxB5P!Q;xmUD*>lY~Y2B>?14;W|p+@b3&Pe7f5fO!V0U9|6~wl%D#9 z;}#;b00yRE0YLHSHTeoTX{GHRJgp~-d2h3MuwcdlGYwkK)vZ=^DhF}ltMosVR|K1q zwJPR%%9TtBoBqo?c9~hwov^N;>qI)CoVo)5gTUZ4?HHpqUrSzDpb|+E^Cb7Oir^mK z^;HucCR?M&&s15WBX%p1bk~jytSM5^E=3apxyU8OOgc#0bbl|dBB%lSkAAt$d zKz6Tm(nVboyKE0|3VR_GkvvEVj?HQ$+atqG`qggDI{>O01T1`xbcv_iF2=w^TBaHY z7jf_)1R>3mftlZW%!7`ru#MfMO{(pz#azz5V1@vV!^An(b=@)22G-qe!hAdnW((a?l3YH7-+EVs0EZcI_Qu-Pj2-C0)4;N+qosKea|Hr%hF)wHr zy-`N!l=NtCOjiYCazc_w``u8qX|t3+7RMXk*SlSFc8p89zm}N-O8k=qZ3EvKvsGPKz+o| zi3(};E|cdBxLWk}a}pIM?}J-+FnoG0dP2a5!fAu~rJHb=e)?y&fjr?9P&vP+rN~Jct?z+Ri6T5O2ydwurZV_BoM@N-4q_`A ztX>g>I*`sZBhbWGThghvgR3k?jR;)1N&|38wPO&Zk!t|F$SL90Rq}P-qAYN4NLo@J zFU8fxK;@i)W-OPMTo9w2R02{^NsMF(IQ*kv;TSE2hE|Qiu^~9(?KK?oUBU+=XIN2e z`7#WTQR1p4o2CgRoKzE}u`LA>)d>p$A&3w;Nao=ofS0*rf~smwgW)|wk{J||tW-wX zHjIyKh8(NGZ7qdi%vQyXdS+@{NZcksQnAV^2uU1?l*7am;NkW4%XS~7f>tyM?$t;N zLd?EKLa|Hu;O7zmO~5KbYxmG9M9UqMYaU$Re12s>52m*~(~ z!^9=sx1a=b4(5ew`(}!W| zPE{u`5ohhft9=^s5mvw^cxM1cQg!+$uVQ&`h|g@8LYhT+-gshvd0&vl@mLVtHaI?p z(Kw$u1w*b!1@)^97PWds<@vH<0xWvA2`R`S+aP^nD}%QI%jtGFx$P*$A>a)^fD4Ln zlMJn@^Y*Wziq4!TeY~_Lyo|^}T1U}%6b*Kv8;W+MLWArDqVpsrI@3uM59iy5B-@6~ z3W?lm@NjNbK;p0^-lBbsDaUrRcGEn!7L0pFqGj38cq883lWj)8(;AFPkJNXngEB-e zxwdMT-rZ~C&{cuy0{iB|uXtgzU*BK9*^)8C&>5^Eo0b)CuiYC$lOJX|&*BPvq$pK8 zB!k3z^y86@Q7gkh&KPF8#j1)|@^BBf?*92N?-XF6S#R_~F~oFqt=EirdQuVq?^x|n zb-ObPY!-3|c!^{1YP!BG@TGZMAp5PUeUm-ch7G!kT2XQi0#MkN-m9g*YHXg3Tuyjr z9>*Afd0m%Pu&xOw^P7PBaathPoG8FG-LEqbMDw9|F1-5nYRlw9N-I_;JUMqJfOY6h zMV>DN3pwXWxN?UPk~?aKX|jG03IkW;%*mo;BMGX#P__u!){&&rnGKau)o0X0=Id35SLBtj+#@Q9tdI{mkA`k7Ik-4uEF@?RgH+6 z8&u!&@LE7t_xC!@y$Zc5!CLKo(VRJ~tCR#=7FP%zPJ%OBhf+!_%EuF9DykC{ETrI{4;|f8=F+_Z{QI-b$MgEJ4I^50X$wS; zoMlodp@HgA4OGwwOy+u(@cc9P@n#$tuO8(Uwrl-5%+*2TIfxVRT)>Gz-5lx;W5=O* zm3>`f?g$-@6j*RL&vk+$hjC<%0!jvN*(rU!vDWD7xn6QBCG+Q5;WjH^9~tzPHS40l zk+(bwkB8w27x@F=|I}xTLrdKs@|x3Y<1~b$aws;< zOLnkRZ?r#%hVK>-4#y({VNC#Kd zHHL=iCa7KdU3St(zpJ!5lX^x;7x>=<-ESilkeOi6K1&*ck}$N5E6?1LsLK#dywo14 zhxwybpe#LcWV>8?^Pu_UtTVvJ&CH>JH4iRU4UIue8>Cs`X&IikrWkFp5%o`o2f$JN z4=9?dEuWlU1~k|HU3}3y-v3enJ^1`sRNnLBhp1So@JMH@SNG3|Y!vCpua=^v5m{Oz z;xj|IycJO&jd*(D>erV&=wIkzTZ;xonC~+^P}-zKT{Y`Buv)cGD>s$Plo(??FLTPShShYlo5TKKoh z^KD?9AaNjnAj1Q_CxiU{hR*t<&&P6 zwE;AEQOVTW*2m^ zzw#B`m?*c)n7+g_qi&Pxk`15*3MZutYpcZN^vU%DESONcq__Ccbco|MTosU9J~h}n zltZh8`!PrLsL5zms%{__F2>No<|zS7?AHr+8?JaWZ==VQ9UXaiqC=FyX$@}r*(RD2 zp4(piJ9kmoEyiFW%2NXFjkVcVG<4=OcL(CuBVu3%6)wej%>XselrP=afWKdxd~i>2 z0n$1VWE73BltVsv#eQKK%HF8&6h^e0G#bPvsCdY!_YjwDCprY;^NG<(F(xX6YV*#756ZqwrQl~4c2wQ=`Yn%_yi!H z9;X+8O)ua4LXG}a$aL4Wxpg=Oue>Ubp!Dw~qo>k|4e*^E3{%4IOI?e5ymFjg8(grr>%}6pTjI zP(M8^{2L|S*KG{Pgx6mO@^@d1mXdaCd@dNn3%R8cblS8MF zlwHUDUX5Hp^7`7O<24et{IN4i_Yhd0Lpq=Ywdj}!g)QDSM*-h*IUW<7awv}f5vfo` zRo?%Jp9O4%SU4z97rI13`G0zSY#EHLU%VF&P=a-?8ClqF`3S(-EiWE+{S02=>s~hl zbj;^^D{Tsqq(~X#1clDbMsEJdL;vcNs2ZUXU~H;USze_P1SSw+iM{$u=aFy&yzJlf z=??}fbr6ig*A^f6QB+eIh6YCJ1Y-^Oj@%wlD0C1HM&zAX9ri#z1Xd^xaRsCM7Y@zn zRfhIv-{;fX8Ekw1)e+<_qjc-QdprRze-KPay|bQeg!m4Ae|PIx5k)OW%VF+}HKWTW ze+S<~3m9WyGX9nPw`-5Fe{YZ7Po>xJE3i#T-~KyBnOJ4Th$J92$g@Ba1b3pmk?JoF$7-gkqDO-hu0=#3n!0e zyzBm+Jmnj67HxAoU&OMk2#d>$2h~)N7ot#L!>N%k6hyb9CxSpuX+n_zdzB=7hL)LV zEA%taBw#CpbGH{k3J{&lMb213Eapa&z=lY5H>$5RE7i7@m6~8FeVjG4=KzuuS@#F# zsW)Zj4sr*W`CA=VBQ$@3=-}WHjeG1MCT!O# z9E}m2+*fs-M7J}=#uP=GT5(gbqIbpdMhA_Gw&zGf;bj|%lm}f)P}>BqO|Z6;TY`(C zC0*U@2!A8;H~Rv&(GI*8f2ZDV!Y1GyeU;kR;fM=TsFW-LflSIAbuPH-p7%jodylxx zE4YrJE`>litGEJ4WmOe8lA_cR3QLt$450SgBPMZw*|P*^x}gSC!U=#ENg}bU8{NAQ z;2lpIK4K+nbf3YlLGY5Oxdy1Jt}C1wq`@;Jso*8kJ)%lrjjL|f|7S+}M-?%sgf_xP zSULcmQ(*o3i%+bg2y0@w_cixi08MXH!zkk+Q`YaJ{GJ8*jWsY~#f~T(v|J}Rxfd!&1D(#cA2O#sMCTvAf1*4GNdk_g+PY_Malxi3Je>d| zTlr&Bzrmw-_{S5PCs8}^esekwO(Q8ncYaP>=i%;Dn(@5NlDB*W22;$H4#6OiF-CGi zHZu}e&SO}>7wa2$Y{$_x-6bnN=7V)HtIKJ~ftoam76yaS0Vqap?xiG6+g zGc8aaChgBT;tz$KBxr1qbvwN?(=;*hMS1rD|DMN?!`L1O9^8w6fk`<0SPj@hP6GB? zBQ4s^N93K5MGNBq5r1*Xl~A%|5MY%GJYJ47XQ-X#uB4(w*a#3$aDD z?j?ivNM-lXLzE7-3z38-)QnUL7C;gb3CaQeF|ih}vMNWX(R2`jIK2nV9ERMOLA`Z3jkanGn`EUj1Ls zpQH@A5p*`KawH#y#OH2!KO&T*joG`O)0si~#|$T1fr!c4ir?l2j!h-^!r;FoTXAP2 zP($26+8RuKaNmmF68ZO$VQR=S4fl6fd;^$!e!82oxepK_IQg$GZ-lK7H&x2##k138 ziMU0I*(%>-;y5*tkrbv}v6&ZeF71qNm4UhMD+MFaj9;hHc`*^}6-t)fiiioukcIjy zwBLfJa}c%C)f(U2X7kthQk(U#0!D3V=sYi#0ng_$Z^j?Y)|p1Zox|H`uf8gpT?p5Y z*sKG``Pa{ng1Pa9UQaQMZFmzeg)hT1yBWrxpSj;|4(|Ko0SR14ah5!wxn)+~?GsZY z{@abyW7Ro-te-gHp4FS}onaupsZE>2M6dvWOuS+9aG8+c?l%%0h|)-;p%ROL0_Q@d zi>2-dLggHNo8G+pg)1fs^pdb7(%g{I^NXXRNPy|%2DH!*!6crk?u-_CNNPPm`ctP{ zkWA@>_nUd-fF=RRF_^+b4sW8Fdih`7GlFfRM0vq_?0qgNVcXTs`&wQ9oT0$@OmafU zevvpnSTSe#U*H;wU-NGbWTX8uXS~nu`3ShMCU-vJrWm>;Rk%H1WVbs%Mp+M->h(U= zv#M+1>H=^1OFD0CR8AYHL4$U<{slwg_&hU5Vq_>{nC?4RN>jttZg&?a#GilvkY|8q zsEnmK)~)H6AiWCUIur+&c#nLGS>Q4c%RaR0AC?q7uA#pC0hJJWgI&N&B0NP-wm$S1 zdpxy^Kv3i|2fPN1ne^Cyba>700P(z)Ehu(-!Xir+05MN!qhsolCCvq#>ETfC(V)4}_^4milM8mojw~la zdXVN+SUAl6qq8CmpO2TBks|&M^xCi>{b3YUV&d~S6=3E#$(%fwAm^s6@r%}}YVc{l zP^&Fib{c)sUc~}Zsb^)z;p-?fXe)|Dp}{H#Kn&-RyS;2&u_Yr~nBl?&sK{B8wa(j} znN&0WsY-d@F&6J{=(V%B$W!c0LsX>Z8mPQMhGX?UcTnAAmB%YGqL<6bhV(<-NB8;a zgL-v^Pr=CFPoP)yzkAE)Ns)C`Gqi(M&=M(a(DJLGr54Crpz@XKwPR-|;M`eAw>t?t zuC-Gt`b#P<%~F1!APK!}Q()MWMl!mU&74v9Kr|aZn2yS*pbYSD`W;wZ{v5S$Pth8y z9Cf}@>aa&e#lGGg2KkC_<1t7ikx`Fkr#3oCUvi@bZoe>_9{yyt?Hhc38NB5lAP3|Q z|CS}Z)bj&0$INH0(ksfkTcoD%53eDrt3(0`cbS0ZVSDVW%kUN2yq!~KxOq8=r5x}AgmK!Z) zo-AS-a-KY)9hA(;o2=d+(;XWr7~J>g41B4`!hRkUyePmfFvVdQ+u-KI@|s<PH#b z3~JgU1uTHEw95NOz2sL2kE#@`KmhoZvLwwCcwP$%*_h5hP zrhNB83Q9$0NQa=iB_JfP7WGyEM6_=6&G3Z#mFD!!8;hyfqft)+u-|o23EFu zCM`_97};!-*%^k`j%PrS=GSXhQsYi0O$`9CsU21s7d8Yjibe-^0iujHDaz#5bp10t zK4U(3t7-CWS#;V%yE_sZFc~ykX+?XOp(Z&DvmBX1FYsl(x@x&_3yh(#`u5o@*@x}s zc-!abg7Zal<>mNJoB_#G?bZ4KTNXZFu+VR?-g0wbH9bJqhI!vrd#eC5y*(4BKLI5)+ zxhV{Dcm^F>m-#`wt!N2CNH$Bu_KiJWLtTv=uez3{7^^S*5r6_ml9!MCj*_>3)Q1h!sZxhK z+LM1i+4ScG%u`!Bb-{06s#^eTzUmZO4nV|z#Q$MWM*Tv zgp9*weeMAh=1FuhsxNf{9{MH)gw(xxF!#4AIpcfZsx(lZvFENGg?-=J1cURu>Xm8o zSGx~Z!NOkNY-Zw7f1|-=rS#gO37Q5`^eYZ7No6OD$)4d}_rayZXJSH7LYb1uYM#DD@|qGl4RX=e-ExwVk*r>&FGsJ< zNc%kY>Q=V9SA50QTXEHtp<{Hohw134{NtLGRXwk+ZnW{$QmS&u$ofkWg9(hVHX7AN zllM|Zxl%Aq^A-`&?>eL>?%NX+y1}PThnHN8%Y}}d5^@+MXl=Su{ukX*JI6^N>A8o} z(ra6~?A`Uzj82~3x^JJu}6AorXC&}oux zAUw^UJtxh1JHSTjl)6fM{4vb%DQ`Mal?)e)L?Q$f2+CZP;FY9`D*)y&R>+8FNNCp7 z%_a0lONfEA5~Zf()dd7;>><@?9o+aEp%e5#>ja!ndx8xilp3;Ig(YFdyZ8hO^E78y zI+VQ5PW!G}f59FQPMOfBXM^?7V?*g9#2KIY92wa9{fPOAZ~66)n6;;U&q%^#kLx{# zw8t>WM<3|Rp+Qt1GP{EA&t}~oZ9U;3w%MU(QJI_?tG6CCo<)#}(_nk&7jPwgk+m3> zl!IlVN`!$_$tY(qHgMnEF{a@@xcZbs4%Z!~#w+id`^Tv9wbT+}C~{`VK`m8_`b!^B62t#~ z(d-T5i+0ivg|8fEx-*ldLo`hj@6ddj6+-lla^ppiZ6|^haOy%6_N$sARkM*P&NWx(q5Ektyp;h|KkyEWs zpXfxpEq&B0&e141E$uppHUjU|tD!y@_{xDHay4r6LPV4!ZZX+vpdX&&%=W-;IZ?>$GX5D&UQQY%scZ^1B4xAR0 zp#UQ$5$qKJQpyDIcaJEzx^j?>%*O`xvshZX{AI9jBAIzA$1HbyiTXC6x;i>LK4H)i ztL#1cLPJ-sA)VU3yS+oss&Bnes$De#8*U#4P^Hdav1cfay+hn5WXt4ZB8e9=oaDGU8HZFTGwTdZuo`1oE&c=zos6J>n@ z^a8geDsmn4K1qLaxJza7u=mL?=oN8|3>qd|KdpV`*`%+DjQel5O7mB=k<7w74(eWq z&cA^%vR*);s|5^AQk#s`K7C}Rn4=bKqDEJ>DdK`%iYcmUHB%n* zc@*0|&?^|T?KhHGdl4wJAETtyxvmx)-yP2pT7TAehf+&(D=y;YN4k|`u*1--h!696 z8#j&&4ewsjtA)9lxEVx=Ec5j4&~YXr!8+&9%s8K;B1OqYRTtmUEFK26dhcGb^)}GP zjevQ|h@+Fm<4LNv@JX5OSEpIe5; z+V&9~x-%-oPTjW#9ln7 zfb!ktH^L4#L&|RY=-A-OG!wI||1;yF2NkdT5UKl4j=t&Dna%T!sP+NEAj<&^{{VuI z_#8l1i3we)NSF+12faR1b*aSny+L+SlmP*E;0eM02PpNxF~kQ@=ENsf2g0DSVaxQ0 zD5_mn!|xT*uo4RQaAS4;x%^l)%Qkx+PljA;XR}*ZI&WVW{I`Ymtc0+fh4GH>PJHl3 zcjcOF>`eLSW9GFVA=fO%kGfT@i1{hFid&C1fj1kjRrCS&vlC&rpzfMXe3O8ug?ebz9d)6ZQ1a z5(BplNd+h}cN$1a2;(r^=H<^w^x8BcSpf*eKo$#Dg^k=d7E$(JB!>*81q<#=h5AvF>4EY~6*go~VOUKvyK z2I4rph*rrgL#}Z^wUAB2ky%o@eKOQ#odE1Y4&_W68b8=dLOJt3j*%eU=v-);)zj#q z|Fb?go%@9coNSEEO0NAyPf?{$2s!!^JGa*$`L1d!T!Bij1-xkV?n@G9BlAiSMYD2< zHIkRlVdCJI=r(dYv#&O|JjejEq-;{r2&uVTj8=%0q(S(mS! z@fJK`*%Q96ZEZm$k{L$uO25iJuBCWWvh;SkaW>0c|Hahnn^K zegjcyLn#z`uOf{P+_g^?0JB7%f$b(oKu099>V>@9a)fBLM5F}E2{AT_zw}BZc*OFL z;;Qzgjll#*LH}sBnqNA@h(uJm4%ZP7>{_SpnYvI#eeH`Nx)_-^tEA$xd}UTrsm|CZ zz6(-QuX)T9S@nKLeYUucK>atuo~v-xL5J4#Q>{=mX$HyY zfn*8p`VPDjRZ&kDh>6B((m&ZlBxL^kdStlX`7Nl@aM*h>>eFN2{(-RZdh{%>Z{!4s`zzYA^O1rgP-dJRSQd2r##*$t@W9=Z zt0Yaxmc&B00@mw;uw}o?T$O48U{j;Bas#1^jXfgbfDsFO?WlU3ky^bZDY&D1 zdEs6VLVsEwW+ez2Jve_veWJhI?7BTp{$nXzr)9T*OLd?)Y7u-|zEwA`}Ixq}qEI4#qSdso)T5g4V$?@q*?i%&K#)ov@Zb-`0S5rT z3B1IA*RlOR1g2cN19Yy*oW_5iWBMK>zmGj53f@#6)BolEybwcZOJktvWs^E={}1=q zDWa0`>;ci(2NnLZlhKcm^vuTwoElc4z^ff-_=CGV8km6Lhai>#tPeny2E+*`+goQ95Nat)#H(v0cl7-OATSOe(mb~c2ph~DD zFVBY_oqS$yfdpWbTEGs{-2@*4;#6TJxq!R(WYyIz7XFUWzimx%J)7I5S90NbRIh2I zCJ=}EMHDAE#>)i~rgOaR|E^rC#O+yZHizxNFWYflfttKaTW13o7FwJoaG?Kyyr#^GQ+O#6CLKnGmeHx`uIGnHnEqKkGh-8s#zdPg?5Wb5~EbPj~FJ zZF*dJ#L(p9^onr=9#3aJuOt!EmnIeFW3(K>)Fa6LXtiS4A@W6!wS+*+xsU47FeF5t z%?h?`UQkU4D!>J=UIr8V_4P;j>@SH-0_x$zv}oO@Bl0yz8{f1`@o$PUJi3e9sM+nLtA+b%nl^L z%>5mq?EM1W44;Y5-W9Qwhp;ZxX@$6yr0eF6vO4 zIrNEYhyfJq-*!jc2YeD6#3CrUvGR^n2u+4dkUrKb64asHLKK0KsSpswL?#(rf^Ikw zeJpWswrbF?J#6?3z|x=WMin&R`k@27D ziDz0+`SHzs#yug@i2+YRTL^xc&&_>*GZS`!=;;PAX{b_9*-CGYNQ7Efc8DZ`c45#N zkzN&R4qVv1ZrQ0JJ;8Qq7)Cr{*A*C-FNW=iAzcAD(w>}j-t`Kml73+k=>?8!y3CNz z?K(UnA@)YK7%YH%#pJ-&PY;CxR^g9}x3)qvO_s|E3KRiR@ZYj|mv{peQ9nLhQ>Uc}PD|MkDOJi(bi?%V&(!@EQ1CwB(29l_A#ZW+1l|mtaqUAo) zBO84xXSCEkbV0?170HJoEFNKkGQcL^j~eY&lOWEkg=R^PyhJ3(?_Y^un~15MG$C|< zzUXy$GLBaP?jx)9SXd3K$7aDw@bWqMHPLAgHP-4z;@3rP7Jzzw-FSz)BwBUOz|cE( z5q>Wt5-zx8Brub3yzwoAXhItD@sMDEL4lB~zqmwzTX)7t|j;1HDC(L`<~> zJMjrmOrHErHIHM;Z|W-pN5-e!Ee3NJm&ZL6#g>MlK`m>6q<0yJ!#ok`$$x!OB}VNb z9nS7rEZmZCcpHidpRX%^0&SU=(|O-`N2;4>^~S%#iGe7g*+03Jg%BfWgnv&@07Fi7 zk!qqdvB>nN5hM%K(AbS$rt0VpUEbPS!$+F%YHHX7@cXivYk0Dfy_IktLj^6O{xNF} z{@Fy_6Whcx?VDDc5X^1_JdOzl#JGXI#TOU2_-pR)bb+IjSC7Lv)F`)!hifF<&5n3@ z^k)9r;@>WSVcFo))CXhiywhu_lW`uw#Yn9ZdWoW`s(+{prX3TAap+N=;14PXDR`fB z7XZj6V(}qwS;&tXLE;Mv9&}y}1VIGkfW{{=TchAkf4p+{n8U!Y08W{w@=c+7wk9u=4@lo1+V1k;Pg&pl)fBJyb@P%1tI5fvdO%DG6@ zyX~z5Ru@a~cvrpaK*=9Aq>XvcHIWJt79Q;)m8kAF@a7`!5o})cGMNaE-LX`P4&S+n zZSk`OpQUG1YInZguY-2t;nCwyMfmKA=Fk;hPT8XZ8~$Ri{eSYRnv>U;vX5ZzT;I5q z(725rf?wS(`5vZycu`fmJE7(j9A1?jkP+5iPw2aTwfMU(WfZ?oCkUArp48aIi(c8L zfB2Im23kLEdy>yvl-kPM7a#U~F##HLJLHWvV{cq|xkHCi^$w>L9R8UOb$wm&GJi_N zk++)_1@2_|p>VsLa-J?q56)(p&v?Jv)_W+KaD8eg!*#D?-*kETw5pT{?F)Og>d5c_ zHHOr!+q8&w&h$zTAh5jQaVe_Nk2JNDxkq6?_yhXAMYm7N%Q=fk!6^bZ^4NfW~50>|lcpISM+8-YnD;>x1`KYHa4g%qRSUfG4 zqWCSbdrnDIG5VLM zbq99_b&(qfYOlkB1@SXB1NYtW2>K(h)ykEq_qf%MZxs^;J}VTiRyfj)00W8 zszr+4&LEDw)V>803t~V6Y?^7>j;60QeWpr+Aiz~zjqj`A6u3ieUTJ^x6ZbvexNXSc z?f$WFHt`N2{AXW4+9}oy)`%JpFp$bQ?LWVMxTm|TDhf3^Ogih6|Kj6?Hq4fs>Rrau z9pC%5{5RzR37VmHB`RR}PL3OJ+qKu=+d+uT6DAn!)QUjgzQBm-o_hegNpsvBgC2!v zgDleUtFki|z7$GPVi`RbwkRui%QZKx;oa;5j9SL#(w(3Td$hpSO(+-hbkoC%_YcJDRf9}IBro90rpfkdw{swGC_tJ%4##uo zr}=F*rTT8>#L5>@hbZ3_*5d40ipR~wlchB(_4noIjPtf}J9mH8; zfy84%;-I2ZlQS`#)hdseRSuH5jcjMGmdv6$`rF+3)Vjy zwhnGhvhNK3uEE;>XHerW*m6C7`~0VF-ig#8jm*5+y=Q!k&MfyMLV&Oxr6qR<1 z3o=8519QD;AZInIE27QfOGL}HVa0YSB@XYxpq4=6b<;~EY0|n`S*Wc~rs~tpnOBc^{=*56J@61Gdfm7ggG}MJu!ZQqPtM$OV zk7eYr#*7NZLdiv1PswLv*Z+Is%TQ7|Lr_T?K^0*4)$FyKk53ZX6GNs@XSJq1JoPSu zVFGLtf2Hjy01)9U(72DuEprkG$WA@JranO_CGPJypaS61B+0-9<5JY9|E5M=gW`15 zbX}LwA|_J_DE2~8PgB$^q%YDXsP|Hhh0g;2!0TDoCM&jd6fnHTu!H|@S)?tCsuaUkn-HW+dp*R?r;{-@upKG27EKu z9IW4euq6HN|GyJ|Tj^$B(xE&^LM=7p0#Kl?Dvhx`MHH?xjkY`jN>ZhaZ2$ohDO0pr z5EM8mQ;_Eq-i`yKlX0`$hur^u$3VB`O@$h3)+O>|q)1iG8gKznWSy3+nR%G(uieU{+| zpBwgXnEK!SpHqmWMS|r|*UVeNk7`3kKM<2EP40OGF- zAvh^WmMufOhF+CnIa9#&f%u|OqL;=a`9y;N?clxF%*E8Ee&tq(1pmyA{vOb1f7bhZ zOV|~v>#5b9_42Ib#6ho8Xu8R|#@!-c(&y+^{gNJlxuF#|z-e4~MKomFn9s!WRl-6D zbYU#2a}11!1>{5P8_-rQbvVhq=qK5;!u!W&0R4`PL)VdW>??MJ*m%N@w_Jl2g|H3m ztwW&&_S@7skBb)8C9g0thBoEo8~&96W3Tt%iEV}*uG>#Ai=TlEOma*Cw;_6sw@GC2 zsrlb`6=;`F|K0VtIJQxyo;bxAT`SY=#FusuUqChRCDyA}31DeS6b^+yG zyr7AOR;rfsLF{{bCuCJ~k7Uld(DZ!MHoG*|;m91m8!lsp#)CDMPA!O@iY=aKI*EE>sg18%XnUjN+=7Ak zknRa{e*fB>8q-u83^dK=f7v^*YX9H;a-V_Ldl~q_kg6qn@hIrr?i$WW16))JJ=-)$ zZ%T)f53u$JZJQJb&>JxPN-l*XiI9!7Sh@oH;VumkpEfuQU~J?W$AMr!z;7eRm;*Av zTK0hBbQ~zU1DrG`cWbc0n*D%oIYb?Va9CoJtOx)|zMZ5g_2QIl!|RxI!D!Hxy+%^$ zj59bo@h!jl5!1eHu#p%Lr%Eh_>m0FrdkQVKC{ZNRFT=WhRZpsP@RhEgw5MZPIlJv; z%SM1@w%hJ9yKv(@LDd@kIWc|YCPFa-o!0>e3PHc7rv$THW)K6SN=t-*LSYgjiHbB@ zhF1Pc*Z}KJuuo1%M3Mo}ibl1!H7;no@+7W(B-TwL%SK$4a8 z8SmI272BjYDJP{cPf_}B%u?vV^2!NeC``L5Y}>V=d`wppS2t3iI$QCQa6cr*h~~}5 z=n{H$>BK(`g>86qJH$Z7+jZD-k?^*AcztScn+184EP;|~QQ^(=z=iZQ+4JZk%FCm% zfJ7V7-+*}5qz&3`AR18}FyL`rZ5xCV5-*!{%FqVgp$%ELNmo}PAAi)1a1CQhth=Qd zA_4YJ(-6tST#Zg&ad1s;hdFMoRCvy=e&)EmF9NWch997rclj}I0VCSgJ>!nMey}>g z4aVvGTzC4EzU+^!gbvr*sv|W^#kw)8a7CxSIEe-jDu^MOLKd+`buLm6;1WRZ16cyJ z;!uo!X{ykvIp<_skp?1hTC9{;a}w6Gai)pBgZ2$Hp>sL)VpQ?Z(nNugm#{);s;CGn z)<-ktxf1=p<_c|N9lL5n^!s2vM14U_g3;kLJRn{lQ6iaj}BUC zi|pLLpjow8z!Av&bp7DeRBIH%(4@>Vt4#igRT{AuPlZU4aJb~ZA3y#3lly(*>2cxF zW+Qv~AKatKOvp?4ePIwsYvBSrIlC@6-kD{iMj^XhfmGo!O$4J~Tl=Xqdx4XkfAYz=(69E`{0F?t!Wb00Lk zjPfBGEL`M38ZQh4=U-&u$8K`0L?WarrZfyAXwoXe&@L=q3=E12_EmbwCa$H2%6r5p z_{<47e}46&P0XNQ=DC;CZUJifbsbc;Ez3P8dbM%kQlIhS;ugl+uYC+`e?j{7_Y+C>dVroL_%h z{m-Yh*E|XlXWgY+sJHGG_VdqmstNbP?|My5c+M}5TX%BQ-^@j(|EV?{C$kbS1jSxm*J4;@sFv{~K+V zzN+Yb31q>c9hKhK5`CHArrK` zf|nZLB5@{Chyq7diiOwl=00d#aUs3{`g=3S7bOTU5aLx_XHWyCPztj-68hzqcYqs* z+m}*kzk$gpnyGwkH-?}^hxhlO^LVwqy z+E~JE{K;hE4dC?q|DE_a$$wAZAoQxTTNBnW9zU+vd)zGJYH9GDc9W;}hyU)oke7Zt zhG~EOt<@Xii(@1C;Q9Pb`?0^LLxhwlI%VkL!`HJ*k~dK5h~#VnLKfk|?r2=ia$Foc z(>t6mLs{90agebC-W@4cb&z@@^6Ek|B*d>Cu8Jw9C)xp4qAj>q%wJf~8XXEDl`MUo zh?;Hhv+Kg2q=h>v>@5imwZdANY`gzPMp5)`-C11Dxi&T7eE>lC`6TM|(t@B+>-dd? zpRuj&{oOL;`_`){9i;)REAcb4?&W;q`(d}B2sGaM%3WPyY^?a~!wuI1rkWDrw|vLC z=OudzURvF@EvU-zqV*sP62?pDD@8uZJl)(PB^Z>tegOo{AR1WNr z;V;0r>ZIzWc5sm6R8jtYrq``Gr7F&JW`C&Y#^3ie(*?3k7)M<}@ z-lJf%D@sMyIIH(bB?IHQ-SyD_>{p&cdC1eEK#!!-(-yR829bXNN-fK;Fv0|}L1Sd) z=kMyDAZS{`fvxL23#Jiffm3(VBGP2np<)*p2RnD^BV3*B>J_akk|>At8!-tR39gp^ zvZE$ZQJ6%(ERrf>&xBW~(~QXzuoq2DW^EpUexv5pSONhJj;uqMS#(0PBNNg|RJfMH z_#Y#{*0*}KHH%?W!>`F+HwZ^)?lQ1QVVOWKRas%-9KG>29L~JYge=frm|wat0LSiL zzE{A+o7p#yK!gM}bEQS%JncsjiVz>G`o-wb5YPBXs(0XS1QpJ;8tv`oD@}I0S?FxvTr){8FN<|J4bDX_@IgD6`C*# z(7aHkHavs|6Wx!c5o^_YQv@@u@j?d#Iic7WMhe2;rbx0oTcU8(IQs^MH!Ol3md~Gm z;Rv2H3yQkDB+?qfC!Y*sdW`uSA)krBcj@T$pZB(%vpiYvg9~CzxdLWsg`eXQ5WP%#7Yy|X; zv$0*sLXQgpQKyvxKalAT(AJ22o)&?&{7PHiOYXVl_EVxOkCN!FN2w!y zR)Guj`j5sMKl;%Jyse4aw2#b!I5r?uujsxZ=WPwOuq+kaVBEe1m*By)z_CU|6w;$@ z-7%ijjXJaqD&KHjfr-3_xV?b*$nWt0+zW2N>*Ol)TDIcFc2i4-;Nt}41s?YzE z)u>IE2#46dup~zwgADJ3>nAb3^yCOW6Sj9;(tQY&bRuw+yhu~`N+&_dYddh9rcKzy zCMSkA;!VK7C0_2u9!lXL0I&@FY#pP2MOL6Xb%m5n$KvOot=^#;=R)rciR$d9-$2F! zN-2{mk+V6a;v&^<$BtHBnuUC<_$inYX6DW{7feXp=`|qcDXV?nG$}V$Kw}sb89Nd= zq~iCEEVtC`xty2Q3*yKH6G}iU8v#%EkVr&iA<|D77;dlmKI*xQ#pjEXF{LfLoH?ya z28rJiw(x?rd!tBry9?w?Qr2m;E3v6NdA|cKy@un}_3f2jEqbT`@H+rPiySDz`zZR zPAEPb`4nSrq}h$JBlJ|fJ#*O|gX;(!i<9-8uoH5v=#dqv!zT_+PD=a^UbQxB;dVd? zEmY4Wkui#PQu9hAhBQ;@D>JepCU32&g}Ele>jP4^+k{4I|%Wj?xUMy@GoAUG+Cy?sFQn8%CWOI;x1q?brS>RXts8^iz zWi%}gs`&tpSWG(Z_QDz7qo#<7TnNE^qK zPBlKxKTktb{GsU>yneQV59h@=7qok068i3i+j@=eQnQx#mWJt7yV%UdjTp}s_;E5h zVkH8r@TJu(#sh#MsR+p6htvLclk!m_15)vje~jW0_hI<*9e zzHyjZRDDLr%(NgC-rdt`vgWG~pDPBH1E=O*D3y|x7n#z|ZIW~1l^2nc(>31O_)Hg( zve6ZtCH??i3@c*xgIisEYHGHL84pakMQH@7sSOc16Cv-Bj={}#$4@xH16^%H1KigM zG?;#s}3l6v)*>)a}Tg9Bt=?HJxj>HB@f#{7gR;j_{iUWfXWu|`B&7sYlDB~<4%V;c3= zQ#EOz@$OYIJ*uPU)7ke48IN7X1S1*dXnz7Okcd>E%Q3i*Gq(5p!EVffo9u{hlhG*FcFRy#5;&@Pl~2 zuXk@(ibV6n$9McWK#|?=y|P`i%2ML5xr;1iPDQHkTz*oCG^pM7t2{yA!>f4qz<{P1 zD6My}%pOp@bKUGRmZzhZX9EF9Sg(Xg*h{eB9fMc56Pxm)#_CTh5a3?s#QWzn|5<&| z?vnnQd9)GI{^a-|ZYqysXveZ)`VcfUdB1s5&*9N94X;nhS)(KU*hbedir&@roZ5o8 zdvV?&uY8YvD2f8@hLIX!dng0G^ALWQU8bPv#*KFAm*)L|8H4L4^3BG|`fYC{X@L=k>>nz_2vfi(r3qzlo+dKbf{ zMSsj3R~x)YJHGx|S_PzqSuToM1rh)3ys!%u3g4LXNX8@K*1@1LKLZp#<1j&L^bMr7wc0Sw0HtA+eTrXL1*>GPW>3+VR$$x- zO{@YDp{jyNh*FGx-cvlhku~x8lK5P9V!i$Jpkc2!{Y|sMQ{%)3k<)_VjCmYi@3Wx{ z*&7> zLE+k)u!*6Gc~^_%D-1ByQi{PMO;YsJNP)7>-X;PG@Gu&)a$>xE;? z0C^@x33wU+pY+6ODCZ0)1Z1YY1%0ldurkJQEJzn{nVZy?K+E;M3uR4TkxYWy`O&#grK03Kw3 zHsl;1xmApp#Ovo-)%~NbrkUw?ooRt9Qy>^KQyY2gbY+?)F}=4xHhYPh#van&u1Oc& z4F$zTBEZ{kf)viDp7l)NbxI=?rq(L>d!F^B&$@o}MEj%P;>>@z>-vv&8)C|d@k;=J zvqFadg9nPQ7-{aL#ph^f=(f8nI07Mb3TwmAoSq+)8aKu;s^*FF_JCQCFt(JuGJ8<{ z9M|dUTXrkX^{EFYRn&;?-emMS!v4!cUu5ul%jM7A#|Cy!Z;#g8koI%^rHK5%12xWu zLdN?)bm>~13o2sUtB9HkNJ;(!Op_BhY;9m_Pf<(m33&%xPD zY4D-kd@x%>P|;7oF1Hxup1A2HU3o{ zX`s!C{ge=(SWVVV9;?XtL;YZ&AkWdEaH6jf#DfI2d$B6^*6-$Le-Y<+FzO&c#hv9> zbF1U|Qh{dqwVPFdzZZqF&#s>Sx1oSVpoud?! zG1+HC@~+%`!&;tRYu2+;iX7iHNg?>?F}3pvP-FB1+o>j^cdRDhxyPFco9Kt~$0}5( z;JfHQGM46Tcv`i767h~4c+If=i_6|UWM?0{`mVe3;fB1KVFz?WZNF@}Y%^4xoXAH7 ztb2~@_ZGKoC&D^ixW$j|cOBN5RP3iH{E2g@Z+V!hFELmXlx9e(L%^r?C0z47ICj7_ z1z$F~K|>$FS#}Mc5*uSLn*0 zvFk1+O4DA)(tZlm*WO3LD;IewaM6?jZ=w@E%+ax{?SlVSE-S5%e%A$z30X+8s}aVI z<=cw@cVVSt8qx%+&1x-8$o>)P4qQRPD%h!h7l_?=MdjEDHQ$+#)nF3v-%zS6kyZ2h z1F=9*gN#M{t|%)zA$!#soy*^wOq_M-gX;;Z`-GsJtJ`~$Z9B&c_1+}3w)bR_)fHIx z3aE5e_KxxNLEJ2e|DC>Q!1CSMl_R}eSUwdI`0B{to!-1#*Xc#VqS!osb~QA7{0zzu zkN{K5He^DnP>AICfxFv+6ggw#=8; zn$h^9TN9#<#{wazk*V|1Q}meKCTi0nwkB$ler{0{??QvXMPZ70%0j!B5yoNxH8i*u zw$a=SnRe#kKkzMFATli0%K-lac%`Fmit-IrH0L@{6c>~tNm8ZNr2k^2y+?m+^$q(L zhU{0UN#|jnJ^TQV1I3|<>lHLN3TI0ZUMlpXpoB}3nAaE^Qs9AuaFSpoLS!JG#0sfN z3Spf38`yDqRhlp8+0FcI;a+N2xRz9%;;G548Pjm_fouo@!R~)@=?nU~rE~^O0}o%j zuK^0d^u@(JqYu_$Gowj-Wm4LlupMf?SPXT7($zNuaFz*mniB$iak!H31y(wBsDBsO zIh+cR9E0$~6O)?I;L=0Wd6xgnRn1+q3k5Z--1+b{dvfSqWqD)0=IIv&T>8~r>Dzmy zb+1ReuB{iljO{E0Sr!@Jeq%@Ei^zsEzC<+KNE2H~zJsq&#(!~ycpA^ipg@M&}7ThfV`1*4T zc!NDHTv6f_B4I7S*IPBIqZ9dr8brl(^3aI_n#fEb7$DnLY#9O{iF1)6P}EI_wqUCb zd_V+RWmHLh)kQB|EeeDrr!1JfE^(=zY?V$cy9n5-OuZM&E^}!W&8AQqUXpCub&2H- zD3S`u$|&k|`}p5r#S6k;E!+#jA4}G|n#run z(ssngxU(Uc^uWq=#Y#c?#7{##CO$pFxg-w{%qRKF6|eBEB#mcOHW~ck4Lt1~z5(~C z$X!RUH&IiTgGjXpdx>t0#YK$_SEl$*W$3Q7ox6DM_row|8+77z1p8O((}2|E2sTB9 zs4l4?5p41`r$a##u+#5J08Riy)P>(!%+kHGUld6xibs+{cW(=&tI&0GKeP_6Q3Ucc z*M`G{qKS(%sQQoi)ul8EsB#lnAqn#U_(ZxjR;khwRqI-k39Tate6Au)srV_7W5LG- z1HUv3KG=xoiwmM^Z+P^MDecx20-_^mjFh0A31IEt-R9|U4ciVPQSGb-R{fEv+Obc7 z^ESZ{bXW!$rVb*#4Ou#_19+h^X@l2MAS8=YFnCB)P5=}+>kdW1y`ZxX81RnsTKEJ>1g}`@=yC;KQ=5uhM{_EW@kM3_ z;v(zWsInsO2NFfB|trQ(hEosey4-KQ;lW63pZ?0D6bQ3-twvf`?+oD za0{|n{@PFtIuQl0Q-=p6gud1-5;^n4F9Ap5$Hggy_}noXNy=PBE=nZ8`KC4gC}6)PaE7PTfEjFvgpCrJ3_nUnhkQ^CiMvoS`dbl=j|e08qMS2 z#J{0fCn}sw26wbixYqVdGM#E7F`?Z_yo{yFbm60m;pO@v^)G(s27?#X~CuH?P!5`iT_;rKfU-SPYwec z3sh*L`knZ20w10vFi4bKcwX^pW+FdBwPQg319$arg#Q|~Z6MCu{Gm3F07d3`#O1d9 z)0Otii_hStri>#&B+dywz>UYc!*)POVQNA4o1=mfs&=bFxHw(oV{(c7nCHEGbhcU! z8U%b%6bKNT#v8C^XTch>ts1P5A;+w3D#WB#*cdnRz(gnFO1k6F~W1sQmPdL^l zcaI}hgyD8&PAA0Eiw2eOgH#|X$5UaRaD89B@+q)dVeqE;ZFgEd6$mTW&VcdiM?_9Q zB*ZGe0k0x$BrI!!;&V98U;QCn$bXp(*!CvekQb@J5>Tw}EE3Wg8ef6>Z*RR(0&4ms zeh9eI$EjnqQSU-8{Vu@@9XdxvXaLLMyW?s)AQWK15|+;`692k@rH~y*hLwbfp3WFQ znZ?j?;vt##|G}L*85CRLL4dFSS(FNiN;r}KYP7V>r@`z~JO{`ZcspF4si8Odunw&P zOgfCnd!bkmEgq`Wpl#7?vs?_pPOc$2p<__&`Jx_Y6wMaQn#~uw5%rgfVNk^1ygPs^ z;xfjwb;9n3PcFdaKi~3}RX(w>eU)Ary+Cjt826>qVu@Ex3GkXz!?#Oe*4BY%R=hw` z+C4$e5(6gFg*EQZU9Pbmko_ZOXy#BRare9g98dzf&e;7DYw-*lZo%jBc5jE%_5zT! z!m$PD1T(9<7c0_{Iw{%SY-LSg3D_d6;>e5nAn@EkOc=a*J2`uv;pvw-NFT-3aQx*K zx{=uvn@N8u;>+ z0WCK+ck5n{{gwk&lA=gM+i9#-*m5KpIxxJ31*yhBxWzSK zV@xp}+2D0R(wb(b<&FB;6`|&O$tk*QMt3#v7qjD)=Kt1b{^jkZ#Z;$vE6`oyqTP(l zQ>4p98~P)Jt#V|ba^7sZdEh=XP!ldnzqhn|wh9Bm5o!QRMMxC^3W{r86@XE0W;(Lc zg+j?sM8pndzizUt`>+iow)soIoQ%_G{j}$tqtcOd^Ln5+CL8j(q^hXR1oV&3irz)) zzic(ov->k&6OlBx?CRnl5-Tc-GoYJN{&}_S;yh3b)DQomVC%O$0?#Sys_BAJtBLQ$ zqDnvM=r-s(+rYql7YOg1>99m63wj^u)I(#r>=Ef)r8~5|GYy!;vkC_t>mY(byR5&{ zg=0|M3*d4&wxjfzrBkx5`|6k56E^!O{Nrq2Y#BBUzIj5^BTcOMHry-HBc>5iVh;!F@-a|~gX5>?lvz06)5 zeb%}TyrJsza-2g5pSZBav}8V@ymrbnKu`T&g3C{}Z{d-_OWv`Ficy_4`K0a5!QiQ9jC=4lSJps#INY-$1>4*Pes zLrkR3_nRy9jZrj}LWX$4hS1v??QQ64Scs{k7mU#;ghB?JJ#PDLeP-xc;K<*sV4X$V zm8s+B1pEYto~@QOH`}H!+#|v(=72Y7pjy+7P8#b00hwNEhlwqmx|#N@AAA=GGv|yG z6Ib8x&kaQBV`&b$m4?5Ei0;Hv=aPUZS z@2&u|o15zvpi$OW+IJdcm`I7f;)rOl7z$TqSz|h!xG1>pnc~Y0G z;|&68{&Lx&Y5}`Gk-BgANDVje1(~F<0#<<5%ZArYCDyE&qMwTRU zsiz%8+GD_4AQqGLHRNHYE)_~-?HJ_vIx`lSpoG?-u{SlstYRSr!=KN(J$%e;+Zn2fQ zDJe=By2L{GkLwfbzs~e5X%~F zbzq-JK@^p2{{8G&a>{=7`{@=Dp7q~e@CgV9ima5#G+ni~YfNSQ>lerL498)j(=i2~ zx7jI;nO2B`OtS>90D`L2LSy<8a!9jcP$x(7SjKJ9@D@7cxNS1A2}PUMJCXuIM_v`6 z-NdwW4l3o!o^on`I0kZuYKU`_zu zqIH(dzc&_$^%G9ZTmL;&f2HnNx8*%$%+wW$X;;fo$rFJr1Mr{VfxXi=1?Mp>s{UKZ6@gWZ9cT8_ESem7hqJr+LvBw;8F zEe|dRG36;QSW}XA4`(Ou8e4l&oxQ&H)3M;L7Q}pRW-6YYASqbpRkk)^8%?J5#6@G@ z#>I2HwYveENP02wQTMIVYUe#EQJ&P49b5-S+0@&K3qItP?Jhg`DcO1`der9(Acdad z@l1JGZNZdnHz%Xzag&1eMUtgx+J?DdK8Qi$C=nq`ghWLKVjyPLZzHBgYC>#2{5da@`e`&K5J zyHlvX5*V+8+_Go+6Zerbo$EiH*`4UYzEbkI8kJV({Iab^lT>>8~Po!%gAO zxA|}>*zLc6PF?s&$!Sl#C8*%p+w|L|*B`c-{!3w@c?Kvz0*uGD(2NF@p9gA^(#7cp zpi$#1WcvK_UO|Pc(QeX2g<0jYsR}Wk7CjMJO0YBux1HvlAz{Gs5g967w8Z3?JJJsu z5}1_7IMM-0B#o;A4gg-Qi6V4#u#~rp1W99f-uo5iK7(LH;QyJ=X{paWKqinW9M#TP zjEqpn$h-1Putje!^w)i<0WH>Snr~u0YflY#m#w~ z482(siI6PTF#^^|p*h!R<&A+P)5wUm14A~2AH1G*1wUoGmgukq6`wjR#%>hu{w2M^ zk?M!ZLTH?EIzDUt>0}OlT`{Z~goZc&!<&cYAa{^N=$pkM&n`*r z;eJz5<9Dd{<8mqN`ThqWnhVim(nqbb>;^C>8wx=B)#S@9(SMp|u$o7(w;$2C zHV1c*h9Ivtw)v5Htf@Le>wJiT4LhTXlw|zek~~ITggJ`1^tx{;@I=wc$Lp#(A*B97 z4^*B_NYT#6lR9qI2FI|Ah(?OCe6SDH@Dy5vLgACg=1SQvVZ(hH9Fw@-1Ca4CgN#S4 zV#I($H7<{f6>dL*O&PA*Iyj*atD+2*-`ONwplGXOn=@9tl#^KWvy)05;>ND+GR4YtF&K9H4x{{0l&sj@|%P%}fP}^Xe1;^vw2#GdxmGSE_IYBlkNp zpaZH?fF}VYZD1^fD+-vT^wRgkzFvZeL(kJp*n$0k$79=yQHyou6bVaq8j4ztnR(*h6 z_l3D}cpsg<$ajY6u8FY)t`&fnT)L9~<-L<=QMammdJ`~(z|F%y)&ZLa=g~yC9-cp3 zpHl+G6=UJMZ29lkXZi89#T<|@%|-*yDOfLK20Bz&2h+u1Wg$pKoj4q@!EQi*Y+oPT z+MU$HuuZZ#TdFU^*8{{vEQmEE$`Ko27{UTZwV2$C@brA{b8Pa}<~>UW=$WY(MaTgI zAq$z)9yz}cw|U*8Aipu=&t9w~eM(t^Lw1#6PM{t7ejoZ6!{fbW4#24#`JuB0Mfs9( zjNt?*!ao6a`Y$>Fm8TF$+-x4U_*}GO5K_K3&C9|fE4f1Ax9c?}_zg!us3RSy*SS?# zVJg;2Dz@;)$$_9LYUU;w&3#QKWF|92Je5M?XP?Z$Z*cI*o*{6BFL)q(KG<$4Z?5tR zBJ*u^Z{g`_lHqu{?65^=%0ah~>W?O-(J3SFgWaQVhL$a3 z0{d(uP=5^Ial&x&7~bAQA6I*}`w>AP2$GHNCft?D!;*#GPgzaq^FOX%o&=+Y2_lrm zy8(${{)b083)cFdxOojj;(g_rqy>DJ{1Dro8FD!$e;k^uk#OGOuGDfF0Z)CI;N$!4 zg%jmQyPodHE5DoyhuqlUbCfF4&*E*~wr+9Td)c6_LD zkJjlHVMWznEYuc)ZgD1BR*D|F_^BJp(|0 z1?t{xmIoYcEQIkMj;Q$^Q&9X`H6VSsNN(6Um%rWr)~pBcZ|z-VOLiPg($fhZZ}EkV z=u7afYY_==x4V1vO-!%rFYwp(gP+I3pFF*)w(Oo3#eD%!-0d2{^7#&q9M?Al1x|nI zU~uGYgx?U9K>g~);KvE;H}&6T+~h+csiiev&KP;E!wl{TcbQB4fMW562hh)jA9 z^{A~~h& zc80!t0(~=pR%QJuiETj2cS8kKCqyGYGYPD3N?#75ops)1t?%b3futv4pLz5$T2r?9 zV*M%P)gD$L8fj%Q!PGpDx<7e|dpfM&^5++ph*c3O`eyf0p@^F^- zK^J{tG<2nMxkCqH&}b7mYU!jyz=-Rx0O4C%i-`NF$}1e-EKbyAv_BMz8A;Ib>-$MNQVxxbR-f6v3WFKL0(;m$qD*0!j; zx_Z{FNh~^-a8VlO@P|Sb*H~Y*rh?%=Sphr!HnHS=Azm~aztz#gvNE^F<3wbLjyRxK zG?oW9OKKY*o;0vgLQjN*T_+)mH&Vu?0{iwS=TAK|>n`8eJNistM4>~0SVx}k@(-uw=#GlP zo%>{(B585D{`93hYW&=l8^>E$cVTj0VA9%UM-UHCLJ*vRjM;p!D5zg4u77=!XVH+0 zG!KeE534PQbJf3(VB(^7tF^sr2F6u#8v9i>WAZg5`7UllykAdB84|nT%0-Y?^);BXz^;2D0YkebJw-#nIv4d_z%&+4A4 z>lK5+tOp#d2H>#ptW&+pB(y#|a?kg>2T-*I)~0uEd(5g!)H1-}O8DX)#AgS~tI_kr zmTk0=Nz{?u9O<)2|(%zbJ3 zRjLl*E%oWA(sknkwOa~w$osCzdb-VOh5HA(iW>EgkzcHQ#h?D9ar=t;$0%dE$*Yw5 z?a};`(|Hs=*{xfffwSuzS?)d9E`-{$jek3QQQCX*@%AS|ve)BJmb%zcz!54N5f44K z)r3;_*lX=treEk;y`g>bdmhQi9(zmCd^KC0aU*xjYLo|?rh$b}I-$+GyxccT@m_E7 z?p}@S97X+fYVR*?ovP4Y|Nr_E-iE(m43T+DN(_Dt_Ios6=Ey$^4%f~Bf*^8HU9bY- ziwdug%WSTxsf8<%$SjxlH4!7pi?pL74H9z#7Oq#bl=xh{e#T?nuTP&(IFh7z+e!?r#YkNL1GUGUS)v*%!3Bo-=O6=stNfvnBr)1@F^d0Jd=`8wxGRr zRXPH;0HpR*DL79uoBHl{l$Q!P-sQ)ZA$~ZxSi&fQh^62qn*Qz=;8}w9z z#2B*>O>p&1rvoqERTkZ(;^R$Phbrvgwf9PCWIh9+&zWwPo8aJ?+y|K3Lmt!oB&0hUXI|i;6 z)be+^`Q?(oo$2G$?m9e>+^U-P{GCan6q=Ub$p(CoKH&B|RxX3m1lNDD~c zN!mH1_h6RL*M(xpc3sF-N4>XND>DoVzU$al&-8D@YP(cBN&l9|@}kAp zOIckgo11*bTjch$kW`#fLlHbC6H*ZIJ37ZkOQbJm3BOJ;|HdReI`CL!YxRyZaXcMe{mGnqAl z6Y*0>UdO9uGhPPNO04=#{8PjYyod^41&`1TRcjFZzss*0TCXDMoA!E$i4V6^fn)=8 za=l>thb3wL5XQEKo#yTO){S_ep**3SXJ!FE3;5cTtHU@(w}@qT^{1>%AU2ly_8dP} zLMy~ykw@czBXukgO&|4?9QX-`X?z5~i3rO@6vmS4ZHNRwA`MUz^$x&RR{$nEJ{2U{ ztA62+xGz3ImCL1_Dy(~An1Tr`3(Iv=$MRs;p^#Ek%FO$^$9h@8dUXYRKI2ck9bN%M zp$9^~hTWM2ssaCYm?-r*cA9Lc03Oq}!M{s*u84m(ltGp+Lj~HLuyV5DtytFfXt1;> z*pAe9Nj*%|LndVn!%v=AR=z?Nh0G$A6zDFILY0$LW~V#u%k*Baw?z7f+(l)jd?Nw> zKq@bvF^MsIhNye#bUAtP^m7%G zg^JmGqQMIvM;Ke>i*V}2>WV9B>IJy z!AJ0jOnmIj4Gs!Q^Cbt&f&C$gSFZCIWL<5JNFVuK%cBY4(Lp`RTnAk1xPa? z@ygqWTdYkiBvEhyuRK;1pg+|L9pSbRGRL8aAoli*lR<$Z1|lsiM4g?sSoXns2Ntoy z1AX|>T3mmLtXXXtxHqCL8OE3OFAs&H%_sA#3mEqluI+HDME534~9B z27iY|{PMdFq`dau=-P_;g}cXIKSaA9ZGt=Q`Y(VFw81lA7$mM|`H@}Sb?KHyWV19h#802hXzv3(myPoKMJz)!uOYxx zL7hP3T;?i#zYRTNbX@~0T)~6Sp^8-S5?i8xl8}L1Yqq$|3K7*hHu$ONsb?2Dpr$A{ z5&7$U>>3kb!r6D?&L}mtBom2kO$1CxnDsO~3ci^B-J;5f8d-&OC^Z10IAZ`BaI6wQQ&G2YY4Y9gtzh!O1G1X}CES3d+WjKJ4fXpjlB~DU9 z3Itb}7B;#GxT5ovOia~~Ajx@Vv>}lI)_We4#!v>Rv#XnXx_D{YfHWn}p*Cts?19r$ z`B4Gd8w(tNA|`3?xvGk2u?juhu#h_E`*TlBI3Ugr+!|m=Puq;Pmqg<+zAOrWYZB_3 zj!Wx132Dti=fS9Z%f&X_4QmCco8ps$<*9x%p<4OK^xw*%3rfExV9!aU$A`y~-4%`@yPLJJcTuK{ySkuh&0 zIvp5M8f->w zyIu)KyYr!|CpxGK)khsm>temNDQvv<|BUL>kMt^4n1KDYt^r@ER4)=u&qw0J`W;*f zS{$jlTe1pkAkiyjR2lfC#TczEuMJ&-wW>W6sS5y)Or(;k_Gws$)q%{6ZAmg-!zmm0 zW1M^46Z{qjAlJnLSw%`SZb+;x6=O_mwq67zy_uAwS%sYhDK!s@tpV8#TojO?w^)(KFC_ z+?_Tf*$#h=PB1X+52T?*>Pl-Ua#gdxyYDp-s&$OZoW2$HT(@J*c3Nb%1|9D!F56!L zn}-DPDi`de+VQXAqj7CxOdrs~1L1yf+_*6nQ>xO~1L?u?abKMCnW)m;11b5hFOEK6 zCiM0cQb~eJ&3JwJixR-#>kHhL~E%@d-s z>flKqOP1=*$2EDx9e257K2B$onYoFm=dqXIRd>6`$&)90k9Dv4E$X&EC?14aYnKs^ z=NRP@M|@LMg;$1y2mZ@Du1z=6VAB0VuN*OlRwkctjh?d+Hg0Bl{>WgYcDtR(1fY>Gio7FkM<4EiWoVpMG4@tw9OpUO34&R_X8Z$Id5Bn zVJo;(HM1SDGp)pSH?}yu&X&aZT7|i*thsUpD2q|oKeW_V6ZZNoR<;+2FIs4KxU2eR zX(crA?Y8QVRBs1H1V|zLJ+B4KUV#!zPRvF@MUuQ6VN^RmkzMfn4OX}cwnaF{{cI}? z#s$dbwme@;T@6rU^%_b9r*5OgtDgIqfP@Bttl-yF)baLddb35&TL6+lp*5lzL6bFE z#BU!nNq|&esE>B1B4Jk{{$n?s_Ru{|DKUoEE3wRrc>Q(y8|H9Re(Nr-#ZQG#PY>@U zG&}shdp9)vWNKj1Th}@K;XCCc)q5!I@Av3WjKeTBWlz4mHz*eE*+xis1LGSeooSxN zR5V>HjqpD$O6XDFeoVgQq=izD&s>=y>W!-?=lK536Zt9Q4D<2#?fb ztMBhnzsm(7}h zBFj;@DUydz-rkRRYAbGb1Rm^dhYhgW)VFJAYd5EvS70d3dbC9M>Ai)4^fLmEQusY7 zMVt`Qhx~aQ9|<&(_y8AJ$pc{fn+S+$hK#<^m!3loUm3ODm{p=MkHc3+P2(3P0WPQt z;~mlyolc~1M49?b<8wFX%T|k2C4xgc$xO;ng&hPd`?f*?6LTgIlzZ$UPq=}}bGuZy z6A1v>LD;8rTWzjUI8*@Zw7oxibc3V*U zjsSu=3ku^VJ9mum;WVknx4^n+bu$ESSlk_jUJ0)UIw)-FCgFOXr^LdnL}h*IS1Ho< z^qZVnmPpwY)9d6^L;GmlZA^-K(<;BXDq{;y%Ny-*`X?F}6UeYJ$tD}F&!n6TulM!< zG<568_NR?gsKcMD`4K=qgOW)=eAk^n;VD!F%3r{*FVbo7Q&YFp--L;d?F*^`0N9d4 zpo=SqVIm(67T~zAF^q4j5u8_v4|#Zt2`ZuJ8{Gvo-(48LTyS(3o$}cA-ZuX#M9yqgrm&gz6|tQlit>%>M51xawHnO3^+%Dh4vyhQBhCjC{fi2u_Ba zqesvhwn_JjI_C_qUqAOjV6ztdIAO4uf{T59gDI>=9+>5Yuv}b(A|#G|G>wG&0O%P4 zRtj_ptr)PNqp_$IDacASDhF|yDjb@CPhZ%z$UjjvONFCJT(;vXoB}{E+`-V6nuo|& z%)!+um&0n+L0CL7Q)W2 z3SgE>{0FnTOXX!>+JX0PBs}Y#_yaYmCX_~hA#>4qwI%F-=^*G2M5olyyJ7=71%K#N zA0%59J?|Y*v^pN4(mXT^2Mq*7menk4Vba`*+J<%Rp*Ejew@v9d{Y`@7@Bbpdm9_G)h@7m?p%r7?!yT;XP1|wZg^({byF{DeDAR?4$k>b@yI;WWLMU1bP152(A0dKpvV^&Mxm^h9SNYwz3(kykhbMOX? zvu|LZhc6sdp+bEpeB6B8JM{E}G4kGc49`+f$#td?Qof^|bsn#JXngv#F_O@$dUkUx z1xPilu{8oq6iT?JtuU%lj_k7MyI@(c@j~`_RJ}s4=>#3Dqqx|4bUFgn(0LozA(kYH zhuf%HQfq_c)ogpux?Y5YA4UWPYkp>j>!JCxxzeGh7Ywr)H3s#=V8oRSa@cFZ`xdG- z*{;DWH!x0zv_z~LnZ%(w0B^`nPA#}G3dD8W);Q8nVP&)m5eMZ@q|O;|v9h-g1N~$e zUD3pt?SvsWl`qgYPim~ITZu+WSl6F(;V{DE7k{>c(y{58PpD~w#LhfEHyuV;m5~9f z<*jy@y@_qIK6h^bzka_|w#FZP-;ZUWa2@yNSsR1=hyVBS_1-*swWgQj{?0&6bD8>_ zeFF-UE&BsFb~)-ao;yS*lW5V+mK2mp{exCAcB=}$$SO?(yh(6ab5PE&&}AJ`+|Wyj zn24>5e!=MYWx!0%?Ba#Odi*9Gl)Nj0hJS^hq(j379DcR0thIgv5{3R3v+UGb08vWe zNcVObjwpK|`LiI2z_URb)bF2NKZH85;C1OM4>>{cW7khFcZgvM&0G|X1CZV1Qntz> zK?{6UTTll3HK1i#y5e_Wsh0uITCl7%ty!;3i6R~+^Jq1W;mo1A(v~RN(bAn5biGdI zEF25p=v$kp8;%4WVlOIfV2Y?wVGW6u@rA>DygUlXm6mq5!3fdA1^yMm(?Fwe_+%K8 zTfK1Z%YZIMsY+*0OQB^?Sc?J=5bn*h_c2ag#N~>1MYtm1wyDdn>%N<>7(eVU7-xV$ z$u-vqb^M*dY62RoP@g`t@;ak?-hRISQ-i0IrE~kA-qru3*#Y3+>RdM$Vsi$6q3?!| zmjuIT>qUVAU!9d69{?G*a!E{Ry+L*4p1Bn|Zn}UV5=TZkw{_hY66Y_>1T%DTv64A_ zx&NarbI!Qs9si^5iBvn4q@U3p*`nB`frlmcg<go>Mbh2z143idRx7*=#`QGE+4o#Q$N}fmbJ#dY5mUFCgPs2+U{q*&?p3H zgZ=f>zkY6xP?OV==BDHMSz-V7wa`QqQU-$U^9dh1fI};_X3j%<@Y?LzH{Q89K1a!z z-oz6Q{>p;nyFEnW_^Eti1Q@VDW@H#A=&~bocoXa-`-e%@T;&AvJ&1}rkO4vgVyz#A z^WXBxnw($X{lc*GS9f;~5|F5u*oLt0#mBZ!UnxNDdWn;)8ar2 zQ|i(ze6#nYYfqfw95UImvi8`^mE}pIWGXdMwFBUrSJ9W)nmXb2NqPzlof^nU}c zp|dad2B#Lu(EGAUL3{GNRO!R-$J=Ym1$Zyp_WeBm>4`w_&O3B(dnIh*gjIxXjUmZK zve$r9Aw_{TJ^1pV9TIL=lm4<3|&)h*Y^{S6J=ExURG+9%Kh2Yg1WBlexIYp66hSM}2tloT?goX2DEShb7i@ zK~btwqx&7o?2vF<9%U;u+l#cy73?3ZS4R~b@2Ydw=iIv+jHN6q?5XyAB7kXi1*erH zcYRrNUBZctVz)xo7HrF4)&hAdBGU@t{6g0R)vUxbQfk%1^;p(z(7{&ETuAVmO&x*( zg8Medbzo_Fdvk+0=8*@u>mL|n*B)4-avP($>TPiZ)D3z zK&u)9y<<^r9OBSat_Co%L~FYVPz{r8SG289-2-5(uAzCmhflN0MnmUv3h;1OXnVD6 z2)`8&SiKXJ(RJ%Kdl^`kCy6@iySkj;Fckr56B%k#Z40Al;zc9JwNbhR3_gKeM2t1x z2XBR~*5N9*T@1vLD4COp7(jqXcJlXS${{m2it!0OT4G)r_|PF>KF<{7>l{6Vb|Q8e z5hGJ<7oTMj%cRAAz@tZr@?WqtdWVkw{<0O<&aHH@bYG?vqe7G$Gw4G{0hDi7#XIJ6 z=n#xKqTSO}me?-c=fG5LH-%qrsMiperRsXo`h1Qa-8bSI6Dt|sVEs3Y-Q@ZU(uVj& zIby6MlwpII-)4#-gDD%2c&we;a||5of881y>R-YPXy%jv_nfg87NP`iziXvw3ij914r(fq}J(vAq!*fFs zwW^d+QlDO&>0QL18pq$B`({o=t-^@KyL|s_efpVRliM--B$#e6qE?wuy2U^0nrN4) z0IJ<8Mew}vmJX1nvPs>13tC%mT)5$KUH8S+6|fs^5z=zxHEP7e4ZudMV?+bx0zhOQ zDfM+J;2<5-w+N)jTa+ra9mi{8N*60pTh96oswxrrSjIV8p|fe!EbNNyg0mP#)vF0a zT&-_~WI+mDwj-DfmzpZpORI|*){k&6QB>?unVgqu0xH|rmOv&|aEXxAFJ-2r5wh<* zGysP`yT_-tf{Na^im2_;%VGv?$ltHFEj&3Y&dctg5&M2RvIv#dM1Mwt<87sUPswSJBcuRg;xyp7I;|+{CL&3ZFP~ zj!Jd)s8W<(vg3C0_Oxf==%)!E-6qrL4}1S5Qr;fk#L|?e<-?kL~2;@$0~Y zc|6)fu%&nM#y#OsRehqEKJt2@$47jI5EvP70BBmp#}4>OPa*FMC3YT2uI8|xApk|e zTwjI8G?Axff;eVhwbCJ$)`F^vrH@gJs^JEvj@zMW&1!g*+N=$ShzejILQ?jk1wP;h z(KK#j10NU5$w_P$RLBYDE4Ka-YtvJ8>iW9zUxVfU>7guAZhp#g7Vtr)zuEisxTl>b zh3OTyD3GY)!Xqy=P6dlgw_Gf6ASF413FVQ}qzardbo4UPcR9~DPJoGa``ed8MQ zwCr|YpEr+Gl`0zQU%B;;^)v1leGy=x3rVF)GZjXJJDfO>%CU-X!PFH0^asaW)?Uc& zP<%+i%ggi8&595Aw1X z9}bod6y+);E+fkHg_Op;M1{z$)rzrT45^-smJxEN0O5hfR{+l71P1%te-ff4LEYY? z=~e4eUXOl%TH@RP60FA8Enx-Eed(3FleA6AXs>&Qk_tcLPh>rbJ-w=KX>(vY*OiYY z6a>b6QEg3)k4_;;Dc!MdhB}so;3|Jq&!t4S3x0NH^bOc(IW!gnX~Kox(=9!BfN<6w zXo`{t`0hF`5RMlV&^LVzy!~)YwZmJ|qIU{~_{npeazMmR&z`wZ$ONW`zK5&V3RlBL zIY&IB5*%(N*{)0y+s8&r-zLk3OgS-5_o4AXjdHSf_fmIJ?JKCF)yBM}T|H87?n-1L z-*E~(QloYz&+Ilm{{m*=uG;}YxOvU>n#iE~*t2q_APhgch$E0S7RjHZu$-{0=N{C` zMmSatjh*A=W4BY*0{!-*BaY31gnA&z^;VpfUvue+eFe%*IolvO0OYHC6g9sFxn4If zaqrlU0air06gpqfb=r3VJ2y@)?x-#ZuE{LFFUIQh(nm5StELgjDlb6NNTIyn+cFFS z1iw*7L2pEgG5)#|h&?>hNRh!uVY|@upD9(dF50HoQNavGUc4*sAgoG=Hxi`TLw!N1 z_&RS_lN69$()>zrF0J6qMGp7SHGf#E>Z)-W+UM<={yr{_-R&6bK_}6au3+G4m2X%~ ziKxjc_ z_-7PsyLyp0{4g_5MM=M*ADi!Oz(+t6|CR!d&NLUK3n z=DB+DbT#8CE-Y;vmVIY!#i{e}M-GN{1U-rI7nc1hy33xy17_yU{EV}^zu6S(_oKmH z$Uf4x)=z177lxV!nQWvmk-QCezrAysw;IB})mQ7FKYFye7w&O3F*-cfb6Ts-l1nw+8#!&L%2kD%ND zBtC~92^0Qcg9BC@h^#~299>Zj2lhQ!-iFn*Nmg$dRETw;EJs?~B=(FEk)pK$W~G6A zK!X3K<0T=vx3L4}UeY5(*d|aPFVOvzc!p(`x3W9~YMob-!93K#l>Q+<34_u{f zoD3wYlpcjByFD$Ofa!bH(&ErbvH3?tVN6GjMP@XzTYia%J~FsYAvZROCtI(4lY9`8B2leZCdh2U+4 zJ3QYJMbT1bfrN#tF|$@}eYqc1SSV_4dhBOvir`)^m-W8fe6fx8ADo&x_KJBdWznLZrxXi}R+h^`4P-8&S4gZ2D zBkxDcyEASQ>M+bq7eh^}jaFJYaO>EaLPn?k!2wRX#4f5+NSKEcm7P%9MAp7=QSv1q z(~I+`|48fgqOEF(-%yu~cJ7@cZ*N`wc(terPh&3EX2v(9pVp>c|FRwm0wz-;$z+N_ z>w4v%E*A)UD!E$Gy%;z~wrav$LtD0*5N&|cp`E~}p}TXCpj=z&#vh=yuGf#;X9l_e zmio27px+CDlfAPgy{RgiOMI;wkkb3;3~5pWf1H|+sC)MnV2I%J`C---f<$ZnkT#WN zMGBVU$YB!Bx8@3D^L$^t^#fhS^q?&&U8TuJN{LJ{*Wib3!*&Se!$(f15oNZbvPULP z2_4P$DleNAGfIs}b<|xa+N>s6p(uf`7Cz#Vw%#i&sSLt-v5{sP|b*JR!u51+$dTd;RP`4Ub4Xh9pKJnxUWg83YX_SJ+l!38go7E&T zRTCbJ_^%`+9RUSCr)r z7tP7Go??&Uec>@ZxrPUc zU?Vp(LsAs6lR%{rn)DnUOSafurVZiXD+L~F>@{!4a7iT*N z4g<4M0}B8ecwOG@=@4_SF_;(2ZA=?;`-Uafo}}v}o(n>TU3F3qD5(odRFoqW>(b0I zAu~#fx*+mN`1qDd_=+WsgpYe2|5I#pK?>-)4%bLW6t+Ud!pIa(-$Z)0*H1CdMNoON zyakqN(qzR{aAFZ}bo~_LT=>O1P6>S$b5gsx;OFBDP~J+=HeZCT$2J!f65LuJHnzGs zC)9EAZ>tiQ!&@AzMsRvwDokIW7nMm>kQP@uf>>oy9-aM#N*SCvqPS`y140M*X|pR< zrq~sGJj8JddTs!PMb`zjcnF3+aJIg4C~Yqsg9o4@jG+r$m^B!GoK^ zX@ugXVwX)hnm!n04GBp11fOUvz}9m)U``;b{P&S(<_rn%<;Jey@T{*{IOF3owH znG}g4s~GbUj^#1-X-~mvb>VOX?$^63Xv}8$BgD9JrZi}&xsE$aLatZAHQ87h&6@@K zkY&(~Mp1(0CowK@$d5E?_2EtKE?2%y7O5K;mB4+_R zGropsx%*Z)gGz=VFuXiahH+F>=bqKZ>f&URAny6!#9G=MN!`sttO!SxgCHv+7T|9g zJ0>hl&&t?oUlF}COCOaf87#do1gGhZ37vV%CO~#f+6KIro4;?)ytuilxsz z5hZW36-w-E%s>8V(f3RMA%`bjVS*o2F&}-CKArNU?F-ExVaPNXJI*8{(3J9CXgw1f zJlNy7_{H6fwf%S2q)W3iro?fCj$hKPftzx_39}op0jJ}ngn!_5r~Ofb^sp-|k0F?I zU0H614b^dQk>v7?Z5@U}yL1-;K4h4qgxF_&J`=LZjU1Oa7zbR(BOBW~WbK-k${BNe zz>Lg*9RlD}O8Sd@Es!NoolR3Y)<`M@w_t;07t3L35dKDi`O**Q=8Hitgx8S__1r5A zDP+pS_Ysg*t-(>llxcikXin~g(H!L;X#xDRR|*3)1Z3ndBnS?Fk+Akv1Sr$i7bIvP zEn~q--e&!p%zCb>@vJ1Zq;wo1R^R@u!9B>~y4^NSn zt=srjpcIjp z2Ipq%OsAl2MogQ1U4n?c7&l~$Ir2bEX?p+Kg}OG$rF-2 zw6w^nn-HG-V80itBk*cCWir=1 zxlsz}2SFZB+OE$EN$wif;aggGD+}3-!Y$R}uXX_*m$Dy{X6k*LB(c09o|1*Ww2)xP z_W~<#+T!A|QoMnft-rewR*Xlv;8|;}x^We1bR!g(cn}0|)LdDv@P#6j%r>(-kPm#jmI3+p6xMN?$=c#Q<$*NS8j=`>vFFHnI>_nR@JR0>lkrEs*fiVb zS2V+>yG`(!p+&3C!O9x3JT0Si;5Y&zls>Uv_f|&47q-&PJxWMbt}k_` zRnDFIU5tYjq9&wc%;lQ(Y$!gF|oZ77!zoH??2hBq~GwqmR2W|6)l@6*1% zE{qHPQPVRw7f%hK1-15g?rxobqZE%%08V^xInN2RbOT6UziTrfp2G90`BZmfW5$gF{irZATdWrD+s46WTSnKiSX<|PsZ_G+yGPe z>bGKoHFti7PQ=bLg2iXUeTf+^JJ5H2p7allb>GQc@(y0zmkfY0-lzXu`=p;I;|9Qr zo4~I#zeom5r_pU#mYD3*fXM;Ai{m^0U{_p=u${-N`0YHTO7vgh5f%(g<$~$m^8C+R zz-jU`vvEfNL9ZwP6&-_f7u-nY0mF?}I2{BRY!&6K>D%?uU#{?6E{BP{asT=k%LRCd zNrb9xk#bHhMa0C|c9@~=Tm@AAk^hX6qb+?zg+Rb*4Po*FBl9a3maZt>6|kZ-FQt<9 z_pM=QDt%c|M_qGPLpXdRlYEL(ssY^*_5M66LS5|l;4H?B&;eNNImT*(A_@)Mfr zj)b$(+6s2n_WI2R`}MkpA;EAIPL&iqG{c@x2TPI@?-Yu*G{9O3ca9FJFR|ULnctFE zBMq{Zvr;!GHt;%gE<%veBpNdQ9Pn7Y1Q89`(G`yVF3zBikvW!E>@I!vqp_va=RBT~RwYlryvRqHJQm4a zen^sIvkQEXUD+bOtS+>&Ak=BIvz5X|-wp!t%OMXouB`)I0QdA`^fdhUCutkwjAReF zHY6KnKqWjPz$K4-{jRNwNXVeaN(t00@K@`Ji9aSaX!%fb;_2ygZVJ|U{W?nZpRXy6 z*~SjR31f+VuHm#*yaozV4+UlbQ3;bNcqO~#OD^|%vsiN0H!6xFNK^RYH17^}LX+kZ zp2cpttd;a~<@7fjEHev9H^x@LtVbzZo1bH>qAUVz97htpADBq*#he^P5gc(rgbh^( z@Ldb6CLqDqCxfhmt3dg~_@Q~CBFIb@Br#SCkSHWF<^y@$I8YI-rA~3@@_&%cm4OH$ zvbUHQBFO~?!dh5}M(JyVJHP!0SxQ|QNqv}ObPJcshRNT#O@pp+P^3TZbb3-#OMa26z!67im<;tpZ!0*%7ITB_zjcqJzU>g?)$X20oT$-AmH)AO4aU2LXarSc({q-RaJY_Z|l?xu$>?>tZJ%EEm zpfoS9Qk~(96I1>@RkP&4qxDF0C&duXPE3QkW&==#PD|`g!5YiA?g6m$`1tN}82GEs zQce4Fx2VCzJ-L-JPBI-2}NwP~}6b`)#SdbTca`uc64e$-run z6YB-Z$%jg4nZyR!NgqX4N4p3k2LN(pDpqO8Z@A{936sxql`@Il3rF&zs_7^ZEDn1~ zwaD(M>kA~4n0Lq}ba4`BiCjocW7~5{sB>z6msrbUT*tCpWHoSLjk<}t*b3N-9s~8;$SS+aHU? z#uv}GGhdYU(m$MqGU0uc>DtIc3(Y;gP6t&zEmV9jT$0uR&h5_{m}5xl<#*Sj@FhUKxd1Wom_86#&ulD)mF4e-}O8lT11*5!l6)1B7}We(!{i%SzDG z+8Q>_y61USt1V>7#A@!^AHnu`0ZGAwDb5ecY!m&vA5E zVU54|V^NWj)hE7iPzC}PC$3Xd ztT^&+s-Z>*cED*=kBPHNRS|5?aO(y5DlCz-cG(-WAL`N#;sa*S2O8DW;qYR2+QMr` z?3c(w?dCaEsWVS#kw(wu&wsy>4(a|Ai3kD#pT{yr>uW&Jpw387TgrhMu00h&=&XtS zNSUib4gp<_QXClJ?Ny{+Dp+&I4#O%8=JZD4ig#KaF-))(~EL^b29*cYYqKe8p-V*zTHooc;A(iH&# zfvh&VBGffvOV>;r%MFf^oe1f~?VlY06I&NjX}dZ}*@^z;TwF(HE_Nq;H-m!$KFD)1 zlK-7RkGM>V^qDBbEJFL;{8WH0sE8u~E21dHqXmrKO@z%t<<(a0uqE!^#MVa<*jONO za}P>bRQ<%=%Yj_VeC9EV+H~}P+*VDnif9luDr@un)I|ILxGVJ1shYNGkC5SiS97t0 zh!jUY1d)sFM*#sF%mSoHQi_Tcm594-M*RT2L%GkOYh@!G#Hdigmoq{r<&<$RUj4eh zo9ov3{(}cHWbJd0P&Bqdg8v=k{DcU%bjL-R(eT5BNq8~1TFyr-bc z;Tr|y4gMsG1n|r?HOTfJeyJjBHhROmq65xJUuf|Fc4Rx1Dm4R#yy4P~5GLDX4sJK& zH6)1KUKwb}6>sn8KHVFhU;kK@)M%ev>%F%n%=?1ub5UF+PkU&ehC7a|N54wi$`LJ2 zIQh!kk2w6RV^;7K!Pq?+51<*Ma({y^j36O*yt!wL!0yyeXcYrKW1@n#B8dGpPb)OJYSM5oBFQ@)ZHE6y}|f9VGNBRY$cTX^!AFfyAsAlaghjF4{{DGiVFz zL4=11@IOfa+<*6Y251ppH;+4D$`XGC=0k9^y03J?dqh=hwFfRK+(fr0_y)4dR%jeJ z^Q?Rzq$LF*8reVtZju&;q>oUT&Ch^RmZ%?`1ry;etG;|HMU+9dh&l4)<#$x)?Hy`C zA>N98yK0`S?AFMKaB?ah>Pj{C?zwQqi?La;3z(`t0QU_?2%Ia+AWv42R=~_Ji@t(& zB@JKm>nI87(AZ-#(LKMO2MLa)>)Uy`VP@5osV*e&3G%i9T>2#W_u*5}deuWt#>3J@ zhQlI#(jyh|L^oe_bGx*X`T_Zo|HI9)Yjz+i`fU4LzO-~-Z~vdB!In{VtzQnxWQkal z1p?I@6ZjeJ=?SL8-%fsAMV%Sl{ulXu)IulNcq0j!D{yggAcYQ4rgBM{-xhI9J%?B; ztI9TX!whPypjuLN7B`RXAMyp0eOFazhVH9^5(Q^V+TJ-Eg z=6C7T28zkUT=<@y!|06Wp@`Y#$aG>^Z-)R)uKJB_CPNv0@PmgtVPYH8A1CkE9|D?^ z_%3Unp;0g5?$AV*<5T;`{tX~ZG!!huqTJDu4sb41sHdA$9-O^%~4QcvQXH!GmDAp^el6Q_h=`Sm;7 zg|MWKzt@+D@Rs$#{e?R{W@l~>z~#VjyW6%vw6~l;RdzmK8WAthd_IYc$Ha$vflA}K z$S$9mgu~xR-fw>|YsO>uH!sGV)Pq1zP~|QDF_Daa&y&aPQ`{a;pMUpD&^IT@M4NVp z`}r3zuv(w`eBu=FJB-2RMf+!tvxCk`pD0)kMIrc}0RQ7&JOf@^-fFG#54CtOlUa_Z z`M?45OTr7e$>c%l2p$~^{4v%tRlpxE6U9!_l}FZ8C;wwJzX4-uG8E4?fzDTf4AA)` z0Z((VCf78;NU_Zwf^M)LeLPKdH|RR@MvCu~XNYCq$C+^&hm9Md7=+K{Dz(S9iJehp z=w#)bQO~G(835LL@Fiiq2~?48!48M7(TYzW$uYS6@{2 z&-K$*I<`Q;<;aTZ_|LyzWA<-aPvoa+OjgZAUX6Dfpg>z8u@+fF|JJ4kBU}$QL9HxMn5&_(!k? zI^`JtB^uK-zGGg%Z5mW6<8c~Sapr43tpLa_0t=OArgdok0_{g*YD_IAW$eGm0VcytLM2ImVet#GCRol9|4vQV*P-$I7< zEs9iq*~UMLCc@lKbNhqJ4c*MN4ISq>b;=i64E7;-rd)ahvd$!IHv8^rgsCLMXG^Dp z%5>5h%na|}@aKRNpIogMB=-RxS^11f(i+6F~gOksyzue>?m zFkQXo4PcP2!x7)UnVjrqJOS)m%<~;Z{~7}ToGiD;G-nQ+MrD`6UZ^zH<2#p`8h+iq zWh}v~#c@VjlQq*yozX&ucmBoh_2G8o8W^WmIb|%_JYh+kIq#2SGp}hZA-F-l;GOd$ z_iaB^ekA*D&LVTaaVYJ&o}gqVH>LPv>^Ec7nL!x0GymrK@6)4<(u$0tu0d11LGhsH z_v6#KawP*$%dii@-RiW^4T%Qw{k#E75o2X*9eyT9mdY;C5Y&*96vS;|Fzj;JXi(V% zkJ5GA15&>0HA;$G`m59>h_u-NFHcr)td^Es4-&|oRrBN?4mi5pXSAx=GIw{G?8gvtd{u+vlIMS z#B`upOh?J3ts&ZHaHKLXMH2HySva_909r@{m=>xOn8OBHPc}&QpU0Ngy4tcbtbY4v zT=VYO5fB%&9n|B3+VT6R%F|Q$W-_tppogX~?B<(HF7f;kyk6N?dWjCx2UFB8`S&9% zt!!4qs$@ypvLafGGBjC(1vj*y609xZL=vAwjT`P71i3`S3=R^N8MqfvlY}&Mj&o_M zHYFC4lw3fw(V)z3s4QG?tb%+#(z`5UW_OTGTShn&;PT^)z)|cYL}NtFS>4&HtUr;) zF;&>Ar{v}9G&uwTl3WOg0V(nn_SfJN<*@C3L_=A$e@IA?N>mW#17dW%D!@M0++`2A zwhuX?pfuQJw62fz9)qKo+q)JQr@%u*shvkt6)qkn=Ut03X4{+OUH?!|AS|dGcRTkDTM-CQyvlHtntV~wwNl2mP<4-n4M^}0z@ zw@~691YKSo?eWzD$`w1xPt5j}-WRb<@6G0Mllqe- zGOPxbO!5a6ZR>^BqMWB-%BPzpqoVA}{W$JneU0R>yQnUpUyW4#58vMaZY{4v`=z0% zZHCnD#c{^I7H=&p!rOVSL5*4W{r3AAt6Y***NGkXrU53nKwE((nkCGDhjTKYE#KyE zAVed=CefbOH><7oy!AALrlB55vx~%zyJQ)vBv|&}q|A~ssKdMhO8T~2vI-TG4ZY@z z)GW;Sm;v%^VQ8_nzPf(q3)Ydyk$k0x*|oTJ>vP}@H9|xEA%8wa39kg!6s+m3E3ck> zHhl|)YM4#y{;;1{5oK|UNcSw24U4CKb0kazEjn9M*Nl5VMGm?&v`ml6;33`Sqy1ph zwY1xuxnc!umL^+0MweVPqbYCtU-8;qv|joM*W|HRNoyeH9kazJqFx$?zGyg0f7+b z87aIMRN!MX^KN!@mwek%0V`JgEE~d%l9*U}^<;H<6{?s)3*woiMGyRk&V%$0k&aR< z9q?e$2<*1yIR^2aQFV(YBhll$+k?W62}o-P^c73A$#T-+SG2j@#NCdTWwEk zUN0$LMo%~+-EB74z+7LA8dEQ@lIfpp)|2K1tG5Yg!H^<@q%M(BZXI&i(JrNVGLVi?VWvRh1MLj?Et7R z(C3=Aq0SQpOyt*=-53n*lJyFjNMo*(59bq(PyI?WKj4Vtg4 zc8iE`;jKk}N=CHkOD<{|QDu|Wn;5J})wc6rOgC)nUrKbqn%48vctK7kuVyVXCcIqZ zzeztObpWd5@zhHM3e$b^q%;d(!9pJNXiv_n>8Mp4P|I2=+zyV%lwsjjm6NPW zp^h!IEG2J-|J(7*OKp!E$=*2PKVFY|xw?YGOiBU`0(_Y&0Za-e?j98Njt%x%Gp0D+ z_s`i*Zqg6)mqszX8ExLt(t-w%ib&a|fmcOzftMs(@GB{$TObkbsOZvuT!%|KajcGzEzM?4e3bT*5nsn#rvzmq#K0+ABn|5ojxFKPRBG z(+2mbOUs_+8*u1HJ|AUQ{$LGQd>C}?DO(@n@$^yzooUod!4I3Ac#g@pp!m{RXAiHS z?R^9K|4VI`n(}vRWKUD5MO2YdYc~VeM{|b_b>$9030I)r4b4Te82}A|uUQ@dUqGP0 zvdIjMYuj_a8M;I=v94uBR#Np~ztiT_p>yP&z37?7;n~_(helylCqW|c`t*_2D(lGW z$+}G^izaY+$StDP>CvW>!EJ-$w4}0tl9d89{9f6Z$KsxVV@2~&Il315=S+Uks4UIj z9(CB}>`;>1fS_}@Q5}eksD}?{QwY5&60xUO6t7Xp$0yN6Tarn4O2_9`#Bh*GCB*;? zFLc2ss)lcVlJ7nOLX=iWUaxPTD{4!RwmJaEI(- zs!BJ!(#2Vf<5&57g;m1V8mDO-SScwHuMph=?cGy%C&D^(zA20~EZls5YHLQDvan1! zVC<-wKVl?EfTOIQ34AT4r%+tI;@LJFXhow$B81@lvoZvSbQ@wYJ6-u(D5+?gJM7W{ z7)GFn+>o{`7%Kj?1rvvjc*%bjJj%M3wO;do)4pL-n8MMSj4y?bCIbXOJpd=zwo`;a zk+HM4*7OIl;GSqT+e!%$DR5fNznWOCM1Dcxr7m8c! zhdJ2A6binj79~)A0|iD7)#g8kwra-*j87-{0ugEsZ_V?6^nqB4QewOj9j{>vwlN3P z=ieuL#xU&awZ|OL7HZ=luGIz&X3Z^>fXNIa4wQkR(E}oy=Q4)kN}B1e!)jnkUBTD% zX}&^GQZ)qvru-5Z+Kq=ZX&f0Rw?Z`wusk(iVleCM=cWVTyRs^XAv) z%>KzeL~!!r&6_>q?P&b+8g*ji&exOWgCGwin%?IEtaI3YbPnPy6|j3?JCA6D+CBZv zusRRva4vA&Sf%P?16r=r#vX|c5|aW0cHQ@&7h;oj6hoYX%V&Pb^_vgxzdQY{kHyXr zCtG|FMno+b;inJ~4DsDh(Al!GEA5kUfC=wM_J+_??esR~zag*U1bU1k--;L!Ey- zK4qs46%-4)1Jyw3+~RW8*X3I9|J>hIT|*%*|4=|7~oPB;w%|C^fj_>_0R2Xsv`{VK{%X4UJ$G$(QJLr z2Uog`vpQL(12ElBJ_=pkz;$WU7^NFT*liG$sQPIHHcCT=AEH`V?8b1mw}b88H@V~U zApgqmPAx1MXNGiSq+nF7M}F?VeXp>m5ytHRLJDm3sW&1@gD|Zo>+O_?SreM6HzAm5 zv-TB$@DIXRk`XQ&)||%293ww6**#g zxMqmrYQ|6XuD(6&Z=D2vjQxQoJZchgx29R}=grc_jlQRHv;S`WB1u1F^oZf$=;hB< z)|9&$k8ot*azWK`H>XlmI}d1%vgKYN9zgoP zC9~@&@}ruEbcFV;c*<_Yk60usdzv&EOang^sI8-2`wh~}I^8nLXzyWixmSPxGS?}` z(zot@{+YO{Il5)xRsq9s0^G?gu4K+O;<40e!f}CY|K>Pc;8^XMc)p%NDe!ciu)=51USS$kx71 zu);&!Gc4~T3}YI%pSF)SHz4nAaekQ|@xobPXJKqq?VrhnhDg;OZXfv-V1Bt18+tZ4 zngf)jU0SeA@lr>q$_!OSSJIzJwn{KB#Cq!|SdMwS3hc_w3AN#>2LT$iO7} zxFxb5+;sa5^!D3u^wI1G>~o;A6I1f_4}H1YUpJc&As=XXm@;v^z3x;TtnG+a`p2Pu%z#6o_po*Otl4}B+Xo62T9gJ z&9zjiCdT8A=lQc%l=aXpuv`yDk8 zV%)3$CG*Adv#UR`TWjdpMc>xWYoTH(2eMF`EK8a(4flyYAlUR%7sgVQ=Cxn;euYpV zkBCQ5%SVj#w_5pfXlxnD(IN*JL{DR8m|?lw&W-y$Y9wn}CkQk;^^-M(Ec<5-Z{^ z400ZaDXr*^~ZD66FIXlcNx1oaKl{F|DM_}jkI*$uB^~q%C-ZTWu z1zK5HgK2swa+^NF#13b6l-7PrNG5Jw<@P5A>Z2`Kxz5|<4EVq_EO~9|0)v_E#0_@^_-VbBx05zF973)eCrzK-rur-8-8fp6HEFp1TP*HT{iuv z6MC_1^iLz`B4J@BnV%a7tWMLQjDE!BilZ|MDOKKGSs2+lsv3>TQw8>cZV6u1&IGiczlXUS2EQe|VoxZ91qEfjHIpc~r{iFfX#s<^U(@_jS*L zV7jFZMAU3RpOb_Y77|m#b~6g#NlK)=nowj@ziN=+kD?$gPKt(+DhH1lq(rUtdY$zY zI>MSdm_VtP1_)qUn1vr*$=z!Y(`2%ZU%sK`F)rWGwMO1G*ajCHi-$0qZRfzyx>8L# zc2-$)F!hDQBHFoqbg5#nX4zhOL{)~|Q6jEhY7MIWPiD@XH%+vTGAtO`9bUAA50(@M z>;>oWgp~7V@jY4qplK<(GEX7%{>??P)N<2=G#{7AqysX+ID>I9p}kh5?K??Nf!JOO z%$5W2DLSK7dAKM<^%sDnr}_`w1!EKri=^bTRL=!huR~h`I>}q6%+R-I&F2WjR8xEW z!b%jwY0roPW>uCd2xBNXKDmN#-+x*^aV*=tQCZFgW)+5p$##@exzNK>gr(FJevZvY zP@PSVe7>K<`t@jTmJ>Igq>`~XXyS+gyUe-%ecr#pki`^`hB)*BDW-yC zbn+COpe9`p%V&(x=1Kny7|e+*_WGf(t>&?&te7h$N~{I}A^vJjn=j)`CCrRP_>`Op zMecRRv4dY1b*%HZwJqtRoKIK5LupH^4N}H*02Ly8q0P%NsYRzp>?n%GJvcU9ibc7i zz6b?+)_b9fhRg?nAC`_-j6l>F3-9?+qT5M;wNtnkJl0Ekvh=+^S~P>|~m zvvLm{ItaAAVA3GB8F242YYrY?ls=J56^ULqmyGaI*o+6Cq`#enDyT=(hQw7k$!drP zaF9!Z&eK3Mz){*$OY*!HBQbFpq;vvg>knWiG0rMzpZf1=H4+;nNsX=Nzg^@mJN@+t zmwzq49)D@R;wjU^WT5!NS4WVbkcyPSLShPwIjlwQTTQ1X?Xv8gPFYJW1ihVZA@o9P zWrHm^GND3Kw96!faxt_ceSy!6l)BcwFT%S4Aha>bPfv7yzT~vP zR`p`De#v!3Jbk}7Uw7zBoXhS7ms0_J2L$ocK&Ml|Ek5)uy-lC2Kt&|GQdk(-8mJEO zRqQrNRyciCk}X&eUPq(H-JoyM%WHsFu`&U-YDWVemAzk$(YUmkiuSo~RY0%^u!8Hh zqIy~A{G25fFBiK-B9Av}rE=Q3_L)Vk}(mswx4_BErGS-BL@EhEqGC`s6GM7$yA7+`~br?(@r8JQhs2Hh+E1tz2uo^ z$uHG*W|!tAg;m)|>LYGLPTR^AjA+|$u_3B34yGJTnivH20j55wzUP5bNF_S~@6HKN zRV2GvY58G@t2A&^;g>qm$RbXI^}YW2*)LU4<_FpgV8b2>#VTLTWVxxbiJnA_>Pl7X zfCX%u<1jn1TZx#S3Mj-u8X(;k3xT8el(#?%s7x_Xi^2WyK8*^1iQ ze4h9gveuGEU`oh#R3u{yjm>HyT`3>>$@rswAPsCRTE;-0X^UfQ<3G+BBq*#Tb>#iwY>ju?ZY5^CqNHF@BvI(6t$E#>e8B%LgxXV z{Kab|^9}|PG`2k-cZ+X0Z2AZIMVx`5c zk9Q!8#q-3fnc;Yyf@>Z$K}XT!kEHimOZQe1C|*-21yHaC)WHx%BnT-YG0c_)u+m?q9Bi9$mqXi<1$P;av;O$D<$g{;kR=nb-wNixz z+db~&pKoxVJ9C7q@{F-|O;^QQ@ZKJYc34pG+I4h5T%ZiQPS+B!6CmdFq2(za$OX)R zaZ|jP4%v^1$W^ImqnxH?ARKry`j*lUwP%oh_)b&f*K!=01SHUo1L%bw>fkV#NpG zq{6ZhcYaUM6K_ToLb!H#8$x;0<`o1BT(+^D%CVoKDLPCd802-qwn#K_?Q}Nz^~qZ+ zl3P2ZCA{*UUM5;rIu@6&vv<#vys-*G*HC28c!?h|T$-i48~Z*d=~?LsZ3Fw9iw~hW zw6fW7|6%0TeLZKoK}SzfEVRiXl^i?%CIPEApbNNx9+TT* z(s$uDhG`mu^YPqmp)g*!1e)L@zrsLwv5^}k4|(3(nh%fZPMIJ|G0&Byr-ux~iAxXE z9KHTu`&SG)>ZbZYN5rHbCPZ(4GI=8)cf7ra<6uXx=U}A;@X#_;UOu*(+ykiu)6?a& zM*l!_$=AbxGEt(ZT~X(kJC*g<4m(ZD@V61I?v2((I@P*QLy1lkujrY}Q#nN2jn(haCIyj)C zc!orlO!)VDGK!z#_9vajg`w4bOZn;sl$aCO)h6TK1cpx%UL2~VjV;hCqI=qlx+N98K) zLVIdgR8sf1sm6a!@>~ucA!U59)}WiB-#?;rFHAns4y|5&=+<3W&}59LC@G8?( zsxAMOafuTz(whh>Up#A_S!I9M!o!hoTs@l(K~qF7wB$CK_*gZxrkFMeQ~Z9xF;z5( ztXQHVrcEV0Y~tdoZ*gv@%WeXe-^WwCq@w^X!F1X=d%jG3Obv}GuJyuP7eXh?W_U?4 z1sRUGP=p3>mvLsy6I3f!yz5P|_N^M=>SgB|k9Ygb)T!Syj$QW?Am#09E!DBktW6(n z*XE16T3nRK^)C1)>@~TJU)j|fMsO8oiE z&xW-l#@*Z?mwO<-jFsj$Vc5t(5;v9q z{r2C__t6e%w>qRK5`l?gJFOdb&QM?-G_d0wOKM0 zLcomdOID1C$7~n!;qfbTwohjUy?t>I3%A1U|8%bj^hW!#^u|@F^BX_MbfG}p=vOKr z%%j)2+zc;(Pn1cB)U{#AQ00|G$_0_h0uBH@ZWY!Pm|{8UdoA2|e%`5vW%cE_OW}@J zt{^}!kAN!$SE`&8IQ#Ibi*%{g{I6`Ge_bvJhf&*Ld)kS6P3K(!3pCKzp4aUY=J_S) z!qzUHp@!DkncnlS`R5}gOk0DGeb`A+ve$&dEa`=@mPI?)K2oZ?x&27(vJQ#L#)d1f zqpGloHjJ5iPp}-TOWG}Lwx#T#WiDx_vaL-@vkGvzy>qqP#U6=VS0lNWEs8%Y5nt;O z#vsJjiI;5?T&d32qnzFicolEW!G4oKT=#~Mz&S0I9&De2v!RFDRa%{2M2~=_Bu_20PM>~n{QzAqz;#SK=q)Pm8g{$~tPC=ogv}Khz_Zv;3%5#0yR)aU$g{`0 z<)ba$qNZqEM~7u0PVxwD5(tIV;7-`Po4-6_=t!#*^5ZN1BG3kwNkEmN3O$vn0L5LV zY(_7pmNHg^RUjmga=e|3I`6BsURs#K-hn@RiyLOmK zG&?>tpI{zsl5son=1i_p0FoS?`ObFw-&|iid{MU|tr1X=L1<`rj{~gl`;Xo@ZiDTv z_XtIDec)y4j&k+*HtC9lHK9e~Qo*v>dd50y*PolF+tu{(N*y}>%9^4o@FdWsvCo5p z=V94nP!X`4$iL-LY*P@_xQZFL$dx28SK@hzix#-lK>!_p8c7+*I=j?j4H01-y$|&l z3ALHN@xXSwnVF%C<81YQ(a^JUYUgnHd<#YEBzS`M{33H=vj|!K#p>%tk|(e|$#)x2 z-3dRnS3dxM^~NggSntl?n}%tOjrD`7kVtN< zCjQcUYWZ>mf_FVmoS;Nva{dOxZ59y^&%u)@Cuc!#shAdDNEA0po-urb(lzS|I-VAU z@pq7t*}qqgquz$gO+#)Il?N7Zc%f%fdV7^JQp%W-D!`Y#1tn}Os@0;0q`fomM6R~p zh0LdDndc6!xrftD_L%be&qRz{eNsJ3iG_t%3Wx`a54{8ug_#w@X5QRqdR={ol zAS|6V`dBiW0ysW?%?xh0{vK7=(1eY(*w1n??WV71mCvd3n34zz-0I*}HRe zi}`95+_x2>NN^77S5$T9f;JwAl|U_&sd;*VOTomOj%0;4UUUTuxk}rAe z+dY1zi`9I!JaZ3OP?yH>@c-3W(~LreV8=1Lp%}7r#V()>`RzCY(aqa&@TB9%$J%qr&gUsH8<--*$dwIm<84JvM5Go zQP~0(B=SPXg_AO33?y`%k2qC;2epA{H6@uTeF^ne{jWz|&oey`h239)o#xVgR>&}3 z>It-D4q^PFaNA)0b!NI@yd9a2yRd#iFK9hpaO5-VAfc^I=+V}G3AaRb7h-)KL=`pY zH$1~TwL8$?7;Ta&%zScw8H(MrY;6vrmo$mNsR6LYvU^? zYS@3$0)_&Y-eK7UNr#+5GR@`Wf_d5MzA?7NFrb`xz5M5A4Y%uxIzyuR^jC$VsSJC? zNQ&ceB26UtU|a}|)o&?!jID)+qn8rTmtn&Yg*Z=2@RATJ=x><{gB)_Pb&3+>XLB`T za+TsvCXSG?9RPfFtdywlFezdYXH5Y<74fK#NE~^zc8RG!0YSAeTG7afgvcNhAM1MY zQHsHaGUc0euv;!eg(+$jd3Z%GmNob z+seOM-1jlV@}F9Hk_B#GN~P{#gz19yW7Sa z>FEL(9JxfvqLgPndi~j5dJ)sVEMJ%hAe1zvK7PA?n9`U1%)jy@U?;8zVDoP8TM(1H z33Cr6%GSI3j4lrgR&WgN4V zD^LY?9Z3Ugdkfo)d37l-4xFaF8NNFT1Ufo`%s6%Ol$^&gIf-y8MoCk<_DK`q)IKD< zt{h5++^X%ird6KN92X{G3Q9T%MhKHQ#-ou4HmkC3s(A-IVA=)7lQhxS{c@J`9vWNiETd8w%b`L-YQt(l$%1{)iM{ULk6Rcc%kKwU z%lUr_50o3VMkLYME@QG83?6L2HBuu*v|j(~-!6E|XcMTR9KrvRbli*kKxlrZyO9wUtQhradFm6Uz}upu{(Vu%`M*pl36c1)cRM4 zLCS^(uW)O%YUfOZEBw208g85OHU9Pp9DGRvDmZ1xPD6Fq|6ST|TtQr0*i|)>b3k0TQ5#3QsY?3nx*z3-hgbk!FFYPOR2U-~kK8LT-pi2P4{Ln<#2EP=1MBMO zn2IH|DbOBtTQ6w{J|FuVBfO)Qb$>r&h@azaTm#)XUn;}KW^5NgJhcXQ;~T_Uvg8=T z70a4wg;p;?;)ozX$-r4wps^-M+B>}Pu+nIKe?&(vogLlH%}GI^(>%Et)zTpN`>l*e zM+`BF@b-YzJh=u4YSk)H>DOWe;Bb}qlREs^N+xfsu z(5$oK{~s+D0%|sj4WrCkw7UrB*G#L4Zys)%sP<@3-})(1t=GS@Cr__ZT`C@^8!D)Tf=IdaWqGo2dkWv=QZjo z%gR`A&9vN4w?AhBsl4X_P`vr%D5H;uNXJHhZcO;nq{TlfJ~H;S&P~Y(8osys@vEbA zBg4chF zSw!p>0f=%7x(dhsO&?PGXpmlY)gofgFTqQZNhV;!mVZq-@!V>;p!-^5=B#OBKWMzr zU;#c>ROU{*OWPRk+Try3#4{UTON8;^2VoPmr#!tLx=%@tOnj`Pc;zv-GkQ`FA}JjU z672xWJc1Z5x*3RL{ATr~p|2QXf(QmeS2V34Ls+HdK)MMg7|FK>xu_pcVU)OzG}Dk4 z^E|}u-Nw*s^T?apBKP8aPOk_jN!B1sVi2tiv0j1GNtGC zcbAti43c>~`PpDFG~PDbrb|XdGNCSL?=nNho4QW*Ch|l0gR?oc%kH2C8h|ukZFF22*}TLe(Acw4zu(F1*w@ZUab)be*)wUo>Wh7lci<`uE>nAGp}N*?@AKH@p_=JODt4r;5Tii?_Z z%Of!lcr_+CVEKY{r2<$wF`r0IAxB=i3v{RNyl#AjA=&G3XTURNc00l)L?dNZZ1 z)sOs?lO0F#BH;9)#kIUlbV|*VA$K8N-k-mJjgg7#i<@&aXM=UJVdFx2?|x;ZG~)dH zT*OoMk;!!w7+(@3D5%eo6j)HJ&3BrR8c_?9lf z2}`A|GBq-t@bw)Y?Z#us$UpqY;3wTnmt9A0XXFx@3fsLaz6CD(F^@X}Bn0AoT(s;! zIEJFwAM;V(3p4sr9+DrF3`#DveX0AH?`gZwJpOIMqhukkV_h$Q2ejO?YDDj^!gCzL z7IpY zOUeYKELIc@JOX8=f&21IqN}fqiE2FLgav)5`9LP*R@8m$gqj{;!QsxWQf}n%gR8kt zZ-l~I;R@*Ao>YrtKN9VM%-R%dde~CmaTX+n&JUA~d+x)!G`YjRPE|B}e?-&^XHSPG zNOraw@*$J4v$?p!68f9Us6BuaGs|gWTwl2U(}mc!>(99&FF7kI!(n2jf5_VrL|)QM zWPF0wBUlO9M*r8?*7+nRf{$|m44~h=Itp4EoY);(lA`wQNhR{;R!FQ^8>$SLWRF(d z_HPB;_L!sF)TCS_eHCV@#M=H3=6L^+Mw&g}y}Ca@$T3ZP_grGeycs`AW!X9c%Y(z7 z;8aLzZm&(Tlf|hb2$No2e|GZ#Kzr>^tVxRg0FL08Xz%$6H*ziLYxAy6^xf&4G%asK zg`QaEq>8G~kIl&f!AomM-2E5$x%?sl}#{*eoBhQay-PK#x$ZuC6QGmknDWM zX%%B~iwWBW1IWi}(F>Y&%raNPUn#|KfM(Hf+>7IyUkqmjed z9FUx(licIKkSkr6EG?QDkThrf@hE64Y~(Wd!%^lzMD@QgkcYhE0d-gB!8Vf1I-0r zg+K0J;Bsoe`RV5C$_qIs`j-XUg9U)xeMnnqGi7L-SN<^c9F??tO&$2qGy{-wRky#_ zs4Apz6#7#x?A`n2JyKaTB#ooLC9!x=c*k{caSrC?RMpnP=TGON2~V(POSB8CE}aV# zrBsWj4r;r%qjAyA9MZ;jIdzg!u0<8*0+qqZBOxM>R zR4*7{Z0G?!1x3L+>8KutJhZmYKPoTiQsRI#3Z|t?t+B`j3_3Ut%@9^&>wEjl{7`iR z^$T0SHm2yB=}*v`{oZ3}Xtybqc_7(gCr>UqbzP_a{)^^{aS6G9Bxi)yPJB?3XefUF zT#f6c-V-sCQa#_d@}*{i4t0F%R-B<289#?G_8wHz2ps*o9M8CZG5oH@`YA0Fg6J1J zdp#8^9C^&Z*Y<)_xAbrEPK~)GvHQL`(o7A3 z!d2?`#?;?}Z2c-=AzX*7sO$VGW;k{V1fRj$nN(6VV}g6G1{i2${h0IG)bm1#`n`J77DYGb##8ls2K? zcjRpYl>W#-yxs8ufKdeHx)YcG6!_+o}{QM5=yQIg)Y3LwFY_j$i&t!>tL}2ru{VmJcq& zF#R-FC7arXOt%ZBi5@}eowA&B@Mvy3&hhuV@e zkGYRsf7e|TtZzC`j>sc#R`q-09 zD&-U)>;o`f0WfL>4^vI=Vb~a_h%IWO$DH|av$6P=1|cjI*9d$Syhm(YA*QZe>jXf2 z0_RsqYyQ|1Q+jWAYFZjTrz)f*1Wzq;MF{$n?P|oE#jwTgRri%aW?vIDKc7}rGJimo zXDE~m8ECq+oWH>o_ol-n-)xN%Gvjho6n2<6{VYjUYI4_%*b+5?Ddx*wv`l_}G45gK zW#0JUEzwe&M`?&TZFcSi%PvN34Uh4D-89g&aXcOYC#}iWp>?0|8PKt)yW85@%$zqH zsa(IkG`K2WDYp^jlE27%&P4;6lT@qxBe9VM;{G39gqMs7ebg~Hn8nD6yZtp|0`5Lp z-Ew4K`dEI<)#t9D`weiJ<;wZ8;xAkQ!51ZqEmgzEVjp{aH?Nzi!whgTP6VhpDM)M? zapZt>jsl>Uwz8nd!O$zna!_FVJFooiMO2(W?oc^e^Y-dT?o$Im7`36Hqp`I?2ZAC& z=v^!53>w37Y47Fb9U}|-Rrj2yUr(?kmxc$zPx*;-v?w@TTb%xYME|}yMpVoA6WPLj zTQoO=wBn1SJuA?xZ15Q83NYDUAJAgPuN z$OM%vH6d${1Mwe=d^iYw@f^0kdh2e)68$8rwLy#9AaP+P@XVTbJ|=6<+0 zuWa@+RZIT$4`QDIG0D;bkNB{ovdf=QbZ?O!ZMvq1s@KkV;1-)7Ri2okEdX^-l^5>H z;S+y(&fpNfN|Jg%-Vg`9J@!vaD)kQ*#!wsh|FYg)YV3MjVql$w+Qs170 zk_9<;?&25PH>mj%QE+UMSh&RWOU#ZUz0Ox;xB~d)FB1V*yo*Wb>22Zd$(^NvZ@j{> z!1Q=Z0xh=tMK}nOLoJ#e`iT!z-sqFsSW`au{e}~d5^XpyPk)b}wlEXhJO&SF{R!X9 z_onMEA6-%H)G&SbxK#oS_0(N9A7an`v4P3K@P(EgUV~R#t@SXg1%Gw%gg0%+Dra zgb9{3-l)frvcMW14@~Xb%DLzbIA3(LHiG*n5&`nG{c1PO+0%8{q#=iD*btmG2kqIt zTi~ikMccejzzbu*bKnozyQtIOLy;)14ro|m_+Y&B&s1lC7$k?!{U~c72Pwv>x!R!f zAb4-%Xk7{tIgRvLnTSi{i^Hk}{U64vMi;tI&6$%Ul!@}qX+SYTK27v> zgW*L|V}ovEC;>_{$DTI;U8B-Jde?S2^3gmrL{-x=&r&+qd*JIxfW#L0^7sKohp6lc zt-+F)cuUW@KQ8Yi!*I`0u$ui$lDUzsv?~lhqJwlEl$j$PPukWyU8R!L=;O8?NFi?D z*;NdnZ=}Z6z(Za*d-{2B$)V(>yH>z`BKvx!s#})QWT2ko;0s29$_iuXMBBA+z5s-F z3zuv@3#3C`v$d;gGM!%nH_7`6wy=hKlS_PFK}@o=kd{P#RkrE58xGnta-08los_`5a% zSMm9RAV$qk^YH6_4&nB^;$jefJwb8rt zX-5<()V`F$?bD_r2xFuzcewL-DDZF}M*RFnB0p3uEm%rv%!M3(s})mZA`NI3(});O zTZ!u{GtW^+F}AJdXpsx7@nYxA9J=JvGQ=#U%{zs@>;kv=vIKr#9x=Ou+)dtSQYYokWIZ`qrGlByRP~3wG)M|guB^ypu+cmvO!L&Mt>)0fHcQ* zs+K!`6-qR=vq^!d@ik<5_h(+m`xbaTOnnF?z&kMm(0hPSC!jkYmO*1&n>{e@exB84 zeQX=IWP`Gbk4<^2Di6))h{R-*cs$HX1S8bWju_H(RfVb#>=&zQh1kA#>t9rr<6l%R zHcNKs?x+gn9xmkZINcD{2`Y14!Kh}|pLPP|j$f4+z8J3=C*W~c7|?1;3+{zx;vI*b z5T8D^GSZm7F^RuQ;Fjm3wS?IMZ^7E91{(b+ktOPL^ zD+*8n)rSgXg8PXpX5CE6Y01h`TzZxcQ7z~@1mZg2_VaIGH>Wc$-1N3p;Fa#iA{zk> zs(yr^(XjV=i`YlCNgWX6DFa-Vh8iTG*j}tv7Tk6tg^19qJl!UNeDt(XX!q#nr0!M1 zVY*P?PPSG3qwt{jhif&Am067rPWF|`jxXgv4UI(U)WA3Xr;OQXoI-uRo$#)EU8P3= zi;rh}coggzc;{baUCmd>k&>aj?E}7lfVh)pA13o1B^& zF-;9|gPs|C?cM>F+Be$=XfZLY3?Otegi5OYV&+;7G=@_d6RO)&u_otJ4!Lk@)RD#h zEB+R6cK&&CLoV*X%Vd%(%hjFewtQ3`ICNVirVW6O3d;=UD3orS{CH`wx3wFA!~y(G z9?|a?uV-NBAsX6`Dw`kv^menSqb&sG+iFmeV;5#U7f~v zI|64!UE%BqDUyzb0L6zRlX1Fx{kCObL*XdA9Imb~IXS);?`b%WPPkdAMvIWkqcKKm zzLbz+SuvqpaS3RtWv?cZN>brCvhrjRD!_t#z8jxL+aoz38ZHRg)&4B!Ss-c3N_8VO zfU3xaV@q0-trXD7ah@IsZ`JI18AThZ-j9h>>H5?O8}Sy8#exU&@G$*M6rmQCQ|R&8 zeU}j)YY;owr#R1l*i|eHhzj`;7+#hlh=I#z!8}1a8WtWOxCZ2rySO$>Hk#c-_R68- z&K0_{<`eSK^sW3qPKt-vBKtf3rfxo7 zO6IyCrEHK({7a56YFZWQt!nY~^dPS14N_*Nn zK3-m6bnJ1wV^|t<&CRpThH*FPjk^jc)w-okgh6YL!=T^uDcru3fqU|Uvf}ge>1JJX zk~lUyNS&LzrkGNXKcpVLx=j-YTZeEZ`K4=z5G>38_bYv8#&2wPTbpVu`$xebv5AE)dSP!-1e zUdi*1IMQLx4?+e8p=7PA!w(OoG@blKpxW_vxZ-A`&EH?WX29xmWW~#^6h=#I+&}^0 zQAfrl2#aXdghK%9uFyV_CT?7GQy1W)M52DJA_g|9SPSmR4IFIC0s)mn%P+OBmkQ*K z8s`+CltrP?#!*JYw|4`@PtBgJk zv;;4I98GzGa**A|HaRM+VRD*#oQZ=7iL7_pi>Y{dTrxTh*RFd%4<2Si6J21>&{ngBmBXjs zp#m61G(LPm1SO2=yF2JU^ajgfQjzOnw|FEq07{|qfuKhUXzfsXO~40+)sGb^L~rs`($pF1T9w*R&pk^#Gb{uF`i7(oQN))k*$6?AajpZLOZ3 zlW^Uyk2rRj71)J2=(BC>qt8L@n;~dcDG$7}vqK*LjvVWHSU-(UdA|u)!n|89z(bos);&L@aUamGMi@H&{?Rm^7BdAKI0?N zCAO=@-fX&%9dWVqY`DEB?CrhRr2E<^BfmIct*A_GO`gD)TLvb{R7gOuvnv`(@!y>= zGCDr-YjYq?+xM^6tKbSmoI=RIY#_i?3Jua_q)P;NBZ&4rLzHt5o&L5)lna+RBHJfwyal+it{V4y+QDawa z?l!=XtE4yA2ZoGlaWGS=;3w70%N?RVm*AvmY2aL+_)Xt_r8G>>7;&zt$+s!PjwcVDVBVVJQY>!40 zIi*Otua_P7kecAC^I>&0BjP5KWpIFvcvV8m29U`Hh)HtYGT?i+Ulnyy466(1kjt4n zk75}daZY}9{)6d;)(+QS?ILhbRLkLD(;TQg->Li(Cq`^NWhaV-&xCafDpHzTFArbNGKm}0UjuB0;?ghK|6sL&x ztAmh18O{L<>3ngP3F2iwbIzzX7sOqKNg3%K1vB#IN}jd$ZuIp3Y2F$J+Rdb-aCIoo zcv&$!AWe~`u&3+U;Cz~KSuA6G5XH(beyucB{l$ukuUv#mRU9`ZFmu}k3k_J9p}K_i z9~HWq2nSw6)Mna)%QC>EVJa4U=7KV+Q8R;@1~BYgI(rq4ABEAeD=uaJgHj>^u@h`G z7FdWKfc*6Wqs}08G~5D!#1tAI&@jzH*A^f$NK4hs9fX|L+1xUz@z-He8S9(qJ~w5zaGq=$&B$`D#a+m!`b2VKF7~jZR_$dWdq6OwiIN)(X@PBC9ZSl@_jno;Wzn;u?)}8 z=V64iX+bscCR%$Ba@C}=YaMcy=wKBg7L94sxtl&C_aBV?r+Od@W3!h|Py1$tpl-Y| zBTAh+Uneyy17s75Gw+h|MSZy&J8?@lV9pjlMbOkWGZSSIuOqd+{{@YXQ5>Su5R<`U4XW>401_B?T1Z%ISNalwBB->q4AKcFC&5J9EV}4!mzKsxHBW zD}NGVTynE_9~!1v21Q^pXf=u%vwvoWPVQ6uMVB}UOv!RHCu$lS@L&}4e6o2K2^da= z!~)bCqj$-je$1rcX0yunNPRQ*?p4FFk0O)#({|b;yoAy1qn;NL3G6li7MJAcu{RfNQZdb zh484{Uc@Wg9n$b-gySENMp34+QvsSG0xN9O!}Mt zQBx8zs2CbgtTxvaldkgYvgL%d^_T7>jr3p?v_AS&w}hyy=>^3Toyfhfy-*) zll29ig{+*0(jFwyw`-4#$_9t4fuezU>aDIC$kC)(DO5D#msY7*A>h}mMqt8EqGQvd z2>J8wf~S$pq#6@Zib=Ya9%5{z1BnU`J~3TY0c2m3c!j7$0BJkIiVvrGvP@zN0F)}2 zHD>bVHUI`FXs1yCPL1zS3i4$9Pi4jl%knBhom$bC9F9*1eF17HFStF+=9XMW?6L!z z0G}0Dutte8LnNY<^3W1$Qy#Dv5b3l_$I$bgTviGWiEZ3Lj0%+%2!@0Z0eFulhB-03 z?g|_{o#!D{hh2;_=71=pqaAFT&_gDRs^YK1nJJ_cht zq3-nydOEC43_WZ&7sYUkt{W*?x5$ZM%4X?NKyZ3>)MLmSw#{=|9)$PlS+K{q)4|i} zcknbg-_7|jR9~MqHNHJWT-8I;!16C*{jJ}(Na(KT5uph+3`gim9-|(v>~Scr0qL+` zOxTgMfmxDG&L>LP64F+hu!}5y3HEU`6A%sOybe(CUpxiLx5u_IYk!L(FAva8fZWHq4gfpPW`3pN}Q0UdAOw3vsr#&OE)&jKaYW9xecfUv! zRbOv;NYu_*&f^H|hh~SH;O&~}qr(p zXlL>T6Va8;M)#(yZ=u#gI*^t~jeUbI@2LGBo$lRx%<*2O?Ub4ooiVMq;2Iix~0}; zg`C%Fgp8}B+L~kh`D%!u1gex1058*2!o|d$ zKxL1R4RVuuBO-hjrvgGC&XWr7OaRp9`#$2G7M*Sgm&F+uq6>pP1hb@ST!5#REvZ+IQf^7@BAzS^ScBlc4PyBh%cofJEH?@-3I2gp#dk>sCZpg-7w_QheTT$dgfk%xW>p&5VI1)`AmXgMUxe09u|& z1|B^9qsYWD?g~NrnGcQl^Jg!;Smz9*Qh$L(!;b#fB@IlcTx~WY2^FJXl52$^-=~Qa z-K65yTo+xwQiH=1s@4h~eu*UpnUyp797_a6s(E!JOO)+i;_J^?Vz0IypX_Ieaq?Tt zeZ~^iUhLuinugWxEYSn^lus9aZt~eu4pICYrz>WWz@(6xZ2s>$hq_8lVWE-&8|4Qz zRq121@Keu|EBl5aJ=SrFt@`Li_cIfQlO@i)ap-o-Rv{`5U%ODZVE{@*+wS)qMdLWYW>6M`Xj~WWgjKoM%iaQF zjS=>PxA+g;x*emVr0Xr)2s+D4rAK=m+E=i6OJxAdE~U3vH3|{_!{SOb?kqP7qib|- zLHW!rBvot%uTg$O=@kAT(Ff*9h|8XP6=;vEdr+&ECtk$X?w6~M2R>yzMN!2c#F6n4 z`o~j0MJ%~3`xSfu^KXCiqm|Dt>VsZ8+mKZcBCv51rKoCy5J*^5?_&?+B4CtO8(e+M zog1@J)A<%=5orth$10WM7!u$E5p_Iuv%c74ZWHA8LI_=3ak&M7c|-cSvvmxN64$AP zbs38Flr*-+>)Wicjx@6It?tgIL7kbCC%$$LJ`NY_;Md_xLHMsNsCPMZ3{J z!t5MQ4cIMvQ0pF2d+?)aAZ&s3ppV%f+~qsQAnb}w32Jk;b_8;@RJf<77t#X!_SwDQ zAafDlq=?2So~S3D07=osEn1IZ`A=)A=adC%uzZ|GgU}Nx#`)Ae!$_Q*$**dnc(WHK zhQ_HN4e%9PFa6_H!Z;G0Qra={>Y0lhVIUullszbN#-b zesLGlBRBQRV(nUy^c%1E2!{wF#g5#Tnk56&F&$AqwZ||s>wO2|eB1P`zJ6@?Rkwc= zA%EdT!C#t^ffN5wSk<~p;}Dx)2J0`+{2ozuEUm|>E2MUM{|-%}J87ZzM?L2Sgw#Bc zSB(qHagueeSFkOcG<0@qe%lnA?I}8k-DHpo4i}+?7-T-gog; z1i_x_>}TLntG7v}|H8OH!%de9Xtbu8FXwYipvUYq%{8*h$OZXxa-MQCEpnL7QFMR| zZ<_VJhp2DB!FeR?Ta>Fv?ci$lbCkfyU_*JA(QN5e4?#%`r5fNokB~mxMSsMEoGTO< zy<6ei3d-I~n_UTsYE9Oy#I=__cePz5ivi#E`YX5#3$al|=8a!L6`Oy_R#f_wul;ws zp`j<oZDJ^e z8`6;N+dyv@5$Vl?Lh%{X4VrH@HcaD&?NE3=YMAlHrr~t7Sv(om`dKRTZq7L7-H}=a zm?pH!tH@;BfscS}csR{SagM{0&ir&Hbsv=GY#XrqOe2T)%x;9baB#fxJ(y{PALnnJ zLB$YQ)~#zW@a^KlS;BdUxh)`CQ@rC46(Et2pjPc61?#WN>^hB4)G()ap>UN<`rs>H z7ffdxWcRkw*vo3=w2BtL$dPpp@2bX`X|Ig_Fs?5x&Ws4i2Qza2^RD|)jh^ceeJ!W2 zB}seN^(T;s2!+ACi)KOmmK0tLu;)U~?x&xjSRi(rOUu6@4Md782=jI_p=NTn~gh5wek}>VUu4o{QmrhQ= zh+t>(sCQ?JFL65VSLkf67jCll;0YP>h&eH zkLqPB&T{c7$L7w9{a55&5E8F1AA`*gDyN#J?6qX5LjxlRV@SJS)|unq9g(8+`+=)l zlN%mwT+fn-vY!JHz||JbW*gQtybT{aZiMsg?39i~o!EqrkHyO=qw(-Zv(a=oxx(HR zxI{TL4WiMpY-9W|mUyZkrVe;4M#1A)_gvPyB8ak@hsCh>Y*Ff1`sn|;3b71GEe-r# zAgjzWWJO1{q)AnQ+W%FH^xVsfcgmF?+Xv~AinrNXo1M|bj)m=zXjzx0^isMtd%7ip zOPGTqBN=`3D4!LOi7{laL-Cib)Zb5ZHY~xFxR^}8;1lwGI;rxVbF^Sxs0;-;I<&C1 z#^bkeBifW&Bc1;4MUm96Fu8H~-adVmQ=FT=m@KtjU5HYi4jh>3tTMbymP|ilR0`Hi zPLdcbS3$o)R8fS=HA``h5f~BuDnZU_f2i0-6&K7&=jAjvxBn$;&xZiQpEDQ=hN0F%nm-DO$+!dU?<#`UvW zDD(`NN;1t_2x6`~lgSnP<3SKDjN^J3uXzf;WlG{o(Z1+zwj~#O@2PgX_k2-3WO;m< z-drH8Fp)wX!=1I^6jmyz6w-P27kARV_e6m^#Y8lE?K_SojU z@IZzjsHzV)WrYZanQQ^-H)VNaq{~n)F+C{Z3Vior>5hXUxhE7E32r`>gk)Qkvr14w z4FHN%nxccuT~kw(Q$t6jELDa5d(mchUcy_%sP+E=3=_Rv$zU6#OLo2W;_hdTufvNM z`Oh06-uhNx!TVzPl5q+?H?LdJfj?KVvDK0KzJ*|aPeUYu@XHP{vrD;d*mjwt(qu<^ zeWrDS`J0I!KR2UMEjUl1oZB{8?yBv0b=vPk`*; zg3xI*V$L@XlBBqjh%qj3E6|}jxu9=?$LAD%*@_p0$T5Akt5h&t7tlNGP{L3%(mp(^ z`H$9A-455nJ#gnvLwaS~{ne5~Z~g3wiRuHiFW&gAQQ}MA>KBVTv{gMj*U-bXs=5+g zE$X^~{<-z?RDaeSd{`cm9n@rnqkcmGD5z!PSg4+So4*oP!P~>J&joP-5vWOXTH3D3 zcMmejo@-ZBcZeCcDWzR{p6?;UusLpf-F;s;wA}uWN4cfgx8UiDKVaygF(Dwm9jNDQ zZ*6+$nf+>dK)}~M9Y#sKA#19Tv@hA6u!dg)6IWf z&^AJBHrlU1!TqeI)}*$;F`2~u09MB~gC4S~v>5+}h?1LBL6nm+$?(il*gzVU5pb#N z)f`PtRDz8}sf^J23_-q3R0>%%xi0a|7W9a+nMFvCbi_CyLOw=xYu?k92t~s#%mZ+H z5a`$|W5-s==I#nGy4jysFN9xZm6gYMkNzW{-rqUgb0s_Wk(%d`{r#I4+|TF#{U0pI zbq!KUU;e-AkQJ|0C^52LVXzvfIEn@JT7e2H` zRuUr|d_GG|?$O{6hqcU>+8#(9v-0(WM^EVg!xAl+XB)0OdN$ShCoCWm4c#jL)Ok7r z<(WwMJRCagt!;ifMnXbb26VSy$B{Ngf7mr&sMyYykczMg;2PAL;Sc+JKe=B-dERu= z(%B}Tcv^5tPSHI8!SVPYh_V#|oRu&1T28*2cZ4BV;U(t5@M2 zpE-ds593&7x8&>e(*Aj>P<)p>e|E;-=uh*finm^!``mM@jgqK#o+*>=ww7uX?7By@ z-WiGP(rDK9wScG9F^*rQ7C^)?6rm-0@y84&Ml%WsDeS)HWTMR2!z zG5F~8&&0xUhL~i$V#0h-nfm&_HJ_WP#4UtGX=Md*PNif}U|2ovCRc2wVNDoxaz+}P z_W@e40^c=A(C9j*Ip?ma zM(1y2geWH2x^bO8%%Zmkf|$sJ@p=z#r9u{x;Q$3mIkY?#jxm5W)J|l(;+<>>5n>9X zX&$jXi=IBaa+d_Y?4o2^>ZIj?mY{pctOPoaLJORM6PqoHDy{7;AgW|=42J$kSCQT$ z**o{-S7*cEadd0!g$&5QiV)PeQm*6mSdzHe(nA?v_gI$rBU{)YrLHO+xk_+jD|MG zcIK#VlLmi88^xEChBy9U0+$K*$pPGC{KerlA~H>${j;v}4;cxOW>D}1+veG`pSxm} zFMdm+c>1aE$k>r0HFmA0Gmow5WmhS^{gujOJMeFRkZq{=ebO+FNdHUh)_X`YrN@J( zAE$I0r6CpiMGD2*gaElr?FY;|NF?k{^`3PI=%u=-&DV5meBGRioRlB zH9lefGd|W^YRYw+{PX(}ZiSR#u9`aEemQw?tT*87CjD8H!d?Z#uKHv?#{(*D?VNot zq^)mqIvl)Fd^(=|om7$7@zw*vlckpc_NJ@`+NwJ(UZrxc2u1;AT-~Bp4Nl#g5qaW4 z!?sb|N#V`B8?q`A;tI3vY)E|hBGjFf>&*}-FM7IP_JYGa!#rRGg}}D5ue2Xj;}H;4 zVT6INZ?%i|xNj#O(gr`k)a)0BpW=n$728n$&G6*OS!vEMz1FvevA9zXH*@q)zj6}; zHx^Vi1AZ<3NB4+jY&4UkN<>`J3~Gos40n7lY@z~ut41$Ic zuSs}d@L_hc#dMlP`um%czz^4@Ww|pUor_;@ELlIcKOx7FfEUWUV>3I3W%(47Xb7*cT@7yH?Vb`2~@xni|0CTDF zbXYok@({iUb`+iN3vn_(^>A5mO+Q{Y`n3ahdxF?mHQ^{irys|4@$PBs~N(H8B>o~W?0=5wJr!S+uDUh7X;`Sg(uFA6 zN%uZNb2$M&P|Q^xmLI6OlmIBQ77CRNw z>*K{{m9;#UcJM2_q!YwUIIVj8eM|6X`o#Ruqe+=b zs5qWGD|~l|!!pSd^lHORXiYbRqW+!`%h2!9367ZU=mN0RGcW3rWZQ9IhrJ4HuVtB~ zmeIT7pkM+TPUe_Wq|+)~$Ilo31T zJ=^}3CYVPzc~s9+d*{v{7{}r9dnlrg)Xi&}v@}ilVq`A->j^>0OQDhY)ZBFO!P^Ta z-Eh1D(b1~qfmHie{@I~#iv7waWwE|TBglL5j!9S>Md6bwOwM(q7B9iTUH6$Y@d5Sh zJqcskP<)7wCyFW&dIvqcY7ByF|Ay1OOoz99tiJeBb7E`S26dU;WkIDW^uf_Vo{lRmiw*2S}wacsQfVQ&kmM zmY1U-p8+jfi%ZSS)B{&#mHIX;1IyOQaiv`n*EQF~;&R4Lr+obb9D>ScLMhR=%Eh=B z*5B`dh)V3&;Ex1hxk8pIxVwgi>jczT#|0b`{E;x?0O7v?O|$rTk3^6Fs8q2xlfw13 zzJ)URY`e2bNo%0W^vmMaO}yE~wLXqSM_4%}y|bHa8}rP?YF&(G2fsg;{EfRv4vI+N zpbP$hShyYjbNN-=KbOr%z{?TG8Js*AQLJ=?^O2<6PMT|vgfhDjGM&Xlbhuw#!$0x; z#P5*e-UfY{pwh@QlCmfhp1TKPAykC3=+6V2LFxK`cLpR9&&KLxIBovaQ%=WUe#YOq z0ZWAL&ckwXrt)l`<|gWI*Osd7uogDJ*7ZXI4$=^MH`TNLM{b> zul*f#5c2xzp0wKf)4hLgjofx6b9mk+so&5|j89m6_iQTRF9U3^8ET?FH@skmjX>1v zYD4m28eaijAarxIq>de>u~p(|RUp@JY{CXG?#`%_n$}sip}a^&U}iOiJ@^>LX0jywP9os!Q9Ej;>vxJssd)S&V`cV4>mMR;^wCC{-|$hXe} z-7+bAit(|q2o-vW=VjoE3v9-6D}>#Q`uBy>Xe>eNFw!DEvz&w8ka2!2B!}VSK>6@d zcw1M!b0REJg(v^}4)0V{ zUlCs`0Hk~siD5UIfSS3;3wUJV!R_M9ERS^ci#edZMVO{m}FnnY_7pSYv7UtL_^{ z^eE_}TnC6P2ObEI^?{&~)fVO+Ix+J#}C3P-jZO#5J9u!>1^o-ncDe0?waLBKEzFX#FZWHC%9EknNvIPH-ry z)pGq#*z6|w#g0-J%#vAgNp1AeNw1#v-b_ zNq@2qM$}&BS2*~iP2XJHCpgQ}9QShAUn8oeOdns-j4d4t_?T$gytpq76Uz(;zx^_( zOnbt8%cU_8F}1D~A}9SeGz0xl2H~}F#s^(p?r8EP(k4VjwS!ZRu1OIp#@wFHsigT? z>en>@fMKel8$c%*INfA6L%MJ`AeJKvdCViB_Y1tTUiD>R>&Zu1icja@`5JM+?D`&8 zF7R9#J#rOxYt()U&`Gc+sj3SEd89z-7ZBP9Rt$<*gHwbn97YgBMw>Wl#=!TVlc`++ z#JBPZ5ll!%m#VRTECk{YVuVamg^-vmj1y3J`M5e$n2}*5B=m>Fg-KSa74JG!T&nk@ zG~;>yK?W+jSvkghiL4<PfOjHy_>Pv>a0nz)OWu2{(uvGS=l31BPqg{>?B$SRs zC4Z@GfP7y0zWeh(!v-4em4VI^zQ;ZbICNW4ToXplTrntv9QhRG+IeYu^IQ97YDB7aCVN&OIE1b;nxu<;b zEI9lhE<4bbrf>qY9$TxtYyC; z72YF?d-Dh`92KZGKQ;OyGmZqG?hJNZdl zRlGuv-LCBTQ~5D?)8C(i?(yBnli#F6d-vXL#&`F8mNVvaOV?8MT1Faxpmqqv{~KAJ zmgv#Z77grBC<3)YphaEwun>YtAu5QH)T{w<)z&SvxNDmR)mYo;f1JXs?$VbZNfR6I zF(h?XRqkkmGCSsjg1kfFc9Ag@A;)mrSsFh06~t>-i@#OHq0Jfrlkfkh{gTq^UQI|X z&k+jH6KVmU2+_rG7*9yS=U`%=uZ^mL?Mz~+4AjBtWGOqd&INi^tSdQ3l@Rru^m#tH- z|HajF=E?Jy)AU1Pjace55b?y$uoFNulJv8aPd{@SgZtnRqMzOI+G5za_m_;%5dHUc zf&qeV-cHB~xoa%4Qpk~nTP*rgSc#0m@^JIRe#NT_H0g4{OhX)9x&M!dzMLN5lb4pY ziG9x0@#Voyb;0h#V{`s%cJ0_~MvezybDF>7c77Zc=j=K1hcE6wN8^d+i|tDc&b{lt z^K0nb{r!8!F}VM?v$s6|dwNkWIZ)TR`S6x+zTq{M56rOURLJIwsW-34Fo|AyTM^N-sxL!yw{Q$wpu-Ond1F_?<$}|pjOJ{cW=M+R$`V1^MAiEDi-{4HRF?!%dF zLg6uS9%dO2j$1W@@Ne{Gg$j4{Qz^`FNQG-3hin*`G-~b&EIsMOK{#lahq)KYZWL_b zJ^a}`^;rKO?%essYKoJsR1+6c*PQ7@*OLaV(N0(au5Z%#8f|T9URk3o;u&7T=TV|c zT?Es&QCdhskk-K&J`imE$f0CRvny@f7r3kjQiLS*qle#K@Se`uqXugy)+u@b{TQ=)8w~Qk_|SjR*-yhsq|pk<)EqHm3k{%1Y(sc)M2Ye z0_AO`HQ^lRgCu$V(7~`o2x%Mz-Y|v;;lEjEj=$2Dmq6=zjQlr!rv{OY`e2-#wgzIk zoBaa=7aLrB{AD6o6y)Z&)%ot{1m_NUKAiw@-}hEGj(cIgJAMvAQwtW3&@L-jRe7Y{ z`ISK_^(vY6bC`x;9iCBvh;i47+*;}k@XEasY3mINRsuNbS2s;>8*zlL+Z9c1N(~24 zgT>9kJpc=Gvy^aaw1C6U{Y~3#Vwc)a{JFnC&Nx;~$&=lhfbo2-IGs`Ieggrnjx0k0 zqKdeFX+eGYyf**@Z5AK~o_tV%We( zGJkUThR5ONp1}aEdjQ#0WTfmeh`dsOJ}`*{XgYWqC%3+bKMnCef|JEQz7GY@aJL)1 z1D>Eaj(pOT;s-;CrvFS(q5x!^m5t?zK*<&3W+!#_71y1T2cDWf!Vr{161{^%iHOGdSJZY3$=MOQ9tQzNo~o4mROXl4T9V6 z>d#w5-(vZcxkM0M!igRQ-4zCK_zCET$+kVP!>foV7qdzy z%|C%%BVd%E3>eS%y_uKjv}ih(LBI`ob{+=**_{f8bXn&(#XtQExir5!PWcKE+^fi@ zWwJ%1;PJMxQ`8pD@E2rRL|GY&YG7-;xPDmZNHp{n7m{9gydR0DjUpAvkC8iR5mbs= zDTKh`!GMG|i4I@}@}e>{(^$y+gDN5cDFDx#)MK#$o0Mo%H`nz#A8^4J-U(WMYNU)!x-15!TVc2~ac{8fH{Y^ffnM!wg%lpl zxBSfE#6L|evU))(OtYLoeV)!2f9{? z3${X+Y44iXXbxhOs-5xEuVi2i8B_|@+_@b3YaANjmQw<;v!3@36@yUqR3^#C)S`x6`}{0E@YPzgP_|ELt?Jd=5UWY zVv#umu==je($*=5P)t}kcC3trV?bwMfn{2j%BNf*b|fQ_jf}yN-2t#N$5Tf!G1pQ~ z+ok1VEs90%2A`t0j!9>e@cSx!O7Vvg~iy44i1sumdQKDG74Mr$v3P-QFf z82+9AW9EH+-kYoTrwIRU|5(?$*IUtIJ$EWf6<_nrQ0;;%=e;TH2k;kwx>NZjt~|C? z`Xnw9uc*2p1kj}OZ^yl_x)Yw0Elj|WbuGG9VqbC1z&-w7^&UA2yKuWbA%|0dZ}cK?YT{YIsOog1>itz5QQ-Fv zCIzpgbyA~!tk}7CC=jxJ>q+K0t=(3Sm#1(cJDc{uW?NUe|5x8SWr@ z+a2-9SY%L1B)x7{+=uIG@0c4WI?-TB&;-hI9%cvAA8JzrzO?VmXLQ0sD!hc|S?~1~ zv~uF9^R-nklL&r#gio;-dOeAnvu}9bMgNFyp>363_Rc}D`u0sj%KV$iiD1rwfZLhj zZpK#?8FlMzFc=BEyO08Kik0bKnytv&bG%tiyj=LKe}j+ouhQ5ye{5eJVwgEkFBy`h z()sEahrU;h!MQ#-d^Meu^smyjqWwRo)ZZjK*+20cz*o7GS9*+(%x{Qy{FQ$SW9aqR zJ`|S)_E$+Iz!iO0Kq+CzT(g@*D^%@+%I#q#p>RGnKKFk0U!Y`8IX>#w%dhSW1w^K#8Zq>zOVI0o zw4S!TKPo62;132ON^u~M4fjZJzu)&;am0<^i1<~YBUldL zj$1o|xi4ZK%abm6;WLOEfYD#LYj;J~raFZiL?d2Vjbj$vcw@h;u$#3??mh-ZSHAKn ze9HzbQGOm+B~U>Lt5NzgrWN+}JscoXB>69WY5<9_f88@O@P&r-(G^E1|7BHK8z!ne$&Ryzd)E5!43fr#z4>>WRM1kEI zYFbZj2NsHt{o=4$%h59^&t9NA39>(75Rb=ciN+RwPobWnFt^(< z2ke;3FwdIMLRcQrUl{@J3mZ&Iy^2ig=XDk$a)M2*8t|2Q4Bp&|0+eL??m?_er^&ir zK%Uc1x`3};mCt;Y`6P$;xExNj8GzwT%3#Zn=czCb?t2kGb9@V4uEsCoGu zZ6dv7?m0d`NL=mnbYkGc}tbaZ*7G?>M|Dq4x=E-6_LF4WlaIs>r9v5%&@(K8~q$@i} z3GcfK-&WE355#!YdyAOS7fpiCXOR0!%2*gwq>7@|arJ@lV~Yud9BI@IAW;K}$wLuA zV520=aFE4Z7j!|#Wq=4H!z4y~BbHP$B2JnF8n2=vK)4!XInp$%Fohjm)G@}bfF%k? zl{Gb}67VsZVTq^qY4DnKr9&jDjn$YNqJuh^NK~g5qMp0TA+bPpJz#}^qY8`^8)@NM zKnPAAo5O=n(#~E4CDv6rkizjYO!PtumL->n;sZsk3Iz;2yIzzY0D_v+P+($!q=@EH zLJF|RB)Y8#1&gfAG0%|{nv)wKN-1;HatMZgyb6O>Oqn>98OE2gEYY4pFd{RSh`fcX z|74REYg|XpPLl;WwP;O^nbG|_{sL6IK3fBX#4t4xF3d@sB@Y=6=2@1ZwH2aR#hNh+ zBtvEJzYvJemX2XHPy`C$xcBbO3@K1P$TSn!_rx=~kE{7Jo_YB8*Cr@eI*Z2$rL+Nw zlH=nN0aN_!>%e;p*8oYDS%!880>PBC3V{ggvch1F!jZhp3e;){gk{AFAXimY zU|yq0IKvAWR&C@M@|$XHQAL#rajjrkvhlxMJh`7uaPijt-oV1{>23=h8o9iiu`M(=g@Ieq4;A2g*z~y_9acH z1_V)Qg+)+eRh2Meb;koBCVlE=2!ZVSPe`X+k?S>ZK(3Q5Zhg}r(JIh zQ2s43x#dOx<=yG$+7}`She5YY5e#LV6`)&|fjI(7m);=Uy)6l$=S~B6KL_jp!M6ZA zfp8Jn1n6m3&30B z{m-JSD=~3jdlZXwY-7w@VQ}SAYY_@$GMAX?nJ;BbsD8d!mkf)gEHWtmY}xBk*97Rn zaQb|Gy}_}`^RM_@K#U+89~V;!I{Vr{)0)Dj7_<;1P~67csoOZV1*@bW)VW2jsaBoRElLnh3am92|V1f@>aa&znNv5LPrTLM} z)k*>pC5Q{mK7;nbo_mF{p($is!>2rsRy&8$i2D7D=7RAVvgl6*uflEo$G5NHpfY~$ zey*wC_$o?}qw|ih!?x-l5)ER!K7=BUak+-emqY72o&{7j0|CBw^`AfUXJw#AztnxM z)O+-9)5gbsXYTqP`2PJJvj&;J8sLzrrpx~XDsjWL6K{o*nrv^k7k97^H#t^rLleCk zS~HT@uAPT$3mnHAIMI5dg9hCb)KB*6i1Xu9L#z{Y?~qp{1CQx!LXBh|oP7$TsY!JT zsc)h>269S{Ep&q1tvzD#7e!U#qdX032SAF`+B*xA_h!7|F|h$dV9)apfTkXo+xEsSS{S?&1FgXrslcBk2f zYXrSy5*^&PC6n5xX94OYGLrBcD>2u!ZBZvWOeUIwRo=tot~Vo+N_WcRVkR2Zk$Dmg zhI}M8s|Li?Nx2NK3N$B*Af8MB;*#vVsh2CCzzx{XhZd4kO101#5?19EBL$$MGQwP9{(-^+e!$^RQ$#p*Ed|pCvvu0Hl`5G9Z(AAvbL5Ll=8QgLDKP z0|mu?l_~*shV%TuP*9gmgE-NqwvdwI)L7%t?vCTGM=b=dNrW1XD|g9EP%c6d{?45A zYENwp-I5vYI#K6)Q0mLFsSZc6k;jU1KG7!))O*`BBtH4(%2S^M6a&<|@nVm~GbbH~ z-5oyye#u!r+irjw!PJD)G5?iVRURHUC%neK26&l4fAv%Lt!8xl>OGeb))+nyw@XIm zv%>J!BrKGq;(lq^H#7X-0YkXh;I&Jzq;2-Tz6BBVK_3q~v(NbXnA>hyy{rhz5LVYv`)>am^ziPXgW<0AM(G*bLXF)BmrG$J3XXRf`VKF^MYPRy0an z)KK@e$7WT7GK(}f1uvaZ9fgHaHD%O)(MPp+Av@i0_#YB4P*H2xa+M(3fw3TIp)yJZ zdhAf!5#i$`QRqyf(+xq4bzz{6QcoWQxw3VEu?R)81mP3cFze}icN$}-EbQaW3QC|% zvIIdPl?fvdmMj^2Vg5aLL)d#cf8`)S#T7#h=QklDWXb==K(8Pww)!l!

e7J0OuURkp=TVS)-f%Cg!3y^A`U}s>@w%2?III(;G@)CvP$WnQC1wt^R(nnm& z8f6f+W@>?^XN_ETbJ?c`3BbyCU-WP&S8lINitLmBU>o)4q4UiZ1HFYB@J`pe^(U8k zyWL_)Psln$gL^5l_qY{K!ceYdD0oAFH+-!Boo;g#6059BOB9bkaf+pCh|W;AdWm`k zt6B_;Q7%?*<~_b?xJ#~=)B-BbB81pDjTo$3P-ynHr1F;{W~e`*&o4s{o3w}!VMQEE zw0O7s%|W_FDcaHcwJ-0rQ)Wc8dAmOPqQCIPf+WSxU4_AK6n_m7HhjyC(+GwIXZfY9 z5fFX@i};7YH_S@xJOdu=Py6U^e}$<6yY#4n3i7ZFTJx|j%gq09hxz(%t>#)^(4X#W zZ@CHV{zgcA8FLKiKCB8yimX+e{sQN2^h)T2deZmU^anT3(+ySSEEy*2zw(M(@gK-1 z3eBbKHAIf-`g`u0WN$}C8ivTsdA&DZ8V6zhLQsGp&%)dHvH6bu3wapNQx)K$W^H&t z{bqa!X#Pr>r~t<+a^RGYyrcA_9kK-D`za3)IaR_cSpw3|7SL zG&vmc&aW@y=921=mcm&2%|Zz%Dp7tCXHuZ(J#e^sE|ef(s8h0(QmFk5hhRt&=0@F6 zMf(lPHanjNo8fHydra7pTh!E;XDk6+of^h6nA#0VPyv@XP^mF>nl7Ozcr=O2tVG)h zjGdrJgt4tMrggme7C_T!1m#r9tRNekcB86M3M6#rJg3iBl$sSCGblUTg#PM2yQ4O$@e+nfpZ*vn9 zdcNjL4qwklo}1i+XRr?c>t;H<{S}?sgFSoxicX!PQ+vwx)U{5HtizU;?FJP>Ex)=G zbEHRa>NS*zXe7KU@70sA(j(x*f%dz8wkqLQ3#~zC6+|XXRt0(@a8j2P; zM7MbJGovp?zE|0|NGkyyIDlP{;KsCZ>UK4=PuI$M_)nYnL3gVx8=B8nt##hX^U9C? zPfQ8bq|5FRmM-MG5Eb=OYa#lVTB}jHo633kg{6J;5Td&E3i%hFZroq$`GHKp9Y5ccR=t$Ru0u`%L#IfSxxtT1Yy9IX#=VJZZ1y!}U@lsC}3 zuGd9m!CtIn@#OQ_aAWIMJGUlOumF^YlYS+@m9P=1(m|GkR#K!}>fBk63PoUMcAS(X zf86SwsuHvRo+<#8nJs)neD|{EF5`hqAv9AV~HxW&C=D=s}9t|qR!GAh^fN%8wQ+u6<=3jsbPr{khnC^`3xUUMmEh5 z(o2PGoUd7sdWhZ}aMlOVD|3EL6ZyW76n#0={;pyOgo!|DHdgOUOWY>PH%TKj zZPtApC__r9X6@a9rX!axOEH;{G5rD7EPpe^QB8t3Vmo~!iB>lh`hRvw=82FvL zIw8BLfv6CC)o$RpZfyUAwj+1*8zCJjavrW}8(iM54ur$b;VT>c-w*o*kZt#y4uUUB z6Vq-bIo7!B=oX~kPN-~=zm@)XS)>f+gm@nBrxTR0h} zocjmoo_X%Y9+$N7xbTzWK}*MS2cMw~f~7yv9ka9Xyg_KKE|g-DD_?6MQ+IBv%b-pm z2MF`G0VuD*U|C-G{+}nRR&CeOywC<0Frr|)&pdPmWD zI6(i1p;J&IHg7ao4|f+MHw_7tn04(z>Jo7Ur_cYoca0K|ZhfW;=gjoouFsy@unt>q zh+xjewSr>5K@M{nh9279|CDmLK-nSFx&B1_g4wtkx-{qs7rmxp!iOYr_^GXNnOKHng1w@jGYy4S|ll832XYlch;I zuhD7Q*@=NOl|$KfymYi5%k`3&xINGcXq;@*kmM_H04R?d)JmfjKDY=Ku7V2z=p~6W z8Z}b58L<>#_!}ud{pUFfz7(0uz7mqYAgL zlj4GmLA13Oc3=vi-#Jj_Lg}(J_l@jhr+zzFpE?UV^xUyulS-$V#a=}ig|G+2P-t|C zhtrAS!e%I}7p{N;HEV_u@=9AQT!rpCv{L&Y&&7q@gOil%(Z^jQYN#=8aQHl2Y;Lv@ zx?NSJ1+U8~jR_ys*n9*YU8=_!Me(hQE&rD;$Z})D_Tlh}{db*c{cn-wA89b#rCW%} zQ^zRWTo=377glcGDY(k2#M`t)d5;c>F9- zUSrr`H4Uh+B=faQ5RIosrD`ZT^-na9{1}M#ie8=MH~VD7mN&WZa+)#-X(FT8!n9tf z!|^OvX0xwce^Sh5pH9CxiWVIwd^?_A&r(*DNwI_PRZzzqo$%kgyHf3o!3vkx z_#pz+O!ICGL&~U%3e<#Q-2!9JboGymyUMXq#;R9ZKb58f`o)o@T3<1&fqx0=s|PCJgidK))Le)Alm6`O!O5 z?tC24Gk_*QH;9Q&xa%}rseZ>hK!OHbU0po_DdEAJ>kJHb*>Nt$57E%LjoC{U@wqqW z=UX<3uXV2izQt2ldM+Q>zgQj|LiJuhWif&v^E?BovkPQ;eXQlARqm*-&aC_jJNaV7 zmRsEN8TpdzS{_^JrItv?R^f@<1O?2WV~A_XyCG?{hh$BdF=T9x1&N|T6>-dn3deB~ z{fy|j8w7-x@ARXM9k=f-lUcldc)bJW{EoQVujGPtlu1bJ)$*YgNtNI;^+T%*@mX3( zk{0oh=WwEqOVlLcUI2l5Nx|u+?i$p%!3BkVedwOhXjpI~UJuGFaLk7sCqDBC+1-Pl z6TCzNPd6gT?;s5xb(iQyb=}bcqRG50j5=A^S$B>2!49iRPF!6*Y(4;+XqYp<+pI8{ z=fUmCngKrRE|#E7qtoxwafD!+C5&1?2c&+LlhX(&oc0zZjp2z>4wytT62rPnx=sJ# zE~Vuq4P{n(PSnEgjnE{%{%!B;`SX4XK^XY{azu6^Q;6_%I~UdosUquFo}g12!1 zOzyNC2Hj%lG%4|_;|}nDAFcay(+GeHUOf znAER7mL0*~RgI1zAQ~?u%;a>m0+d8Ga1RKd>~~fv^uVBLAl@_hpYu$azh2Uh3DZ7J zpVW?WjbP?1tcIkt)9DcNsk;EAY|lWWl!WWnNb%6Rv7`0)2h@R43R<#0s&u~~hU=Rh zhEI{KyV`M7SE6DMf(DvHjTvuL#A?8KFiMtDucPIzXA~}3Tpn$PQ_x+YN}qg%k$#1i zf%C*Ji=pcFv;xzFRE^HSRyY9*B`TV>4o=a?@mM}EaZz53{pBeAGgKT9k<6O@F6u$1 zq~WU)X%M7`=Vf7t#O$h6M&RM#_&NpjE3zpQ z!8ihY7UQe!zn-!_a}mB2z5^HSm1JFSBg7KPG2T)gE9*9=y?1@o0ViPZTQtPSlWQTd zdr$P*>!;85I63^=l-J4mx|O+%$GfhU^6rhwv+KLdyPSJFpBZBr4)-j&nREZp?A(0= z(fe+1YbpXpi|!V3$@cm*!Fm4d+6vxz&?|=ytzVl7Ll=pSU94{yzk+eNoW%*Jt8nd= zhqoR>Clq0`g+D{xuj12HjI;d@+7YWLJ|Kzn9B26qM$ly|jYfI)+j|WE$YB6W?Mjtr z%`SlwB(4{{v#Xm8pUVk)Mqvk?{cF-iLs=2e8QKk@hI4n1Z;m;-<5ZWoa)}s*!Doxj z09#eiE?r2##hY!#3`bTg+%o|EL0b*y)kfEeCabYsv;|jy-Ze@X2)l40rsm7+P=PjY zI{RJRvm9T=gckBC`X}h<=y|8fTa4&x7&^v-eQ8r&=W|!}No{+B+J2eP-R}LV;JEJz zW8S=qV&1gM!4Jktsy|+ZZNDN_>;*Y;qcd?3gJX#KnkVL~tA2ze+~+kB+>T!GnHy@T zS6{k@-7|(*Z5|HN0trYAS#&Idw>t8}H=lUyDsq4y_~Ws)#bda(;``9RVpu~)n1dI71>`=;^V4C2%U3vhsGM&{ z!jVN~9CUm`asr?VH%PwsEk#}eW{Q{EvQBj?4K-Nhl4jhUDDdyJRDK#(gB6%c%&NoD zQ@CcYL*5WqbbM{q*DaO;%t=jbk@@*NvM>4s{9vAU7G4yHKteb{cf7~bRNvRZQ;LPk zVC0G%mnQH{-dl(7PqXwO*f!h(?hur|X$z=adF>IHqIVAC37M=fTm*@~rH{IZ3;8V1 z+b>Q<5NY*9U@uNoc%XsW3*X3#B8^!44niM^pSld~cfjZx6GU6MZKaCh(1w6GV^rxT~NHOxqtP?LDjJN#F%LvIBl7 zn@)N#bVOA94gi?m4uePsiitA{#Ek7`X3{wVO!Z6yAft&RJut~Y3Obg(JbJLp-KBVx zKY}iu-4hvR$`PM32?jF30?>V?3{DEO-J9v21gYBv>*YLEZ0bkX@TwSgK~_-PxxgP8 zxKW@TI7SvTZR*@K2LlVa1IF1U6gB+9|pjk3(! zxX)ro3hvqsFPD`na4t#7a}s}>}Hckiv`@9t(R?% zYM$MgJ_(OVTWGARnhoXe^_qsLqaY z_Nt0laM&XOh@U z71EGHK#%};x=lf0t8j%&zl19fRu@?~QRV2%53*;>xTDg}xX`^G$rZspG`eMmCq0kn z7`zrk>4>sY8mSRf6BXwOzzuqhKFVA7rv^@3yneNTDpj=^@~NVZ3;3kv*oFhfsiTeA z`sj~O788c+gbAup38?LTJfl&zYi}bkjI&jN6U<}Uup&vC@M2`2S6NrmuuA4U{+MEDp!s_tq zp6Av&nGP+NXwodZH!KKtp2t2UnV=bwbKM@)R9kUVHjlFylfK=qZhW}B<>=wEKsEAq z=iUO2UfsPk06DL?D&po%KqGvN#OUR8(-fl9xZdKX%zk+JHV#743w|05H`D{G(UHo% zq?vi%rX-qG4EVBzSYb8{XITa%dZrK|6oBnZrMz|eYhKGkDAjo%DV0_gK04m0(b^M6 zBgWKbM!@gw*SeyMJ#@7tXQU0c2FD0EaJ1H#YRekpo3ux`1{rvAKTaN}@X@O%$jOEX zua+K;wcXRh0iR`DqM0(hs*nR(fLG5KTMo2GSa)K*BM`K>juqfZNH1!BMl_aIpP;%m z`8QxC2X!`k7w%f?_B6I0pLOBa=ds(NM5E~{ip>q%0?e9?4nTsQ1gY|7$OI?X+(vOd zA!aloc)mG|eNI$6uRct2IyH0~g_9cuD>A_;onoXpI^FeEV`+n#IKid`EC>IcJ?io1 zsoaPUD>%wf%JCezOHgEgxjpGDXJzkK%quCceGYCSt7Qcg z#2HX>{O6r#zFd_NvC(|jM%<5RbNp6xn6ASNte%iQx(sV5n;P!VdOa~=f%_}Q|4={8 zR%cVZA=!#X7j#sN1iA~AC7#FTV-sK^%eKLhj@812DhNm(#h{LqIrub>&_NH6#-5)* zu>K~_vX;RrIXx!74hqO$m<#tpWRk-1xz{A1<_+0YH+$fIOd#+=lm+#tTNs^*{m%UU z+kU9As9%iO9KfesM3@8ub$U`$&sfT1PSu6}==OBJqRC1!>52<)U0H|*@y*Vz5ddWh zyvEe=|5nE@hOnBE-X=F!x;7aoW9~pnT_>96Pvhf2u7ZQO5;X)RGljtvHnR~Oz&Z%w zq)2^)aq{l0m6A6t1A@9v!paf^UFup(33|vO>c|E3N~I94uAaFh4Q`)Mt2ITUP|b5y za$B}*1)tAfoggI&5zi84IF5L1tVhNPxcjHA5k1wI2pylf5l7FWj(Gh0=VO;@T=hJh zQ>|n~@dj|v!Mf{X@m{g&ZQfou?b3uhnu>?T^*0wg-{3u z#5!^&PDUPhXTvq3gAVR3+YjHXvDuki(_BoJ)BclLt~s~Zj!d9$l7L(k`wOE}J3A*v zQ$mOdJv)oic^CFC%8w8F%3Q>{Ax%j1wPuvnrYXG_7Y=Q&1NyqM`m+&Z*|Yr+9t48! z)EW4h3M?=y;AVNUww&o`{#0Lzlr{q<->1PU>F#l!#^55_EjW-k+6%@hc=y3<&agN@ z^KAZamoHB2>3aQmXs~Yq3q)3$RAc?>)g!E@*iM+q(KWZg>T*aI7XqFMbEcU#MsK`32 zL0tM34Vcu#z_vmpT6p$=jGU4Y^kVUU9?ov4giFWOpdx=5L9f?zF^C*Z;fqHYy79FK zB~|wTt)QowloCP>vHo2fqnAnCErXIVqp!!R+GD%G*o}qN0-g=ifI%ju6R_stV;7nh zI!RXleu@DKi1opspP#cDop48wK~vWb(G^WNNO9bCWwD?+t}CI79Tp|{J_}uyt+l__d<@gu{owCt!|Tnw}V+hsR(~9M8KoAawF`JbG|6 z?X}3-EM~L27{l^JSn_Zu?)S}zGq(uCiG@eY!*6hA9b(R5-ZZpW>TQ$JZY{DS)P4kY z5cD$;%(OtDa0bF#;S^h`q!3@OfQDB>KRs-K$1&;eAHQje3JQN=`Bl7FXuQ2!&tNR7 zV~Mxyw`cC>gbp|J3w8$!wQTd9#^}MHpEANYX%&=`QK20ysNce%)t++y&N|X3p4zeC zKU0ts780D8g>{0H2Y#!%bwYiyp=(COok^XyxV&JN$iIiGW-e9`rG!&~ZM9!Hp#S$O zWf66dCp^SgiI&hrUzHBY>)S?i9mO^gk;6W8wc_ zX2nn=3Mb#;X9_j+ZjwI+a{iRO7e8oJ%u>oA=?JlVJ4BXJQk zXvg86?v47u!55v`O2|Q+CmHmE^W|wmBWilyVG*ffHGQa%`mMIFU)STr5jgV}|3W$L1A)(-ro(VIO>G|DTQfcSEpfm{vFHMZISaR_CnUE5;rLI;scun6m)kG*E=lu> z#YGAa=ieT~Y=DXm5oMgf2W)LcN5`bK#v#YY>0;BfGrd)!F!~7@*}=J@X@%{DXFmoc zqLuVvYm%37R!mXpC993Z&Otjnyl{K@s?>sxnwhj`p=)a!cVXv$rP zv$RWc{LK%*`}yJ36(}sHM!v5pB z|EIyF(fo3l^BW@HY0JP@ua900OG7wbl+Ro%wg9XUKR@i)N%@2XPFVn6(|lUHS*$iVZx;rbFaXSzxH|K^K(+^< zctziGrzlD)0ukCpww z1C`j%g@xJ>RD{A@wGHYT*cxfQKaV>_D0KfH_d5Ngyf;HCRQe|E1A|!@dYcz!L5?AN zBEfbS<^(*bg}ZIYzQH5Qqc7`glnT@f-H_?TXdf99Xya+fF0I{|Z(NoyNLziA|K`pJ z!L)6Mvc_v4)KfU1@VtAZHmqS_zNU2$ln}AmH24#3ya&t(i3!uU3JJN7)soR>L?=l} z^mQwMYWAvDV<8gQYN;qNp$I`d@Jr#{lcex8L^W0s!wYS|%SR~EU7+8pMNfeMeO!M0geQAwX)pMioAdM>}J zd?g)`NxzEE*tMk$#xvCPEt`e2EAtD+RlsG2XVcm2`LEdY6pJooc>6+?U zn5-ie`SJo$4XY$Uht11b1~6#&BhPysUf>{M)xbergQ72};iaAtKp?}_JgIZ5z-pfL zY)IsZO9~G(02%@leq7+}AoT?%xE|=1AKoGoDJlhp2AycSa|I0#QyAE-ys~(7LB0p0jOB3t9>V+!T@M7JVYJUHg^Q>74MSolsxCvW5=y+08)i6zgC!v7wMd z#%TD&+Hd^lZ~t1DtfKAK27vDK!k6A)EvNq}8SL_YK@5|2lnE?~-v$e=0Q@rnVNcB?S~$6{JX^vkII4#SafC7*zh9 z#8x2Nf9-EL#+lkgCWi$=%$^F5y{|O0H2&xEkuqOTv?gTS0B35`Um>ATpSt07iUEQ8 zgfTKO_dGyCU$NC|C~;D>t(CAI!i{2t@ubf}t+%dt2LlJ=cz2L+D@rp_;7p{t{olFoY{)97pi?Mw2FK$1bZg)U!h;2K4r`6 z1POss!r1c2O0M}=SNYS>j_C*?PJACfTIheK_nl>x#vTE7p_Iy@M+UMN0@SUqP>fdPJ8m3r$uRTAp?vD07X`2F@xq2FL?{rCrToU!*=1r0^u`Dous?t~@as6(5}U0Z27W z<}zgkOqrl_RCN`*2!WJ5;XPyU+qjZIB0C`n{QRx{p_7d2pRu|8s+4myZj%8@q}F!B zT(faq{dk#As#8^+1~0@z+B9=05<167XQ%Ec;-WW|hvp)!84gj#?e=`uCxPkc$AyZw z2$M6}TOM`HhHKv3j!5O?BrL(>W&aWiP1Gclg$U(a*b2O)!*A9Qv3$ zvcJC_u?}PLm&;XKZkI`qS=VLvhOr2*+{1vht~cR;br^@sU#UXa3zegNuBA9I)}=b6 zyFFh3#y%;{Ko{w$8*87!*RH?@E>}SLsO4?8 zv~2_@{(v+>(>A2!1DJ+U_t8*AKKhNh=%LG{6BbPu5Ol2i*LnpzND7oqO=}aA{$amr ziL?cfk0@N*RJl4Krvy?&XslW}HW)r)zdJ7Mlp#2I>h^P_0Y$%i=x;i~TeuD)Yd?*KG6Aa-ICxQ#KDV*De8Y59I}iJ|N_gV84IHdb#tiJY@g;??n; z35e~!*c3UQ+Pnq#YqoAhmn@lbSD2PHg+9@W(}j44BY%NuWL5lDOo3UB27`(qn>S-? zzvk*iTElW$(K?N>#I?OMn(V_145jg zc_+Lv6+}6G8;!&&UKF>HQO)ph=OWJ{FTFNg++yZPNsVoy_s#)sQ30&P0haUAp%OC% zvO3YTM0q=`YiTWVtdVewu&=N$s0MKVU z$t9oWiO-35@WC?DoPDk;a?ZnEI(%gkJ~?!$hj$oGrcJUOtW4DT9DX}QHk6S9D@&&= zj6Sj(U?u65g{1q+{@9fo3A42SO$!w%*Q!Q;K?~ckKEG7q7pTp>{%y5j&nRkM$A}k` zu4vWXv2`slztF&zMnhL91%vLD70B(EniNtmID)!#wwX`mZFsx)4@iSg?-8A&SZ}J! z21<~E90mMI5J%Xt6TeSzq)62`st5JEfBFXV@6xJ)k-{Vk!M7mbY&LpKN{CD?0G-~1 zsOGjbP$$Va@x|5WOXe6l<+%xu<{W9n!XOdZ01@v6|p@d!)OSCQYZ(*GV?g zVWAG_1_i}O5mFaIzgO$sd<}%1G614Qc5jI3aaruutN^4!pI^?nk`KZEvU|=s_x~-8WtktV>b}%Mqt1q zJMWpxfNYXre_E}=??=>>F(tIH&8qlg6nRJgwTItZNrc0r9R@;3`c+IkD*Uc=JIVX# zRW@x$t_^*zz-`gvJjv&Us3B9-1s3CiE4E8SRr`D$^ui<7QX8aCeuzU-6C1I8)LnaU zV;yUU!W&_86iNWJ0b}juZmqU7Xw4Pe(IYJ+JfUc18R;`XmReSLCl4U&^~Fif@ox1V z*uCb@h}ioXDrYANtYbVU)Dt}{I6yq0$rMb}V?BD{9>?3>NfMVC1S zT5FX7t~3glwArVFn!Udp$jkNJO1$L1J$2r_&D@h{?;-!_p1=RJ7Wr{Igbk z%K?an^aHotIvhiydCQmjNnK+8NjG75X zz@Z5KP7(@RHhHhvf|WulvfEen(TT83I%NNvlzBvcf*)~(jew}bE=7=rF;1H9-k7{% z_AQSCgh%tksaqXsTY$%Q2P_@vHb(ka-OW z*K>OdtEploVAI7Cy)w=f!+c)0A+T6QRFQ{%{pp*-?I)p?7a%_gYF zF@`CF0b;zpSII=DzMn8W(ZA#`0mG~HEaC(%WVL{KP9@#Ub8m)qvbS8K!}TGIEIDuC z5L5YIwe`|3ya2|(SAXlQnDEii5`{vps0A>3+Asti@CvXCKSnW~dYfGS#Hm`c)Y-jx zGl(C&6SdE~ZDael!#y5rXBWT^fleGM8BLN}IIVH=Bw<#9@~@lB7CSy1#^AA|3r=OW z9_aGZ>d~#3xH|^@*RZ7umgdP>E;XZ(h0K4THVcDDRRlFo-$o-P4epOVNuW?DF59@l z|GThm$+vQlj8bVAH$MqOMTK9+FjGfW0yB|Ju(+ z7r<>>zr#15lHap=EI%hyj2oAx#!fz-+%m!G%@>c4kR(!!(sMJE)9XHgd*b&(&=*gO zh_&-Q#QMkG<+T62^gZvNRnVJg-EzZT>~C64(rdQwLdWjK=yu0fcEQcaaf68B$I`dg zI5z$8_V^XDr)lK$%XeQ~1H*o5%-3S=$1=*p_WIFP@QSwG=&lZyi;$0HQFjEcJ;qO+CFmeuN{lp2n^C0I^C_EA29E*EcKvck9dW z6o`QotHc$+rZSXSna^UF8sntr-CFgP?}&jwg0M}1`*#h%S*4Hh-1XC5_zbO% zHBJ-vbfqeHqo;XYEgjzk)WObofi3K=j10%xBD!4Ytaf|sCH);Ab!XxEZ}Dz9TErHX zF%7q~GT~GmqTX~E8Iy24@(th|!Q#csR|AF&8a2jqz66Y(EmEHc`p}q6w!mRC zQtpIZAUgJtFRHS4s^v#gxe6*Ac}!+wn{E8p#6N-&Oo$EN-2np*mjk%Maw9_O@GpHV zT==WW(=jOG1hy`@YqswPDHUz`l%PS4f6lCtThhh^C|00&276Hl-@ixb4DR^j~DV z1SP)wIlW-DK1MM6X^~7ZHfdM`*ydIs!A=8*^k}_?dTV>zaE5g8lxNQ0X`1F8p@ifP zg?^d)B)zxAXEvIN5avnUlESXIuZ6Nwls=C23plfb%{Wd?z5UUZF~K*^r+R<8R4BSZ z8p;qZS6`BOsp_f*_&V3%QwpI%bD3$6ZsM*UJmeu#DLV|^Tf}9XHlWx28e)PEX5bEg zqq@oyKBhJIPIe!W)TmgS+`E%4r<&flyIY`Y^^~^pQ>#iAZDp_xZCqf{4i!(eRC5$C z+wgQxkp6r5G-TE#eZmIU%Hn^&opwSuQ!TTdv#JjqCB_)Yr|k-!lzUpHR;*>3C5)Uw zh6pPswi`?JXvec?E0@)Zw*GCn6xPc3hq&{p@}p4fEppSIS=)~2A8kyN zn7+%?a?MEG)N`vaS$qw+M+x@G9q~k{7_%PRry2a-v*0GuE65+Q1 zJP4!UnZp&zPB(xe5MFciZg=ZQet9Z0*iyhlymS!1GV)MN7Z$*5P^+g$ z5Xh4aT~BD55QoXqAy#g^lmbb_JmKCZg3iMP1r3Qy=D!7~8A=t4WgoT2G#-g~F zy2I^xQ?J>L+g>wdo1=&@)eIT=gPkLd9@6nNiU-kU&sLx~0~-J7J%IX<4gUt^on1$| z8IXn7VPSU%sUQ%_;B-pP;l$bN=4qe_HpxzHpQaJHH3!b{k+{sp4RlTzeX3#jsrfrt zp9@(jTKmQ>uyR(Yc;gI;)_53ZQI7?g0HU-ErV|Bje=!F#x;3g)WJqX6f>sEkOIJXm z(fsfnGgor2!jVYFRg=$JG}}I0pBB722cU>pHXk*X`mn}Vg6(@j z4@M87bno@bW8?hrO2-c3d6*iAOh-DiDW(Z|1iZ=>`FAEDW9@Jk8i`(j{Q8O})8!a& zTO%E^2{&@NNax8)(uWcprC)}VF(Ty{%~HDH?Mwn_+@;dZ{!9Wr36?zQYOiQVJ+Meg z7}5wKx;$OL`n1xwqPmJVI!9O#U~Cj<^gVGj7^gzXLA438UL*#nFZZ~4Vw{9`KZx#_ zBaSbV89ipLw9>}uHgXg05hn_~ENx|Q^al$O>Qg0$;Y7b>_k!^1#$?C# ze(5WYv4&~^Ra6njuVe@aB-$RAn|_Y*A;hC3Ssj4W2RK8({bdFeo_lP6a1}GoS3tao z6efDAPENT!1qpU+JlH6@Mr({wyvD$M<#B?A*wNl^Yu@QuZ<2gs-!(ad>t z4?f)4y#Hus_8);ecG#E+i)>UZN8j?X)7nHi_Uik}oV`!x4qO zALXZ8zzMFZgv~>318%P~Y*gbuJ{@srI=BivSYtoFaN;xec{E1`u1=Jtu~;ITObMb} zXvaWB_Lv~Xn){#+astcw?*zQfI|W~r9dQS0b?i9dIsNf=C#3%tJy5lWick8%IZ0Qj zn=rCgHBlHn+PJ0xZTSnhSS;vdL?(o~Qs+amIz#x1Gj}fWJ$B)|?=dz*dK3)#arSJ5 zm^+dEG+UboICBRbr!W);pQ;*%a`wJbm^*9$68`hd;V4c&{6=9xkpemL?g|rAcgu;i z#KF8g%8`xgFLrarBIj)cznKnTP9z!*=j9QH-u_h6(M-FUF=XPIQz`b46K4(yyrk$uSV1{ zR1w|0xLc|%hs}n&S;}a#Ui0QV(X3(t=wgO@EY`MVV-qv&R=8s)u@eoUI{NubqV_0$ z;zMnP9fP&r1=U@nN#ujozRz9q&BXF6~ zNIA;p{M6*ABd~A!2|LjTKl*VV_dO5o5e1RjdzLOPD9EwDoQ!tJ4)-xBDs87qB0&|p z!h&xFj4F$e(e#huJJQm=>a?gI20iIT9L|oSkx&F$5N$E^8|vs`ePL#8enq6wo^yAx zg(4GXw|royp93$PnA4QUKfVqpyYHIvT36|W-UnCDiBDz`L=b8% zlXXSfu$Be#f1(Mj084`uGAF&h0PY1op7XMMcl5?(%FbGEgDXP-_8Xd&3I;azi>H`a zxJGBWA#wNvk%$vGy-g{F@!j)j+ns;!6Ht5>ytSEA@ECqdBn=f=V~%xKmzHrrjv|Rl zl0pz`s9UXTs^Nsx4iB2Si3+hlILd +local customCallouts = {} + +local fa = require("fa") + +---Converts a valid CSS color string or hexadecimal to RGBA format +---@param color string The color in hex (#RRGGBB) or named format +---@param alpha number The alpha value between 0 and 1 +---@return string rgba The color in rgba() or rgb(from color) format +local function colorToRgba(color, alpha) + if color:sub(1,1) == "#" then + local r = tonumber(color:sub(2,3), 16) + local g = tonumber(color:sub(4,5), 16) + local b = tonumber(color:sub(6,7), 16) + return string.format("rgba(%d, %d, %d, %.2f)", r, g, b, alpha) + else + -- For named colors, we use the functional notation of rgba() + return string.format("rgb(from %s r g b / %.0f%%)", color, alpha * 100) + end +end + +---CSS named color to hex lookup (without #, uppercase) +---@type table +local cssNamedColors = { + -- CSS Level 1 + black = "000000", silver = "C0C0C0", gray = "808080", white = "FFFFFF", + maroon = "800000", red = "FF0000", purple = "800080", fuchsia = "FF00FF", + green = "008000", lime = "00FF00", olive = "808000", yellow = "FFFF00", + navy = "000080", blue = "0000FF", teal = "008080", aqua = "00FFFF", + -- Extended common colors + orange = "FFA500", pink = "FFC0CB", brown = "A52A2A", + cyan = "00FFFF", grey = "808080", + crimson = "DC143C", coral = "FF7F50", gold = "FFD700", + indigo = "4B0082", violet = "EE82EE", + steelblue = "4682B4", dodgerblue = "1E90FF", + forestgreen = "228B22", tomato = "FF6347", + darkorange = "FF8C00", firebrick = "B22222", + slategray = "708090", darkred = "8B0000", +} + +---Converts a color string to a 6-digit uppercase hex value (without #) +---@param color string The color in hex (#RRGGBB) or named CSS format +---@return string|nil hex The 6-digit hex string, or nil if conversion fails +local function colorToHex(color) + if color:sub(1, 1) == "#" then + return color:sub(2):upper() + end + return cssNamedColors[color:lower()] +end + +---Adds HTML dependency for bundled FontAwesome 7 Free Solid font +local function ensureFontAwesomeDeps() + quarto.doc.add_html_dependency({ + name = "fontawesome-7-free", + version = "7.2.0", + stylesheets = {"assets/css/fontawesome.css"}, + resources = { + { name = "fa-solid-900.woff2", path = "assets/webfonts/fa-solid-900.woff2" } + } + }) +end + +---Checks if a string represents a Font Awesome icon +---@param icon string|nil The icon string to check +---@return boolean is_fa True if the string starts with "fa-" +local function isFontAwesomeIcon(icon) + return icon ~= nil and icon:sub(1, 3) == "fa-" +end + +---Generates CSS for all defined custom callouts +---@param isRevealJS boolean Whether the output format is RevealJS +---@return string css The generated CSS rules +local function generateCustomCSS(isRevealJS) + local css = "" + -- RevealJS uses `.reveal div.callout...` selectors with higher specificity, + -- so we need to match that prefix. It also uses `.callout-title` instead + -- of `.callout-header` for the header element. + local prefix = isRevealJS and ".reveal " or "" + local headerClass = isRevealJS and ".callout-title" or ".callout-header" + + -- Translate YAML callout information for custom callouts + for type, callout in pairs(customCallouts) do + if callout.color then + local color = pandoc.utils.stringify(callout.color) + + -- Base color + css = css .. string.format("%sdiv.callout-%s.callout {\n", prefix, type) + css = css .. string.format(" border-left-color: %s;\n", color) + css = css .. "}\n" + + -- Header background + css = css .. string.format("%sdiv.callout-%s.callout-style-default %s {\n", prefix, type, headerClass) + css = css .. string.format(" background-color: %s;\n", colorToRgba(color, 0.13)) + css = css .. "}\n" + + -- Collapse Icon (not supported in RevealJS) + if not isRevealJS then + css = css .. string.format("div.callout-%s .callout-toggle::before {", type) + css = css .. " background-image: url('data:image/svg+xml,');" + css = css .. "}\n" + end + + -- Icon Styling + css = css .. string.format("%sdiv.callout-%s.callout-style-default .callout-icon::before, %sdiv.callout-%s.callout-titled .callout-icon::before {\n", prefix, type, prefix, type) + + if callout.icon_symbol then + local icon_symbol_str = pandoc.utils.stringify(callout.icon_symbol) + if isFontAwesomeIcon(icon_symbol_str) then + -- Font Awesome icon + css = css .. " font-family: 'Font Awesome 7 Free';\n" + css = css .. " font-weight: 900;\n" + css = css .. " font-style: normal;\n" + css = css .. string.format(" content: '%s' !important;\n", fa.fa_unicode(icon_symbol_str)) + else + -- Custom icon symbol + css = css .. string.format(" content: '%s';\n", icon_symbol_str) + end + css = css .. " background-image: none;\n" + else + -- The fallback case + local escapedColor = color:gsub("#", "%%23") -- Escape # in hex colors + css = css .. string.format(" background-image: url('data:image/svg+xml,');\n", escapedColor) + end + + css = css .. "}\n" + + end + end + return css +end + + +---Generates LaTeX \definecolor commands for PDF output +---@return string latex The LaTeX color definitions +local function generatePdfStyles() + local latex = "" + for type, callout in pairs(customCallouts) do + if callout.color then + local hex = colorToHex(pandoc.utils.stringify(callout.color)) + if hex then + latex = latex .. string.format( + "\\definecolor{quarto-callout-%s-color}{HTML}{%s}\n", type, hex + ) + latex = latex .. string.format( + "\\definecolor{quarto-callout-%s-color-frame}{HTML}{%s}\n", type, hex + ) + end + end + end + return latex +end + +---Generates Typst color definitions for Typst output +---@return string typst The Typst style definitions +local function generateTypstStyles() + local typst = "" + for type, callout in pairs(customCallouts) do + if callout.color then + local hex = colorToHex(pandoc.utils.stringify(callout.color)) + if hex then + typst = typst .. string.format( + '#let quarto-callout-%s-color = rgb("#%s")\n', type, hex + ) + typst = typst .. string.format( + '#let quarto-callout-%s-color-frame = rgb("#%s")\n', type, hex + ) + end + end + end + return typst +end + +---Parses custom callout definitions from document metadata +---@param meta pandoc.Meta The document metadata +local function parseCustomCallouts(meta) + if not meta['custom-callout'] then return end + + for k, v in pairs(meta['custom-callout']) do + if type(v) == "table" then + customCallouts[k] = { + type = tostring(k), + title = v.title or k:gsub("^%l", string.upper), + icon = v.icon == 'true' or nil, + appearance = v.appearance or nil, + collapse = v.collapse or nil, + icon_symbol = v['icon-symbol'] or nil, + color = v.color or nil, + background_color = v['background-color'] or nil + } + end + end + + + -- Detect format and inject appropriate styles + local isRevealJS = quarto.doc.is_format("revealjs") + if quarto.doc.is_format("html") or isRevealJS then + local customCSS = generateCustomCSS(isRevealJS) + if customCSS ~= "" then + quarto.doc.include_text('in-header', '') + end + -- Load FontAwesome font dependency (HTML only) + for _, callout in pairs(customCallouts) do + if callout.icon_symbol and isFontAwesomeIcon(pandoc.utils.stringify(callout.icon_symbol)) then + ensureFontAwesomeDeps() + break + end + end + elseif quarto.doc.is_format("pdf") then + local pdfStyles = generatePdfStyles() + if pdfStyles ~= "" then + quarto.doc.include_text('in-header', pdfStyles) + end + elseif quarto.doc.is_format("typst") then + local typstStyles = generateTypstStyles() + if typstStyles ~= "" then + quarto.doc.include_text('in-header', typstStyles) + end + end + +end + + +---Converts a div to a custom callout if it matches a defined custom callout +---@param div pandoc.Div The div to potentially convert +---@return pandoc.Div|quarto.Callout converted The converted callout or original div +local function convertToCustomCallout(div) + -- Check if the div has classes + for _, class in ipairs(div.classes) do + + -- Check if the class matches a custom callout + local callout = customCallouts[class] + + if callout then + -- Use the default title if not provided + local title = callout.title + + -- Check to see if the title is specified in the div content + if div.content[1] ~= nil and div.content[1].t == "Header" then + title = div.content[1] + div.content:remove(1) + end + + -- Create a new Callout with the custom callout parameters + local calloutParams = { + type = callout.type, + content = div.content, + title = div.attributes.title or title, + icon = div.attributes.icon or callout.icon, + appearance = div.attributes.appearance or callout.appearance, + collapse = div.attributes.collapse or callout.collapse + } + + return quarto.Callout(calloutParams) + end + end + + + return div +end + +---Walks the Pandoc document and processes divs to +---convert to custom callouts +---@class pandoc.Doc +---@field blocks pandoc.Blocks +---@param doc pandoc.Doc The Pandoc document +---@return pandoc.Doc doc The processed document +local function customCalloutFilter(doc) + + -- Walk the AST and process divs + doc.blocks = doc.blocks:walk({ + Div = convertToCustomCallout + }) + + -- Return the modified document + return doc +end + +-- Return the Pandoc filter +return { + ---@type fun(meta: pandoc.Meta) + Meta = parseCustomCallouts, + ---@type fun(doc: pandoc.Doc): pandoc.Doc + Pandoc = customCalloutFilter +} \ No newline at end of file diff --git a/quarto/_extensions/coatless-quarto/custom-callout/fa.lua b/quarto/_extensions/coatless-quarto/custom-callout/fa.lua new file mode 100644 index 0000000..7dd9757 --- /dev/null +++ b/quarto/_extensions/coatless-quarto/custom-callout/fa.lua @@ -0,0 +1,2594 @@ +local fa_icons = { + ["fa-0"] = "\\30", + ["fa-1"] = "\\31", + ["fa-2"] = "\\32", + ["fa-3"] = "\\33", + ["fa-4"] = "\\34", + ["fa-5"] = "\\35", + ["fa-6"] = "\\36", + ["fa-7"] = "\\37", + ["fa-8"] = "\\38", + ["fa-9"] = "\\39", + ["fa-exclamation"] = "\\21", + ["fa-hashtag"] = "\\23", + ["fa-dollar-sign"] = "\\24", + ["fa-dollar"] = "\\24", + ["fa-usd"] = "\\24", + ["fa-percent"] = "\\25", + ["fa-percentage"] = "\\25", + ["fa-asterisk"] = "\\2a", + ["fa-plus"] = "\\2b", + ["fa-add"] = "\\2b", + ["fa-less-than"] = "\\3c", + ["fa-equals"] = "\\3d", + ["fa-greater-than"] = "\\3e", + ["fa-question"] = "\\3f", + ["fa-at"] = "\\40", + ["fa-a"] = "\\41", + ["fa-b"] = "\\42", + ["fa-c"] = "\\43", + ["fa-d"] = "\\44", + ["fa-e"] = "\\45", + ["fa-f"] = "\\46", + ["fa-g"] = "\\47", + ["fa-h"] = "\\48", + ["fa-i"] = "\\49", + ["fa-j"] = "\\4a", + ["fa-k"] = "\\4b", + ["fa-l"] = "\\4c", + ["fa-m"] = "\\4d", + ["fa-n"] = "\\4e", + ["fa-o"] = "\\4f", + ["fa-p"] = "\\50", + ["fa-q"] = "\\51", + ["fa-r"] = "\\52", + ["fa-s"] = "\\53", + ["fa-t"] = "\\54", + ["fa-u"] = "\\55", + ["fa-v"] = "\\56", + ["fa-w"] = "\\57", + ["fa-x"] = "\\58", + ["fa-y"] = "\\59", + ["fa-z"] = "\\5a", + ["fa-faucet"] = "\\e005", + ["fa-faucet-drip"] = "\\e006", + ["fa-house-chimney-window"] = "\\e00d", + ["fa-house-signal"] = "\\e012", + ["fa-temperature-arrow-down"] = "\\e03f", + ["fa-temperature-down"] = "\\e03f", + ["fa-temperature-arrow-up"] = "\\e040", + ["fa-temperature-up"] = "\\e040", + ["fa-trailer"] = "\\e041", + ["fa-bacteria"] = "\\e059", + ["fa-bacterium"] = "\\e05a", + ["fa-box-tissue"] = "\\e05b", + ["fa-hand-holding-medical"] = "\\e05c", + ["fa-hand-sparkles"] = "\\e05d", + ["fa-hands-bubbles"] = "\\e05e", + ["fa-hands-wash"] = "\\e05e", + ["fa-handshake-slash"] = "\\e060", + ["fa-handshake-alt-slash"] = "\\e060", + ["fa-handshake-simple-slash"] = "\\e060", + ["fa-head-side-cough"] = "\\e061", + ["fa-head-side-cough-slash"] = "\\e062", + ["fa-head-side-mask"] = "\\e063", + ["fa-head-side-virus"] = "\\e064", + ["fa-house-chimney-user"] = "\\e065", + ["fa-house-laptop"] = "\\e066", + ["fa-laptop-house"] = "\\e066", + ["fa-lungs-virus"] = "\\e067", + ["fa-people-arrows"] = "\\e068", + ["fa-people-arrows-left-right"] = "\\e068", + ["fa-plane-slash"] = "\\e069", + ["fa-pump-medical"] = "\\e06a", + ["fa-pump-soap"] = "\\e06b", + ["fa-shield-virus"] = "\\e06c", + ["fa-sink"] = "\\e06d", + ["fa-soap"] = "\\e06e", + ["fa-stopwatch-20"] = "\\e06f", + ["fa-shop-slash"] = "\\e070", + ["fa-store-alt-slash"] = "\\e070", + ["fa-store-slash"] = "\\e071", + ["fa-toilet-paper-slash"] = "\\e072", + ["fa-users-slash"] = "\\e073", + ["fa-virus"] = "\\e074", + ["fa-virus-slash"] = "\\e075", + ["fa-viruses"] = "\\e076", + ["fa-vest"] = "\\e085", + ["fa-vest-patches"] = "\\e086", + ["fa-arrow-trend-down"] = "\\e097", + ["fa-arrow-trend-up"] = "\\e098", + ["fa-arrow-up-from-bracket"] = "\\e09a", + ["fa-austral-sign"] = "\\e0a9", + ["fa-baht-sign"] = "\\e0ac", + ["fa-bitcoin-sign"] = "\\e0b4", + ["fa-bolt-lightning"] = "\\e0b7", + ["fa-book-bookmark"] = "\\e0bb", + ["fa-camera-rotate"] = "\\e0d8", + ["fa-cedi-sign"] = "\\e0df", + ["fa-chart-column"] = "\\e0e3", + ["fa-chart-gantt"] = "\\e0e4", + ["fa-clapperboard"] = "\\e131", + ["fa-closed-captioning-slash"] = "\\e135", + ["fa-clover"] = "\\e139", + ["fa-code-compare"] = "\\e13a", + ["fa-code-fork"] = "\\e13b", + ["fa-code-pull-request"] = "\\e13c", + ["fa-colon-sign"] = "\\e140", + ["fa-cruzeiro-sign"] = "\\e152", + ["fa-display"] = "\\e163", + ["fa-dong-sign"] = "\\e169", + ["fa-elevator"] = "\\e16d", + ["fa-filter-circle-xmark"] = "\\e17b", + ["fa-florin-sign"] = "\\e184", + ["fa-folder-closed"] = "\\e185", + ["fa-franc-sign"] = "\\e18f", + ["fa-guarani-sign"] = "\\e19a", + ["fa-gun"] = "\\e19b", + ["fa-hands-clapping"] = "\\e1a8", + ["fa-house-user"] = "\\e1b0", + ["fa-home-user"] = "\\e1b0", + ["fa-indian-rupee-sign"] = "\\e1bc", + ["fa-indian-rupee"] = "\\e1bc", + ["fa-inr"] = "\\e1bc", + ["fa-kip-sign"] = "\\e1c4", + ["fa-lari-sign"] = "\\e1c8", + ["fa-litecoin-sign"] = "\\e1d3", + ["fa-manat-sign"] = "\\e1d5", + ["fa-mask-face"] = "\\e1d7", + ["fa-mill-sign"] = "\\e1ed", + ["fa-money-bills"] = "\\e1f3", + ["fa-naira-sign"] = "\\e1f6", + ["fa-notdef"] = "\\e1fe", + ["fa-panorama"] = "\\e209", + ["fa-peseta-sign"] = "\\e221", + ["fa-peso-sign"] = "\\e222", + ["fa-plane-up"] = "\\e22d", + ["fa-rupiah-sign"] = "\\e23d", + ["fa-stairs"] = "\\e289", + ["fa-timeline"] = "\\e29c", + ["fa-truck-front"] = "\\e2b7", + ["fa-turkish-lira-sign"] = "\\e2bb", + ["fa-try"] = "\\e2bb", + ["fa-turkish-lira"] = "\\e2bb", + ["fa-vault"] = "\\e2c5", + ["fa-wand-magic-sparkles"] = "\\e2ca", + ["fa-magic-wand-sparkles"] = "\\e2ca", + ["fa-wheat-awn"] = "\\e2cd", + ["fa-wheat-alt"] = "\\e2cd", + ["fa-wheelchair-move"] = "\\e2ce", + ["fa-wheelchair-alt"] = "\\e2ce", + ["fa-bangladeshi-taka-sign"] = "\\e2e6", + ["fa-bowl-rice"] = "\\e2eb", + ["fa-person-pregnant"] = "\\e31e", + ["fa-house-chimney"] = "\\e3af", + ["fa-home-lg"] = "\\e3af", + ["fa-house-crack"] = "\\e3b1", + ["fa-house-medical"] = "\\e3b2", + ["fa-cent-sign"] = "\\e3f5", + ["fa-plus-minus"] = "\\e43c", + ["fa-sailboat"] = "\\e445", + ["fa-section"] = "\\e447", + ["fa-shrimp"] = "\\e448", + ["fa-brazilian-real-sign"] = "\\e46c", + ["fa-chart-simple"] = "\\e473", + ["fa-diagram-next"] = "\\e476", + ["fa-diagram-predecessor"] = "\\e477", + ["fa-diagram-successor"] = "\\e47a", + ["fa-earth-oceania"] = "\\e47b", + ["fa-globe-oceania"] = "\\e47b", + ["fa-bug-slash"] = "\\e490", + ["fa-file-circle-plus"] = "\\e494", + ["fa-shop-lock"] = "\\e4a5", + ["fa-virus-covid"] = "\\e4a8", + ["fa-virus-covid-slash"] = "\\e4a9", + ["fa-anchor-circle-check"] = "\\e4aa", + ["fa-anchor-circle-exclamation"] = "\\e4ab", + ["fa-anchor-circle-xmark"] = "\\e4ac", + ["fa-anchor-lock"] = "\\e4ad", + ["fa-arrow-down-up-across-line"] = "\\e4af", + ["fa-arrow-down-up-lock"] = "\\e4b0", + ["fa-arrow-right-to-city"] = "\\e4b3", + ["fa-arrow-up-from-ground-water"] = "\\e4b5", + ["fa-arrow-up-from-water-pump"] = "\\e4b6", + ["fa-arrow-up-right-dots"] = "\\e4b7", + ["fa-arrows-down-to-line"] = "\\e4b8", + ["fa-arrows-down-to-people"] = "\\e4b9", + ["fa-arrows-left-right-to-line"] = "\\e4ba", + ["fa-arrows-spin"] = "\\e4bb", + ["fa-arrows-split-up-and-left"] = "\\e4bc", + ["fa-arrows-to-circle"] = "\\e4bd", + ["fa-arrows-to-dot"] = "\\e4be", + ["fa-arrows-to-eye"] = "\\e4bf", + ["fa-arrows-turn-right"] = "\\e4c0", + ["fa-arrows-turn-to-dots"] = "\\e4c1", + ["fa-arrows-up-to-line"] = "\\e4c2", + ["fa-bore-hole"] = "\\e4c3", + ["fa-bottle-droplet"] = "\\e4c4", + ["fa-bottle-water"] = "\\e4c5", + ["fa-bowl-food"] = "\\e4c6", + ["fa-boxes-packing"] = "\\e4c7", + ["fa-bridge"] = "\\e4c8", + ["fa-bridge-circle-check"] = "\\e4c9", + ["fa-bridge-circle-exclamation"] = "\\e4ca", + ["fa-bridge-circle-xmark"] = "\\e4cb", + ["fa-bridge-lock"] = "\\e4cc", + ["fa-bridge-water"] = "\\e4ce", + ["fa-bucket"] = "\\e4cf", + ["fa-bugs"] = "\\e4d0", + ["fa-building-circle-arrow-right"] = "\\e4d1", + ["fa-building-circle-check"] = "\\e4d2", + ["fa-building-circle-exclamation"] = "\\e4d3", + ["fa-building-circle-xmark"] = "\\e4d4", + ["fa-building-flag"] = "\\e4d5", + ["fa-building-lock"] = "\\e4d6", + ["fa-building-ngo"] = "\\e4d7", + ["fa-building-shield"] = "\\e4d8", + ["fa-building-un"] = "\\e4d9", + ["fa-building-user"] = "\\e4da", + ["fa-building-wheat"] = "\\e4db", + ["fa-burst"] = "\\e4dc", + ["fa-car-on"] = "\\e4dd", + ["fa-car-tunnel"] = "\\e4de", + ["fa-child-combatant"] = "\\e4e0", + ["fa-child-rifle"] = "\\e4e0", + ["fa-children"] = "\\e4e1", + ["fa-circle-nodes"] = "\\e4e2", + ["fa-clipboard-question"] = "\\e4e3", + ["fa-cloud-showers-water"] = "\\e4e4", + ["fa-computer"] = "\\e4e5", + ["fa-cubes-stacked"] = "\\e4e6", + ["fa-envelope-circle-check"] = "\\e4e8", + ["fa-explosion"] = "\\e4e9", + ["fa-ferry"] = "\\e4ea", + ["fa-file-circle-exclamation"] = "\\e4eb", + ["fa-file-circle-minus"] = "\\e4ed", + ["fa-file-circle-question"] = "\\e4ef", + ["fa-file-shield"] = "\\e4f0", + ["fa-fire-burner"] = "\\e4f1", + ["fa-fish-fins"] = "\\e4f2", + ["fa-flask-vial"] = "\\e4f3", + ["fa-glass-water"] = "\\e4f4", + ["fa-glass-water-droplet"] = "\\e4f5", + ["fa-group-arrows-rotate"] = "\\e4f6", + ["fa-hand-holding-hand"] = "\\e4f7", + ["fa-handcuffs"] = "\\e4f8", + ["fa-hands-bound"] = "\\e4f9", + ["fa-hands-holding-child"] = "\\e4fa", + ["fa-hands-holding-circle"] = "\\e4fb", + ["fa-heart-circle-bolt"] = "\\e4fc", + ["fa-heart-circle-check"] = "\\e4fd", + ["fa-heart-circle-exclamation"] = "\\e4fe", + ["fa-heart-circle-minus"] = "\\e4ff", + ["fa-heart-circle-plus"] = "\\e500", + ["fa-heart-circle-xmark"] = "\\e501", + ["fa-helicopter-symbol"] = "\\e502", + ["fa-helmet-un"] = "\\e503", + ["fa-hill-avalanche"] = "\\e507", + ["fa-hill-rockslide"] = "\\e508", + ["fa-house-circle-check"] = "\\e509", + ["fa-house-circle-exclamation"] = "\\e50a", + ["fa-house-circle-xmark"] = "\\e50b", + ["fa-house-fire"] = "\\e50c", + ["fa-house-flag"] = "\\e50d", + ["fa-house-flood-water"] = "\\e50e", + ["fa-house-flood-water-circle-arrow-right"] = "\\e50f", + ["fa-house-lock"] = "\\e510", + ["fa-house-medical-circle-check"] = "\\e511", + ["fa-house-medical-circle-exclamation"] = "\\e512", + ["fa-house-medical-circle-xmark"] = "\\e513", + ["fa-house-medical-flag"] = "\\e514", + ["fa-house-tsunami"] = "\\e515", + ["fa-jar"] = "\\e516", + ["fa-jar-wheat"] = "\\e517", + ["fa-jet-fighter-up"] = "\\e518", + ["fa-jug-detergent"] = "\\e519", + ["fa-kitchen-set"] = "\\e51a", + ["fa-land-mine-on"] = "\\e51b", + ["fa-landmark-flag"] = "\\e51c", + ["fa-laptop-file"] = "\\e51d", + ["fa-lines-leaning"] = "\\e51e", + ["fa-location-pin-lock"] = "\\e51f", + ["fa-locust"] = "\\e520", + ["fa-magnifying-glass-arrow-right"] = "\\e521", + ["fa-magnifying-glass-chart"] = "\\e522", + ["fa-mars-and-venus-burst"] = "\\e523", + ["fa-mask-ventilator"] = "\\e524", + ["fa-mattress-pillow"] = "\\e525", + ["fa-mobile-retro"] = "\\e527", + ["fa-money-bill-transfer"] = "\\e528", + ["fa-money-bill-trend-up"] = "\\e529", + ["fa-money-bill-wheat"] = "\\e52a", + ["fa-mosquito"] = "\\e52b", + ["fa-mosquito-net"] = "\\e52c", + ["fa-mound"] = "\\e52d", + ["fa-mountain-city"] = "\\e52e", + ["fa-mountain-sun"] = "\\e52f", + ["fa-oil-well"] = "\\e532", + ["fa-people-group"] = "\\e533", + ["fa-people-line"] = "\\e534", + ["fa-people-pulling"] = "\\e535", + ["fa-people-robbery"] = "\\e536", + ["fa-people-roof"] = "\\e537", + ["fa-person-arrow-down-to-line"] = "\\e538", + ["fa-person-arrow-up-from-line"] = "\\e539", + ["fa-person-breastfeeding"] = "\\e53a", + ["fa-person-burst"] = "\\e53b", + ["fa-person-cane"] = "\\e53c", + ["fa-person-chalkboard"] = "\\e53d", + ["fa-person-circle-check"] = "\\e53e", + ["fa-person-circle-exclamation"] = "\\e53f", + ["fa-person-circle-minus"] = "\\e540", + ["fa-person-circle-plus"] = "\\e541", + ["fa-person-circle-question"] = "\\e542", + ["fa-person-circle-xmark"] = "\\e543", + ["fa-person-dress-burst"] = "\\e544", + ["fa-person-drowning"] = "\\e545", + ["fa-person-falling"] = "\\e546", + ["fa-person-falling-burst"] = "\\e547", + ["fa-person-half-dress"] = "\\e548", + ["fa-person-harassing"] = "\\e549", + ["fa-person-military-pointing"] = "\\e54a", + ["fa-person-military-rifle"] = "\\e54b", + ["fa-person-military-to-person"] = "\\e54c", + ["fa-person-rays"] = "\\e54d", + ["fa-person-rifle"] = "\\e54e", + ["fa-person-shelter"] = "\\e54f", + ["fa-person-walking-arrow-loop-left"] = "\\e551", + ["fa-person-walking-arrow-right"] = "\\e552", + ["fa-person-walking-dashed-line-arrow-right"] = "\\e553", + ["fa-person-walking-luggage"] = "\\e554", + ["fa-plane-circle-check"] = "\\e555", + ["fa-plane-circle-exclamation"] = "\\e556", + ["fa-plane-circle-xmark"] = "\\e557", + ["fa-plane-lock"] = "\\e558", + ["fa-plate-wheat"] = "\\e55a", + ["fa-plug-circle-bolt"] = "\\e55b", + ["fa-plug-circle-check"] = "\\e55c", + ["fa-plug-circle-exclamation"] = "\\e55d", + ["fa-plug-circle-minus"] = "\\e55e", + ["fa-plug-circle-plus"] = "\\e55f", + ["fa-plug-circle-xmark"] = "\\e560", + ["fa-ranking-star"] = "\\e561", + ["fa-road-barrier"] = "\\e562", + ["fa-road-bridge"] = "\\e563", + ["fa-road-circle-check"] = "\\e564", + ["fa-road-circle-exclamation"] = "\\e565", + ["fa-road-circle-xmark"] = "\\e566", + ["fa-road-lock"] = "\\e567", + ["fa-road-spikes"] = "\\e568", + ["fa-rug"] = "\\e569", + ["fa-sack-xmark"] = "\\e56a", + ["fa-school-circle-check"] = "\\e56b", + ["fa-school-circle-exclamation"] = "\\e56c", + ["fa-school-circle-xmark"] = "\\e56d", + ["fa-school-flag"] = "\\e56e", + ["fa-school-lock"] = "\\e56f", + ["fa-sheet-plastic"] = "\\e571", + ["fa-shield-cat"] = "\\e572", + ["fa-shield-dog"] = "\\e573", + ["fa-shield-heart"] = "\\e574", + ["fa-square-nfi"] = "\\e576", + ["fa-square-person-confined"] = "\\e577", + ["fa-square-virus"] = "\\e578", + ["fa-staff-snake"] = "\\e579", + ["fa-rod-asclepius"] = "\\e579", + ["fa-rod-snake"] = "\\e579", + ["fa-staff-aesculapius"] = "\\e579", + ["fa-sun-plant-wilt"] = "\\e57a", + ["fa-tarp"] = "\\e57b", + ["fa-tarp-droplet"] = "\\e57c", + ["fa-tent"] = "\\e57d", + ["fa-tent-arrow-down-to-line"] = "\\e57e", + ["fa-tent-arrow-left-right"] = "\\e57f", + ["fa-tent-arrow-turn-left"] = "\\e580", + ["fa-tent-arrows-down"] = "\\e581", + ["fa-tents"] = "\\e582", + ["fa-toilet-portable"] = "\\e583", + ["fa-toilets-portable"] = "\\e584", + ["fa-tower-cell"] = "\\e585", + ["fa-tower-observation"] = "\\e586", + ["fa-tree-city"] = "\\e587", + ["fa-trowel"] = "\\e589", + ["fa-trowel-bricks"] = "\\e58a", + ["fa-truck-arrow-right"] = "\\e58b", + ["fa-truck-droplet"] = "\\e58c", + ["fa-truck-field"] = "\\e58d", + ["fa-truck-field-un"] = "\\e58e", + ["fa-truck-plane"] = "\\e58f", + ["fa-users-between-lines"] = "\\e591", + ["fa-users-line"] = "\\e592", + ["fa-users-rays"] = "\\e593", + ["fa-users-rectangle"] = "\\e594", + ["fa-users-viewfinder"] = "\\e595", + ["fa-vial-circle-check"] = "\\e596", + ["fa-vial-virus"] = "\\e597", + ["fa-wheat-awn-circle-exclamation"] = "\\e598", + ["fa-worm"] = "\\e599", + ["fa-xmarks-lines"] = "\\e59a", + ["fa-child-dress"] = "\\e59c", + ["fa-child-reaching"] = "\\e59d", + ["fa-file-circle-check"] = "\\e5a0", + ["fa-file-circle-xmark"] = "\\e5a1", + ["fa-person-through-window"] = "\\e5a9", + ["fa-plant-wilt"] = "\\e5aa", + ["fa-stapler"] = "\\e5af", + ["fa-train-tram"] = "\\e5b4", + ["fa-table-cells-column-lock"] = "\\e678", + ["fa-table-cells-row-lock"] = "\\e67a", + ["fa-web-awesome"] = "\\e682", + ["fa-thumbtack-slash"] = "\\e68f", + ["fa-thumb-tack-slash"] = "\\e68f", + ["fa-table-cells-row-unlock"] = "\\e691", + ["fa-chart-diagram"] = "\\e695", + ["fa-comment-nodes"] = "\\e696", + ["fa-file-fragment"] = "\\e697", + ["fa-file-half-dashed"] = "\\e698", + ["fa-hexagon-nodes"] = "\\e699", + ["fa-hexagon-nodes-bolt"] = "\\e69a", + ["fa-square-binary"] = "\\e69b", + ["fa-pentagon"] = "\\e790", + ["fa-non-binary"] = "\\e807", + ["fa-spiral"] = "\\e80a", + ["fa-picture-in-picture"] = "\\e80b", + ["fa-mobile-vibrate"] = "\\e816", + ["fa-single-quote-left"] = "\\e81b", + ["fa-single-quote-right"] = "\\e81c", + ["fa-bus-side"] = "\\e81d", + ["fa-septagon"] = "\\e820", + ["fa-heptagon"] = "\\e820", + ["fa-aquarius"] = "\\e845", + ["fa-aries"] = "\\e846", + ["fa-cancer"] = "\\e847", + ["fa-capricorn"] = "\\e848", + ["fa-gemini"] = "\\e849", + ["fa-leo"] = "\\e84a", + ["fa-libra"] = "\\e84b", + ["fa-pisces"] = "\\e84c", + ["fa-sagittarius"] = "\\e84d", + ["fa-scorpio"] = "\\e84e", + ["fa-taurus"] = "\\e84f", + ["fa-virgo"] = "\\e850", + ["fa-martini-glass-empty"] = "\\f000", + ["fa-glass-martini"] = "\\f000", + ["fa-music"] = "\\f001", + ["fa-magnifying-glass"] = "\\f002", + ["fa-search"] = "\\f002", + ["fa-heart"] = "\\f004", + ["fa-star"] = "\\f005", + ["fa-user"] = "\\f007", + ["fa-user-alt"] = "\\f007", + ["fa-user-large"] = "\\f007", + ["fa-film"] = "\\f008", + ["fa-film-alt"] = "\\f008", + ["fa-film-simple"] = "\\f008", + ["fa-table-cells-large"] = "\\f009", + ["fa-th-large"] = "\\f009", + ["fa-table-cells"] = "\\f00a", + ["fa-th"] = "\\f00a", + ["fa-table-list"] = "\\f00b", + ["fa-th-list"] = "\\f00b", + ["fa-check"] = "\\f00c", + ["fa-xmark"] = "\\f00d", + ["fa-close"] = "\\f00d", + ["fa-multiply"] = "\\f00d", + ["fa-remove"] = "\\f00d", + ["fa-times"] = "\\f00d", + ["fa-magnifying-glass-plus"] = "\\f00e", + ["fa-search-plus"] = "\\f00e", + ["fa-magnifying-glass-minus"] = "\\f010", + ["fa-search-minus"] = "\\f010", + ["fa-power-off"] = "\\f011", + ["fa-signal"] = "\\f012", + ["fa-signal-5"] = "\\f012", + ["fa-signal-perfect"] = "\\f012", + ["fa-gear"] = "\\f013", + ["fa-cog"] = "\\f013", + ["fa-house"] = "\\f015", + ["fa-home"] = "\\f015", + ["fa-home-alt"] = "\\f015", + ["fa-home-lg-alt"] = "\\f015", + ["fa-clock"] = "\\f017", + ["fa-clock-four"] = "\\f017", + ["fa-road"] = "\\f018", + ["fa-download"] = "\\f019", + ["fa-inbox"] = "\\f01c", + ["fa-arrow-rotate-right"] = "\\f01e", + ["fa-arrow-right-rotate"] = "\\f01e", + ["fa-arrow-rotate-forward"] = "\\f01e", + ["fa-redo"] = "\\f01e", + ["fa-arrows-rotate"] = "\\f021", + ["fa-refresh"] = "\\f021", + ["fa-sync"] = "\\f021", + ["fa-rectangle-list"] = "\\f022", + ["fa-list-alt"] = "\\f022", + ["fa-lock"] = "\\f023", + ["fa-flag"] = "\\f024", + ["fa-headphones"] = "\\f025", + ["fa-headphones-alt"] = "\\f025", + ["fa-headphones-simple"] = "\\f025", + ["fa-volume-off"] = "\\f026", + ["fa-volume-low"] = "\\f027", + ["fa-volume-down"] = "\\f027", + ["fa-volume-high"] = "\\f028", + ["fa-volume-up"] = "\\f028", + ["fa-qrcode"] = "\\f029", + ["fa-barcode"] = "\\f02a", + ["fa-tag"] = "\\f02b", + ["fa-tags"] = "\\f02c", + ["fa-book"] = "\\f02d", + ["fa-bookmark"] = "\\f02e", + ["fa-print"] = "\\f02f", + ["fa-camera"] = "\\f030", + ["fa-camera-alt"] = "\\f030", + ["fa-font"] = "\\f031", + ["fa-bold"] = "\\f032", + ["fa-italic"] = "\\f033", + ["fa-text-height"] = "\\f034", + ["fa-text-width"] = "\\f035", + ["fa-align-left"] = "\\f036", + ["fa-align-center"] = "\\f037", + ["fa-align-right"] = "\\f038", + ["fa-align-justify"] = "\\f039", + ["fa-list"] = "\\f03a", + ["fa-list-squares"] = "\\f03a", + ["fa-outdent"] = "\\f03b", + ["fa-dedent"] = "\\f03b", + ["fa-indent"] = "\\f03c", + ["fa-video"] = "\\f03d", + ["fa-video-camera"] = "\\f03d", + ["fa-image"] = "\\f03e", + ["fa-location-pin"] = "\\f041", + ["fa-map-marker"] = "\\f041", + ["fa-circle-half-stroke"] = "\\f042", + ["fa-adjust"] = "\\f042", + ["fa-droplet"] = "\\f043", + ["fa-tint"] = "\\f043", + ["fa-pen-to-square"] = "\\f044", + ["fa-edit"] = "\\f044", + ["fa-arrows-up-down-left-right"] = "\\f047", + ["fa-arrows"] = "\\f047", + ["fa-backward-step"] = "\\f048", + ["fa-step-backward"] = "\\f048", + ["fa-backward-fast"] = "\\f049", + ["fa-fast-backward"] = "\\f049", + ["fa-backward"] = "\\f04a", + ["fa-play"] = "\\f04b", + ["fa-pause"] = "\\f04c", + ["fa-stop"] = "\\f04d", + ["fa-forward"] = "\\f04e", + ["fa-forward-fast"] = "\\f050", + ["fa-fast-forward"] = "\\f050", + ["fa-forward-step"] = "\\f051", + ["fa-step-forward"] = "\\f051", + ["fa-eject"] = "\\f052", + ["fa-chevron-left"] = "\\f053", + ["fa-chevron-right"] = "\\f054", + ["fa-circle-plus"] = "\\f055", + ["fa-plus-circle"] = "\\f055", + ["fa-circle-minus"] = "\\f056", + ["fa-minus-circle"] = "\\f056", + ["fa-circle-xmark"] = "\\f057", + ["fa-times-circle"] = "\\f057", + ["fa-xmark-circle"] = "\\f057", + ["fa-circle-check"] = "\\f058", + ["fa-check-circle"] = "\\f058", + ["fa-circle-question"] = "\\f059", + ["fa-question-circle"] = "\\f059", + ["fa-circle-info"] = "\\f05a", + ["fa-info-circle"] = "\\f05a", + ["fa-crosshairs"] = "\\f05b", + ["fa-ban"] = "\\f05e", + ["fa-cancel"] = "\\f05e", + ["fa-arrow-left"] = "\\f060", + ["fa-arrow-right"] = "\\f061", + ["fa-arrow-up"] = "\\f062", + ["fa-arrow-down"] = "\\f063", + ["fa-share"] = "\\f064", + ["fa-mail-forward"] = "\\f064", + ["fa-expand"] = "\\f065", + ["fa-compress"] = "\\f066", + ["fa-minus"] = "\\f068", + ["fa-subtract"] = "\\f068", + ["fa-circle-exclamation"] = "\\f06a", + ["fa-exclamation-circle"] = "\\f06a", + ["fa-gift"] = "\\f06b", + ["fa-leaf"] = "\\f06c", + ["fa-fire"] = "\\f06d", + ["fa-eye"] = "\\f06e", + ["fa-eye-slash"] = "\\f070", + ["fa-triangle-exclamation"] = "\\f071", + ["fa-exclamation-triangle"] = "\\f071", + ["fa-warning"] = "\\f071", + ["fa-plane"] = "\\f072", + ["fa-calendar-days"] = "\\f073", + ["fa-calendar-alt"] = "\\f073", + ["fa-shuffle"] = "\\f074", + ["fa-random"] = "\\f074", + ["fa-comment"] = "\\f075", + ["fa-magnet"] = "\\f076", + ["fa-chevron-up"] = "\\f077", + ["fa-chevron-down"] = "\\f078", + ["fa-retweet"] = "\\f079", + ["fa-cart-shopping"] = "\\f07a", + ["fa-shopping-cart"] = "\\f07a", + ["fa-folder"] = "\\f07b", + ["fa-folder-blank"] = "\\f07b", + ["fa-folder-open"] = "\\f07c", + ["fa-arrows-up-down"] = "\\f07d", + ["fa-arrows-v"] = "\\f07d", + ["fa-arrows-left-right"] = "\\f07e", + ["fa-arrows-h"] = "\\f07e", + ["fa-chart-bar"] = "\\f080", + ["fa-bar-chart"] = "\\f080", + ["fa-camera-retro"] = "\\f083", + ["fa-key"] = "\\f084", + ["fa-gears"] = "\\f085", + ["fa-cogs"] = "\\f085", + ["fa-comments"] = "\\f086", + ["fa-star-half"] = "\\f089", + ["fa-arrow-right-from-bracket"] = "\\f08b", + ["fa-sign-out"] = "\\f08b", + ["fa-thumbtack"] = "\\f08d", + ["fa-thumb-tack"] = "\\f08d", + ["fa-arrow-up-right-from-square"] = "\\f08e", + ["fa-external-link"] = "\\f08e", + ["fa-arrow-right-to-bracket"] = "\\f090", + ["fa-sign-in"] = "\\f090", + ["fa-trophy"] = "\\f091", + ["fa-upload"] = "\\f093", + ["fa-lemon"] = "\\f094", + ["fa-phone"] = "\\f095", + ["fa-square-phone"] = "\\f098", + ["fa-phone-square"] = "\\f098", + ["fa-unlock"] = "\\f09c", + ["fa-credit-card"] = "\\f09d", + ["fa-credit-card-alt"] = "\\f09d", + ["fa-rss"] = "\\f09e", + ["fa-feed"] = "\\f09e", + ["fa-hard-drive"] = "\\f0a0", + ["fa-hdd"] = "\\f0a0", + ["fa-bullhorn"] = "\\f0a1", + ["fa-certificate"] = "\\f0a3", + ["fa-hand-point-right"] = "\\f0a4", + ["fa-hand-point-left"] = "\\f0a5", + ["fa-hand-point-up"] = "\\f0a6", + ["fa-hand-point-down"] = "\\f0a7", + ["fa-circle-arrow-left"] = "\\f0a8", + ["fa-arrow-circle-left"] = "\\f0a8", + ["fa-circle-arrow-right"] = "\\f0a9", + ["fa-arrow-circle-right"] = "\\f0a9", + ["fa-circle-arrow-up"] = "\\f0aa", + ["fa-arrow-circle-up"] = "\\f0aa", + ["fa-circle-arrow-down"] = "\\f0ab", + ["fa-arrow-circle-down"] = "\\f0ab", + ["fa-globe"] = "\\f0ac", + ["fa-wrench"] = "\\f0ad", + ["fa-list-check"] = "\\f0ae", + ["fa-tasks"] = "\\f0ae", + ["fa-filter"] = "\\f0b0", + ["fa-briefcase"] = "\\f0b1", + ["fa-up-down-left-right"] = "\\f0b2", + ["fa-arrows-alt"] = "\\f0b2", + ["fa-users"] = "\\f0c0", + ["fa-link"] = "\\f0c1", + ["fa-chain"] = "\\f0c1", + ["fa-cloud"] = "\\f0c2", + ["fa-flask"] = "\\f0c3", + ["fa-scissors"] = "\\f0c4", + ["fa-cut"] = "\\f0c4", + ["fa-copy"] = "\\f0c5", + ["fa-paperclip"] = "\\f0c6", + ["fa-floppy-disk"] = "\\f0c7", + ["fa-save"] = "\\f0c7", + ["fa-square"] = "\\f0c8", + ["fa-bars"] = "\\f0c9", + ["fa-navicon"] = "\\f0c9", + ["fa-list-ul"] = "\\f0ca", + ["fa-list-dots"] = "\\f0ca", + ["fa-list-ol"] = "\\f0cb", + ["fa-list-1-2"] = "\\f0cb", + ["fa-list-numeric"] = "\\f0cb", + ["fa-strikethrough"] = "\\f0cc", + ["fa-underline"] = "\\f0cd", + ["fa-table"] = "\\f0ce", + ["fa-wand-magic"] = "\\f0d0", + ["fa-magic"] = "\\f0d0", + ["fa-truck"] = "\\f0d1", + ["fa-money-bill"] = "\\f0d6", + ["fa-caret-down"] = "\\f0d7", + ["fa-caret-up"] = "\\f0d8", + ["fa-caret-left"] = "\\f0d9", + ["fa-caret-right"] = "\\f0da", + ["fa-table-columns"] = "\\f0db", + ["fa-columns"] = "\\f0db", + ["fa-sort"] = "\\f0dc", + ["fa-unsorted"] = "\\f0dc", + ["fa-sort-down"] = "\\f0dd", + ["fa-sort-desc"] = "\\f0dd", + ["fa-sort-up"] = "\\f0de", + ["fa-sort-asc"] = "\\f0de", + ["fa-envelope"] = "\\f0e0", + ["fa-arrow-rotate-left"] = "\\f0e2", + ["fa-arrow-left-rotate"] = "\\f0e2", + ["fa-arrow-rotate-back"] = "\\f0e2", + ["fa-arrow-rotate-backward"] = "\\f0e2", + ["fa-undo"] = "\\f0e2", + ["fa-gavel"] = "\\f0e3", + ["fa-legal"] = "\\f0e3", + ["fa-bolt"] = "\\f0e7", + ["fa-zap"] = "\\f0e7", + ["fa-sitemap"] = "\\f0e8", + ["fa-umbrella"] = "\\f0e9", + ["fa-paste"] = "\\f0ea", + ["fa-file-clipboard"] = "\\f0ea", + ["fa-lightbulb"] = "\\f0eb", + ["fa-arrow-right-arrow-left"] = "\\f0ec", + ["fa-exchange"] = "\\f0ec", + ["fa-cloud-arrow-down"] = "\\f0ed", + ["fa-cloud-download"] = "\\f0ed", + ["fa-cloud-download-alt"] = "\\f0ed", + ["fa-cloud-arrow-up"] = "\\f0ee", + ["fa-cloud-upload"] = "\\f0ee", + ["fa-cloud-upload-alt"] = "\\f0ee", + ["fa-user-doctor"] = "\\f0f0", + ["fa-user-md"] = "\\f0f0", + ["fa-stethoscope"] = "\\f0f1", + ["fa-suitcase"] = "\\f0f2", + ["fa-bell"] = "\\f0f3", + ["fa-mug-saucer"] = "\\f0f4", + ["fa-coffee"] = "\\f0f4", + ["fa-hospital"] = "\\f0f8", + ["fa-hospital-alt"] = "\\f0f8", + ["fa-hospital-wide"] = "\\f0f8", + ["fa-truck-medical"] = "\\f0f9", + ["fa-ambulance"] = "\\f0f9", + ["fa-suitcase-medical"] = "\\f0fa", + ["fa-medkit"] = "\\f0fa", + ["fa-jet-fighter"] = "\\f0fb", + ["fa-fighter-jet"] = "\\f0fb", + ["fa-beer-mug-empty"] = "\\f0fc", + ["fa-beer"] = "\\f0fc", + ["fa-square-h"] = "\\f0fd", + ["fa-h-square"] = "\\f0fd", + ["fa-square-plus"] = "\\f0fe", + ["fa-plus-square"] = "\\f0fe", + ["fa-angles-left"] = "\\f100", + ["fa-angle-double-left"] = "\\f100", + ["fa-angles-right"] = "\\f101", + ["fa-angle-double-right"] = "\\f101", + ["fa-angles-up"] = "\\f102", + ["fa-angle-double-up"] = "\\f102", + ["fa-angles-down"] = "\\f103", + ["fa-angle-double-down"] = "\\f103", + ["fa-angle-left"] = "\\f104", + ["fa-angle-right"] = "\\f105", + ["fa-angle-up"] = "\\f106", + ["fa-angle-down"] = "\\f107", + ["fa-laptop"] = "\\f109", + ["fa-tablet-button"] = "\\f10a", + ["fa-mobile-button"] = "\\f10b", + ["fa-quote-left"] = "\\f10d", + ["fa-quote-left-alt"] = "\\f10d", + ["fa-quote-right"] = "\\f10e", + ["fa-quote-right-alt"] = "\\f10e", + ["fa-spinner"] = "\\f110", + ["fa-circle"] = "\\f111", + ["fa-face-smile"] = "\\f118", + ["fa-smile"] = "\\f118", + ["fa-face-frown"] = "\\f119", + ["fa-frown"] = "\\f119", + ["fa-face-meh"] = "\\f11a", + ["fa-meh"] = "\\f11a", + ["fa-gamepad"] = "\\f11b", + ["fa-keyboard"] = "\\f11c", + ["fa-flag-checkered"] = "\\f11e", + ["fa-terminal"] = "\\f120", + ["fa-code"] = "\\f121", + ["fa-reply-all"] = "\\f122", + ["fa-mail-reply-all"] = "\\f122", + ["fa-location-arrow"] = "\\f124", + ["fa-crop"] = "\\f125", + ["fa-code-branch"] = "\\f126", + ["fa-link-slash"] = "\\f127", + ["fa-chain-broken"] = "\\f127", + ["fa-chain-slash"] = "\\f127", + ["fa-unlink"] = "\\f127", + ["fa-info"] = "\\f129", + ["fa-superscript"] = "\\f12b", + ["fa-subscript"] = "\\f12c", + ["fa-eraser"] = "\\f12d", + ["fa-puzzle-piece"] = "\\f12e", + ["fa-microphone"] = "\\f130", + ["fa-microphone-slash"] = "\\f131", + ["fa-shield"] = "\\f132", + ["fa-shield-blank"] = "\\f132", + ["fa-calendar"] = "\\f133", + ["fa-fire-extinguisher"] = "\\f134", + ["fa-rocket"] = "\\f135", + ["fa-circle-chevron-left"] = "\\f137", + ["fa-chevron-circle-left"] = "\\f137", + ["fa-circle-chevron-right"] = "\\f138", + ["fa-chevron-circle-right"] = "\\f138", + ["fa-circle-chevron-up"] = "\\f139", + ["fa-chevron-circle-up"] = "\\f139", + ["fa-circle-chevron-down"] = "\\f13a", + ["fa-chevron-circle-down"] = "\\f13a", + ["fa-anchor"] = "\\f13d", + ["fa-unlock-keyhole"] = "\\f13e", + ["fa-unlock-alt"] = "\\f13e", + ["fa-bullseye"] = "\\f140", + ["fa-ellipsis"] = "\\f141", + ["fa-ellipsis-h"] = "\\f141", + ["fa-ellipsis-vertical"] = "\\f142", + ["fa-ellipsis-v"] = "\\f142", + ["fa-square-rss"] = "\\f143", + ["fa-rss-square"] = "\\f143", + ["fa-circle-play"] = "\\f144", + ["fa-play-circle"] = "\\f144", + ["fa-ticket"] = "\\f145", + ["fa-square-minus"] = "\\f146", + ["fa-minus-square"] = "\\f146", + ["fa-arrow-turn-up"] = "\\f148", + ["fa-level-up"] = "\\f148", + ["fa-arrow-turn-down"] = "\\f149", + ["fa-level-down"] = "\\f149", + ["fa-square-check"] = "\\f14a", + ["fa-check-square"] = "\\f14a", + ["fa-square-pen"] = "\\f14b", + ["fa-pen-square"] = "\\f14b", + ["fa-pencil-square"] = "\\f14b", + ["fa-square-arrow-up-right"] = "\\f14c", + ["fa-external-link-square"] = "\\f14c", + ["fa-share-from-square"] = "\\f14d", + ["fa-share-square"] = "\\f14d", + ["fa-compass"] = "\\f14e", + ["fa-square-caret-down"] = "\\f150", + ["fa-caret-square-down"] = "\\f150", + ["fa-square-caret-up"] = "\\f151", + ["fa-caret-square-up"] = "\\f151", + ["fa-square-caret-right"] = "\\f152", + ["fa-caret-square-right"] = "\\f152", + ["fa-euro-sign"] = "\\f153", + ["fa-eur"] = "\\f153", + ["fa-euro"] = "\\f153", + ["fa-sterling-sign"] = "\\f154", + ["fa-gbp"] = "\\f154", + ["fa-pound-sign"] = "\\f154", + ["fa-rupee-sign"] = "\\f156", + ["fa-rupee"] = "\\f156", + ["fa-yen-sign"] = "\\f157", + ["fa-cny"] = "\\f157", + ["fa-jpy"] = "\\f157", + ["fa-rmb"] = "\\f157", + ["fa-yen"] = "\\f157", + ["fa-ruble-sign"] = "\\f158", + ["fa-rouble"] = "\\f158", + ["fa-rub"] = "\\f158", + ["fa-ruble"] = "\\f158", + ["fa-won-sign"] = "\\f159", + ["fa-krw"] = "\\f159", + ["fa-won"] = "\\f159", + ["fa-file"] = "\\f15b", + ["fa-file-lines"] = "\\f15c", + ["fa-file-alt"] = "\\f15c", + ["fa-file-text"] = "\\f15c", + ["fa-arrow-down-a-z"] = "\\f15d", + ["fa-sort-alpha-asc"] = "\\f15d", + ["fa-sort-alpha-down"] = "\\f15d", + ["fa-arrow-up-a-z"] = "\\f15e", + ["fa-sort-alpha-up"] = "\\f15e", + ["fa-arrow-down-wide-short"] = "\\f160", + ["fa-sort-amount-asc"] = "\\f160", + ["fa-sort-amount-down"] = "\\f160", + ["fa-arrow-up-wide-short"] = "\\f161", + ["fa-sort-amount-up"] = "\\f161", + ["fa-arrow-down-1-9"] = "\\f162", + ["fa-sort-numeric-asc"] = "\\f162", + ["fa-sort-numeric-down"] = "\\f162", + ["fa-arrow-up-1-9"] = "\\f163", + ["fa-sort-numeric-up"] = "\\f163", + ["fa-thumbs-up"] = "\\f164", + ["fa-thumbs-down"] = "\\f165", + ["fa-arrow-down-long"] = "\\f175", + ["fa-long-arrow-down"] = "\\f175", + ["fa-arrow-up-long"] = "\\f176", + ["fa-long-arrow-up"] = "\\f176", + ["fa-arrow-left-long"] = "\\f177", + ["fa-long-arrow-left"] = "\\f177", + ["fa-arrow-right-long"] = "\\f178", + ["fa-long-arrow-right"] = "\\f178", + ["fa-person-dress"] = "\\f182", + ["fa-female"] = "\\f182", + ["fa-person"] = "\\f183", + ["fa-male"] = "\\f183", + ["fa-sun"] = "\\f185", + ["fa-moon"] = "\\f186", + ["fa-box-archive"] = "\\f187", + ["fa-archive"] = "\\f187", + ["fa-bug"] = "\\f188", + ["fa-square-caret-left"] = "\\f191", + ["fa-caret-square-left"] = "\\f191", + ["fa-circle-dot"] = "\\f192", + ["fa-dot-circle"] = "\\f192", + ["fa-wheelchair"] = "\\f193", + ["fa-lira-sign"] = "\\f195", + ["fa-shuttle-space"] = "\\f197", + ["fa-space-shuttle"] = "\\f197", + ["fa-square-envelope"] = "\\f199", + ["fa-envelope-square"] = "\\f199", + ["fa-building-columns"] = "\\f19c", + ["fa-bank"] = "\\f19c", + ["fa-institution"] = "\\f19c", + ["fa-museum"] = "\\f19c", + ["fa-university"] = "\\f19c", + ["fa-graduation-cap"] = "\\f19d", + ["fa-mortar-board"] = "\\f19d", + ["fa-language"] = "\\f1ab", + ["fa-fax"] = "\\f1ac", + ["fa-building"] = "\\f1ad", + ["fa-child"] = "\\f1ae", + ["fa-paw"] = "\\f1b0", + ["fa-cube"] = "\\f1b2", + ["fa-cubes"] = "\\f1b3", + ["fa-recycle"] = "\\f1b8", + ["fa-car"] = "\\f1b9", + ["fa-automobile"] = "\\f1b9", + ["fa-taxi"] = "\\f1ba", + ["fa-cab"] = "\\f1ba", + ["fa-tree"] = "\\f1bb", + ["fa-database"] = "\\f1c0", + ["fa-file-pdf"] = "\\f1c1", + ["fa-file-word"] = "\\f1c2", + ["fa-file-excel"] = "\\f1c3", + ["fa-file-powerpoint"] = "\\f1c4", + ["fa-file-image"] = "\\f1c5", + ["fa-file-zipper"] = "\\f1c6", + ["fa-file-archive"] = "\\f1c6", + ["fa-file-audio"] = "\\f1c7", + ["fa-file-video"] = "\\f1c8", + ["fa-file-code"] = "\\f1c9", + ["fa-life-ring"] = "\\f1cd", + ["fa-circle-notch"] = "\\f1ce", + ["fa-paper-plane"] = "\\f1d8", + ["fa-clock-rotate-left"] = "\\f1da", + ["fa-history"] = "\\f1da", + ["fa-heading"] = "\\f1dc", + ["fa-header"] = "\\f1dc", + ["fa-paragraph"] = "\\f1dd", + ["fa-sliders"] = "\\f1de", + ["fa-sliders-h"] = "\\f1de", + ["fa-share-nodes"] = "\\f1e0", + ["fa-share-alt"] = "\\f1e0", + ["fa-square-share-nodes"] = "\\f1e1", + ["fa-share-alt-square"] = "\\f1e1", + ["fa-bomb"] = "\\f1e2", + ["fa-futbol"] = "\\f1e3", + ["fa-futbol-ball"] = "\\f1e3", + ["fa-soccer-ball"] = "\\f1e3", + ["fa-tty"] = "\\f1e4", + ["fa-teletype"] = "\\f1e4", + ["fa-binoculars"] = "\\f1e5", + ["fa-plug"] = "\\f1e6", + ["fa-newspaper"] = "\\f1ea", + ["fa-wifi"] = "\\f1eb", + ["fa-wifi-3"] = "\\f1eb", + ["fa-wifi-strong"] = "\\f1eb", + ["fa-calculator"] = "\\f1ec", + ["fa-bell-slash"] = "\\f1f6", + ["fa-trash"] = "\\f1f8", + ["fa-copyright"] = "\\f1f9", + ["fa-eye-dropper"] = "\\f1fb", + ["fa-eye-dropper-empty"] = "\\f1fb", + ["fa-eyedropper"] = "\\f1fb", + ["fa-paintbrush"] = "\\f1fc", + ["fa-paint-brush"] = "\\f1fc", + ["fa-cake-candles"] = "\\f1fd", + ["fa-birthday-cake"] = "\\f1fd", + ["fa-cake"] = "\\f1fd", + ["fa-chart-area"] = "\\f1fe", + ["fa-area-chart"] = "\\f1fe", + ["fa-chart-pie"] = "\\f200", + ["fa-pie-chart"] = "\\f200", + ["fa-chart-line"] = "\\f201", + ["fa-line-chart"] = "\\f201", + ["fa-toggle-off"] = "\\f204", + ["fa-toggle-on"] = "\\f205", + ["fa-bicycle"] = "\\f206", + ["fa-bus"] = "\\f207", + ["fa-closed-captioning"] = "\\f20a", + ["fa-shekel-sign"] = "\\f20b", + ["fa-ils"] = "\\f20b", + ["fa-shekel"] = "\\f20b", + ["fa-sheqel"] = "\\f20b", + ["fa-sheqel-sign"] = "\\f20b", + ["fa-cart-plus"] = "\\f217", + ["fa-cart-arrow-down"] = "\\f218", + ["fa-diamond"] = "\\f219", + ["fa-ship"] = "\\f21a", + ["fa-user-secret"] = "\\f21b", + ["fa-motorcycle"] = "\\f21c", + ["fa-street-view"] = "\\f21d", + ["fa-heart-pulse"] = "\\f21e", + ["fa-heartbeat"] = "\\f21e", + ["fa-venus"] = "\\f221", + ["fa-mars"] = "\\f222", + ["fa-mercury"] = "\\f223", + ["fa-mars-and-venus"] = "\\f224", + ["fa-transgender"] = "\\f225", + ["fa-transgender-alt"] = "\\f225", + ["fa-venus-double"] = "\\f226", + ["fa-mars-double"] = "\\f227", + ["fa-venus-mars"] = "\\f228", + ["fa-mars-stroke"] = "\\f229", + ["fa-mars-stroke-up"] = "\\f22a", + ["fa-mars-stroke-v"] = "\\f22a", + ["fa-mars-stroke-right"] = "\\f22b", + ["fa-mars-stroke-h"] = "\\f22b", + ["fa-neuter"] = "\\f22c", + ["fa-genderless"] = "\\f22d", + ["fa-server"] = "\\f233", + ["fa-user-plus"] = "\\f234", + ["fa-user-xmark"] = "\\f235", + ["fa-user-times"] = "\\f235", + ["fa-bed"] = "\\f236", + ["fa-train"] = "\\f238", + ["fa-train-subway"] = "\\f239", + ["fa-subway"] = "\\f239", + ["fa-battery-full"] = "\\f240", + ["fa-battery"] = "\\f240", + ["fa-battery-5"] = "\\f240", + ["fa-battery-three-quarters"] = "\\f241", + ["fa-battery-4"] = "\\f241", + ["fa-battery-half"] = "\\f242", + ["fa-battery-3"] = "\\f242", + ["fa-battery-quarter"] = "\\f243", + ["fa-battery-2"] = "\\f243", + ["fa-battery-empty"] = "\\f244", + ["fa-battery-0"] = "\\f244", + ["fa-arrow-pointer"] = "\\f245", + ["fa-mouse-pointer"] = "\\f245", + ["fa-i-cursor"] = "\\f246", + ["fa-object-group"] = "\\f247", + ["fa-object-ungroup"] = "\\f248", + ["fa-note-sticky"] = "\\f249", + ["fa-sticky-note"] = "\\f249", + ["fa-clone"] = "\\f24d", + ["fa-scale-balanced"] = "\\f24e", + ["fa-balance-scale"] = "\\f24e", + ["fa-hourglass-start"] = "\\f251", + ["fa-hourglass-1"] = "\\f251", + ["fa-hourglass-half"] = "\\f252", + ["fa-hourglass-2"] = "\\f252", + ["fa-hourglass-end"] = "\\f253", + ["fa-hourglass-3"] = "\\f253", + ["fa-hourglass"] = "\\f254", + ["fa-hourglass-empty"] = "\\f254", + ["fa-hand-back-fist"] = "\\f255", + ["fa-hand-rock"] = "\\f255", + ["fa-hand"] = "\\f256", + ["fa-hand-paper"] = "\\f256", + ["fa-hand-scissors"] = "\\f257", + ["fa-hand-lizard"] = "\\f258", + ["fa-hand-spock"] = "\\f259", + ["fa-hand-pointer"] = "\\f25a", + ["fa-hand-peace"] = "\\f25b", + ["fa-trademark"] = "\\f25c", + ["fa-registered"] = "\\f25d", + ["fa-tv"] = "\\f26c", + ["fa-television"] = "\\f26c", + ["fa-tv-alt"] = "\\f26c", + ["fa-calendar-plus"] = "\\f271", + ["fa-calendar-minus"] = "\\f272", + ["fa-calendar-xmark"] = "\\f273", + ["fa-calendar-times"] = "\\f273", + ["fa-calendar-check"] = "\\f274", + ["fa-industry"] = "\\f275", + ["fa-map-pin"] = "\\f276", + ["fa-signs-post"] = "\\f277", + ["fa-map-signs"] = "\\f277", + ["fa-map"] = "\\f279", + ["fa-message"] = "\\f27a", + ["fa-comment-alt"] = "\\f27a", + ["fa-circle-pause"] = "\\f28b", + ["fa-pause-circle"] = "\\f28b", + ["fa-circle-stop"] = "\\f28d", + ["fa-stop-circle"] = "\\f28d", + ["fa-bag-shopping"] = "\\f290", + ["fa-shopping-bag"] = "\\f290", + ["fa-basket-shopping"] = "\\f291", + ["fa-shopping-basket"] = "\\f291", + ["fa-universal-access"] = "\\f29a", + ["fa-person-walking-with-cane"] = "\\f29d", + ["fa-blind"] = "\\f29d", + ["fa-audio-description"] = "\\f29e", + ["fa-phone-volume"] = "\\f2a0", + ["fa-volume-control-phone"] = "\\f2a0", + ["fa-braille"] = "\\f2a1", + ["fa-ear-listen"] = "\\f2a2", + ["fa-assistive-listening-systems"] = "\\f2a2", + ["fa-hands-asl-interpreting"] = "\\f2a3", + ["fa-american-sign-language-interpreting"] = "\\f2a3", + ["fa-asl-interpreting"] = "\\f2a3", + ["fa-hands-american-sign-language-interpreting"] = "\\f2a3", + ["fa-ear-deaf"] = "\\f2a4", + ["fa-deaf"] = "\\f2a4", + ["fa-deafness"] = "\\f2a4", + ["fa-hard-of-hearing"] = "\\f2a4", + ["fa-hands"] = "\\f2a7", + ["fa-sign-language"] = "\\f2a7", + ["fa-signing"] = "\\f2a7", + ["fa-eye-low-vision"] = "\\f2a8", + ["fa-low-vision"] = "\\f2a8", + ["fa-font-awesome"] = "\\f2b4", + ["fa-font-awesome-flag"] = "\\f2b4", + ["fa-font-awesome-logo-full"] = "\\f2b4", + ["fa-handshake"] = "\\f2b5", + ["fa-handshake-alt"] = "\\f2b5", + ["fa-handshake-simple"] = "\\f2b5", + ["fa-envelope-open"] = "\\f2b6", + ["fa-address-book"] = "\\f2b9", + ["fa-contact-book"] = "\\f2b9", + ["fa-address-card"] = "\\f2bb", + ["fa-contact-card"] = "\\f2bb", + ["fa-vcard"] = "\\f2bb", + ["fa-circle-user"] = "\\f2bd", + ["fa-user-circle"] = "\\f2bd", + ["fa-id-badge"] = "\\f2c1", + ["fa-id-card"] = "\\f2c2", + ["fa-drivers-license"] = "\\f2c2", + ["fa-temperature-full"] = "\\f2c7", + ["fa-temperature-4"] = "\\f2c7", + ["fa-thermometer-4"] = "\\f2c7", + ["fa-thermometer-full"] = "\\f2c7", + ["fa-temperature-three-quarters"] = "\\f2c8", + ["fa-temperature-3"] = "\\f2c8", + ["fa-thermometer-3"] = "\\f2c8", + ["fa-thermometer-three-quarters"] = "\\f2c8", + ["fa-temperature-half"] = "\\f2c9", + ["fa-temperature-2"] = "\\f2c9", + ["fa-thermometer-2"] = "\\f2c9", + ["fa-thermometer-half"] = "\\f2c9", + ["fa-temperature-quarter"] = "\\f2ca", + ["fa-temperature-1"] = "\\f2ca", + ["fa-thermometer-1"] = "\\f2ca", + ["fa-thermometer-quarter"] = "\\f2ca", + ["fa-temperature-empty"] = "\\f2cb", + ["fa-temperature-0"] = "\\f2cb", + ["fa-thermometer-0"] = "\\f2cb", + ["fa-thermometer-empty"] = "\\f2cb", + ["fa-shower"] = "\\f2cc", + ["fa-bath"] = "\\f2cd", + ["fa-bathtub"] = "\\f2cd", + ["fa-podcast"] = "\\f2ce", + ["fa-window-maximize"] = "\\f2d0", + ["fa-window-minimize"] = "\\f2d1", + ["fa-window-restore"] = "\\f2d2", + ["fa-square-xmark"] = "\\f2d3", + ["fa-times-square"] = "\\f2d3", + ["fa-xmark-square"] = "\\f2d3", + ["fa-microchip"] = "\\f2db", + ["fa-snowflake"] = "\\f2dc", + ["fa-spoon"] = "\\f2e5", + ["fa-utensil-spoon"] = "\\f2e5", + ["fa-utensils"] = "\\f2e7", + ["fa-cutlery"] = "\\f2e7", + ["fa-rotate-left"] = "\\f2ea", + ["fa-rotate-back"] = "\\f2ea", + ["fa-rotate-backward"] = "\\f2ea", + ["fa-undo-alt"] = "\\f2ea", + ["fa-trash-can"] = "\\f2ed", + ["fa-trash-alt"] = "\\f2ed", + ["fa-rotate"] = "\\f2f1", + ["fa-sync-alt"] = "\\f2f1", + ["fa-stopwatch"] = "\\f2f2", + ["fa-right-from-bracket"] = "\\f2f5", + ["fa-sign-out-alt"] = "\\f2f5", + ["fa-right-to-bracket"] = "\\f2f6", + ["fa-sign-in-alt"] = "\\f2f6", + ["fa-rotate-right"] = "\\f2f9", + ["fa-redo-alt"] = "\\f2f9", + ["fa-rotate-forward"] = "\\f2f9", + ["fa-poo"] = "\\f2fe", + ["fa-images"] = "\\f302", + ["fa-pencil"] = "\\f303", + ["fa-pencil-alt"] = "\\f303", + ["fa-pen"] = "\\f304", + ["fa-pen-clip"] = "\\f305", + ["fa-pen-alt"] = "\\f305", + ["fa-octagon"] = "\\f306", + ["fa-down-long"] = "\\f309", + ["fa-long-arrow-alt-down"] = "\\f309", + ["fa-left-long"] = "\\f30a", + ["fa-long-arrow-alt-left"] = "\\f30a", + ["fa-right-long"] = "\\f30b", + ["fa-long-arrow-alt-right"] = "\\f30b", + ["fa-up-long"] = "\\f30c", + ["fa-long-arrow-alt-up"] = "\\f30c", + ["fa-hexagon"] = "\\f312", + ["fa-file-pen"] = "\\f31c", + ["fa-file-edit"] = "\\f31c", + ["fa-maximize"] = "\\f31e", + ["fa-expand-arrows-alt"] = "\\f31e", + ["fa-clipboard"] = "\\f328", + ["fa-left-right"] = "\\f337", + ["fa-arrows-alt-h"] = "\\f337", + ["fa-up-down"] = "\\f338", + ["fa-arrows-alt-v"] = "\\f338", + ["fa-alarm-clock"] = "\\f34e", + ["fa-circle-down"] = "\\f358", + ["fa-arrow-alt-circle-down"] = "\\f358", + ["fa-circle-left"] = "\\f359", + ["fa-arrow-alt-circle-left"] = "\\f359", + ["fa-circle-right"] = "\\f35a", + ["fa-arrow-alt-circle-right"] = "\\f35a", + ["fa-circle-up"] = "\\f35b", + ["fa-arrow-alt-circle-up"] = "\\f35b", + ["fa-up-right-from-square"] = "\\f35d", + ["fa-external-link-alt"] = "\\f35d", + ["fa-square-up-right"] = "\\f360", + ["fa-external-link-square-alt"] = "\\f360", + ["fa-right-left"] = "\\f362", + ["fa-exchange-alt"] = "\\f362", + ["fa-repeat"] = "\\f363", + ["fa-code-commit"] = "\\f386", + ["fa-code-merge"] = "\\f387", + ["fa-desktop"] = "\\f390", + ["fa-desktop-alt"] = "\\f390", + ["fa-gem"] = "\\f3a5", + ["fa-turn-down"] = "\\f3be", + ["fa-level-down-alt"] = "\\f3be", + ["fa-turn-up"] = "\\f3bf", + ["fa-level-up-alt"] = "\\f3bf", + ["fa-lock-open"] = "\\f3c1", + ["fa-location-dot"] = "\\f3c5", + ["fa-map-marker-alt"] = "\\f3c5", + ["fa-microphone-lines"] = "\\f3c9", + ["fa-microphone-alt"] = "\\f3c9", + ["fa-mobile-screen-button"] = "\\f3cd", + ["fa-mobile-alt"] = "\\f3cd", + ["fa-mobile"] = "\\f3ce", + ["fa-mobile-android"] = "\\f3ce", + ["fa-mobile-phone"] = "\\f3ce", + ["fa-mobile-screen"] = "\\f3cf", + ["fa-mobile-android-alt"] = "\\f3cf", + ["fa-money-bill-1"] = "\\f3d1", + ["fa-money-bill-alt"] = "\\f3d1", + ["fa-phone-slash"] = "\\f3dd", + ["fa-image-portrait"] = "\\f3e0", + ["fa-portrait"] = "\\f3e0", + ["fa-reply"] = "\\f3e5", + ["fa-mail-reply"] = "\\f3e5", + ["fa-shield-halved"] = "\\f3ed", + ["fa-shield-alt"] = "\\f3ed", + ["fa-tablet-screen-button"] = "\\f3fa", + ["fa-tablet-alt"] = "\\f3fa", + ["fa-tablet"] = "\\f3fb", + ["fa-tablet-android"] = "\\f3fb", + ["fa-ticket-simple"] = "\\f3ff", + ["fa-ticket-alt"] = "\\f3ff", + ["fa-rectangle-xmark"] = "\\f410", + ["fa-rectangle-times"] = "\\f410", + ["fa-times-rectangle"] = "\\f410", + ["fa-window-close"] = "\\f410", + ["fa-down-left-and-up-right-to-center"] = "\\f422", + ["fa-compress-alt"] = "\\f422", + ["fa-up-right-and-down-left-from-center"] = "\\f424", + ["fa-expand-alt"] = "\\f424", + ["fa-baseball-bat-ball"] = "\\f432", + ["fa-baseball"] = "\\f433", + ["fa-baseball-ball"] = "\\f433", + ["fa-basketball"] = "\\f434", + ["fa-basketball-ball"] = "\\f434", + ["fa-bowling-ball"] = "\\f436", + ["fa-chess"] = "\\f439", + ["fa-chess-bishop"] = "\\f43a", + ["fa-chess-board"] = "\\f43c", + ["fa-chess-king"] = "\\f43f", + ["fa-chess-knight"] = "\\f441", + ["fa-chess-pawn"] = "\\f443", + ["fa-chess-queen"] = "\\f445", + ["fa-chess-rook"] = "\\f447", + ["fa-dumbbell"] = "\\f44b", + ["fa-football"] = "\\f44e", + ["fa-football-ball"] = "\\f44e", + ["fa-golf-ball-tee"] = "\\f450", + ["fa-golf-ball"] = "\\f450", + ["fa-hockey-puck"] = "\\f453", + ["fa-broom-ball"] = "\\f458", + ["fa-quidditch"] = "\\f458", + ["fa-quidditch-broom-ball"] = "\\f458", + ["fa-square-full"] = "\\f45c", + ["fa-table-tennis-paddle-ball"] = "\\f45d", + ["fa-ping-pong-paddle-ball"] = "\\f45d", + ["fa-table-tennis"] = "\\f45d", + ["fa-volleyball"] = "\\f45f", + ["fa-volleyball-ball"] = "\\f45f", + ["fa-hand-dots"] = "\\f461", + ["fa-allergies"] = "\\f461", + ["fa-bandage"] = "\\f462", + ["fa-band-aid"] = "\\f462", + ["fa-box"] = "\\f466", + ["fa-boxes-stacked"] = "\\f468", + ["fa-boxes"] = "\\f468", + ["fa-boxes-alt"] = "\\f468", + ["fa-briefcase-medical"] = "\\f469", + ["fa-fire-flame-simple"] = "\\f46a", + ["fa-burn"] = "\\f46a", + ["fa-capsules"] = "\\f46b", + ["fa-clipboard-check"] = "\\f46c", + ["fa-clipboard-list"] = "\\f46d", + ["fa-person-dots-from-line"] = "\\f470", + ["fa-diagnoses"] = "\\f470", + ["fa-dna"] = "\\f471", + ["fa-dolly"] = "\\f472", + ["fa-dolly-box"] = "\\f472", + ["fa-cart-flatbed"] = "\\f474", + ["fa-dolly-flatbed"] = "\\f474", + ["fa-file-medical"] = "\\f477", + ["fa-file-waveform"] = "\\f478", + ["fa-file-medical-alt"] = "\\f478", + ["fa-kit-medical"] = "\\f479", + ["fa-first-aid"] = "\\f479", + ["fa-circle-h"] = "\\f47e", + ["fa-hospital-symbol"] = "\\f47e", + ["fa-id-card-clip"] = "\\f47f", + ["fa-id-card-alt"] = "\\f47f", + ["fa-notes-medical"] = "\\f481", + ["fa-pallet"] = "\\f482", + ["fa-pills"] = "\\f484", + ["fa-prescription-bottle"] = "\\f485", + ["fa-prescription-bottle-medical"] = "\\f486", + ["fa-prescription-bottle-alt"] = "\\f486", + ["fa-bed-pulse"] = "\\f487", + ["fa-procedures"] = "\\f487", + ["fa-truck-fast"] = "\\f48b", + ["fa-shipping-fast"] = "\\f48b", + ["fa-smoking"] = "\\f48d", + ["fa-syringe"] = "\\f48e", + ["fa-tablets"] = "\\f490", + ["fa-thermometer"] = "\\f491", + ["fa-vial"] = "\\f492", + ["fa-vials"] = "\\f493", + ["fa-warehouse"] = "\\f494", + ["fa-weight-scale"] = "\\f496", + ["fa-weight"] = "\\f496", + ["fa-x-ray"] = "\\f497", + ["fa-box-open"] = "\\f49e", + ["fa-comment-dots"] = "\\f4ad", + ["fa-commenting"] = "\\f4ad", + ["fa-comment-slash"] = "\\f4b3", + ["fa-couch"] = "\\f4b8", + ["fa-circle-dollar-to-slot"] = "\\f4b9", + ["fa-donate"] = "\\f4b9", + ["fa-dove"] = "\\f4ba", + ["fa-hand-holding"] = "\\f4bd", + ["fa-hand-holding-heart"] = "\\f4be", + ["fa-hand-holding-dollar"] = "\\f4c0", + ["fa-hand-holding-usd"] = "\\f4c0", + ["fa-hand-holding-droplet"] = "\\f4c1", + ["fa-hand-holding-water"] = "\\f4c1", + ["fa-hands-holding"] = "\\f4c2", + ["fa-handshake-angle"] = "\\f4c4", + ["fa-hands-helping"] = "\\f4c4", + ["fa-parachute-box"] = "\\f4cd", + ["fa-people-carry-box"] = "\\f4ce", + ["fa-people-carry"] = "\\f4ce", + ["fa-piggy-bank"] = "\\f4d3", + ["fa-ribbon"] = "\\f4d6", + ["fa-route"] = "\\f4d7", + ["fa-seedling"] = "\\f4d8", + ["fa-sprout"] = "\\f4d8", + ["fa-sign-hanging"] = "\\f4d9", + ["fa-sign"] = "\\f4d9", + ["fa-face-smile-wink"] = "\\f4da", + ["fa-smile-wink"] = "\\f4da", + ["fa-tape"] = "\\f4db", + ["fa-truck-ramp-box"] = "\\f4de", + ["fa-truck-loading"] = "\\f4de", + ["fa-truck-moving"] = "\\f4df", + ["fa-video-slash"] = "\\f4e2", + ["fa-wine-glass"] = "\\f4e3", + ["fa-user-astronaut"] = "\\f4fb", + ["fa-user-check"] = "\\f4fc", + ["fa-user-clock"] = "\\f4fd", + ["fa-user-gear"] = "\\f4fe", + ["fa-user-cog"] = "\\f4fe", + ["fa-user-pen"] = "\\f4ff", + ["fa-user-edit"] = "\\f4ff", + ["fa-user-group"] = "\\f500", + ["fa-user-friends"] = "\\f500", + ["fa-user-graduate"] = "\\f501", + ["fa-user-lock"] = "\\f502", + ["fa-user-minus"] = "\\f503", + ["fa-user-ninja"] = "\\f504", + ["fa-user-shield"] = "\\f505", + ["fa-user-slash"] = "\\f506", + ["fa-user-alt-slash"] = "\\f506", + ["fa-user-large-slash"] = "\\f506", + ["fa-user-tag"] = "\\f507", + ["fa-user-tie"] = "\\f508", + ["fa-users-gear"] = "\\f509", + ["fa-users-cog"] = "\\f509", + ["fa-scale-unbalanced"] = "\\f515", + ["fa-balance-scale-left"] = "\\f515", + ["fa-scale-unbalanced-flip"] = "\\f516", + ["fa-balance-scale-right"] = "\\f516", + ["fa-blender"] = "\\f517", + ["fa-book-open"] = "\\f518", + ["fa-tower-broadcast"] = "\\f519", + ["fa-broadcast-tower"] = "\\f519", + ["fa-broom"] = "\\f51a", + ["fa-chalkboard"] = "\\f51b", + ["fa-blackboard"] = "\\f51b", + ["fa-chalkboard-user"] = "\\f51c", + ["fa-chalkboard-teacher"] = "\\f51c", + ["fa-church"] = "\\f51d", + ["fa-coins"] = "\\f51e", + ["fa-compact-disc"] = "\\f51f", + ["fa-crow"] = "\\f520", + ["fa-crown"] = "\\f521", + ["fa-dice"] = "\\f522", + ["fa-dice-five"] = "\\f523", + ["fa-dice-four"] = "\\f524", + ["fa-dice-one"] = "\\f525", + ["fa-dice-six"] = "\\f526", + ["fa-dice-three"] = "\\f527", + ["fa-dice-two"] = "\\f528", + ["fa-divide"] = "\\f529", + ["fa-door-closed"] = "\\f52a", + ["fa-door-open"] = "\\f52b", + ["fa-feather"] = "\\f52d", + ["fa-frog"] = "\\f52e", + ["fa-gas-pump"] = "\\f52f", + ["fa-glasses"] = "\\f530", + ["fa-greater-than-equal"] = "\\f532", + ["fa-helicopter"] = "\\f533", + ["fa-infinity"] = "\\f534", + ["fa-kiwi-bird"] = "\\f535", + ["fa-less-than-equal"] = "\\f537", + ["fa-memory"] = "\\f538", + ["fa-microphone-lines-slash"] = "\\f539", + ["fa-microphone-alt-slash"] = "\\f539", + ["fa-money-bill-wave"] = "\\f53a", + ["fa-money-bill-1-wave"] = "\\f53b", + ["fa-money-bill-wave-alt"] = "\\f53b", + ["fa-money-check"] = "\\f53c", + ["fa-money-check-dollar"] = "\\f53d", + ["fa-money-check-alt"] = "\\f53d", + ["fa-not-equal"] = "\\f53e", + ["fa-palette"] = "\\f53f", + ["fa-square-parking"] = "\\f540", + ["fa-parking"] = "\\f540", + ["fa-diagram-project"] = "\\f542", + ["fa-project-diagram"] = "\\f542", + ["fa-receipt"] = "\\f543", + ["fa-robot"] = "\\f544", + ["fa-ruler"] = "\\f545", + ["fa-ruler-combined"] = "\\f546", + ["fa-ruler-horizontal"] = "\\f547", + ["fa-ruler-vertical"] = "\\f548", + ["fa-school"] = "\\f549", + ["fa-screwdriver"] = "\\f54a", + ["fa-shoe-prints"] = "\\f54b", + ["fa-skull"] = "\\f54c", + ["fa-ban-smoking"] = "\\f54d", + ["fa-smoking-ban"] = "\\f54d", + ["fa-store"] = "\\f54e", + ["fa-shop"] = "\\f54f", + ["fa-store-alt"] = "\\f54f", + ["fa-bars-staggered"] = "\\f550", + ["fa-reorder"] = "\\f550", + ["fa-stream"] = "\\f550", + ["fa-stroopwafel"] = "\\f551", + ["fa-toolbox"] = "\\f552", + ["fa-shirt"] = "\\f553", + ["fa-t-shirt"] = "\\f553", + ["fa-tshirt"] = "\\f553", + ["fa-person-walking"] = "\\f554", + ["fa-walking"] = "\\f554", + ["fa-wallet"] = "\\f555", + ["fa-face-angry"] = "\\f556", + ["fa-angry"] = "\\f556", + ["fa-archway"] = "\\f557", + ["fa-book-atlas"] = "\\f558", + ["fa-atlas"] = "\\f558", + ["fa-award"] = "\\f559", + ["fa-delete-left"] = "\\f55a", + ["fa-backspace"] = "\\f55a", + ["fa-bezier-curve"] = "\\f55b", + ["fa-bong"] = "\\f55c", + ["fa-brush"] = "\\f55d", + ["fa-bus-simple"] = "\\f55e", + ["fa-bus-alt"] = "\\f55e", + ["fa-cannabis"] = "\\f55f", + ["fa-check-double"] = "\\f560", + ["fa-martini-glass-citrus"] = "\\f561", + ["fa-cocktail"] = "\\f561", + ["fa-bell-concierge"] = "\\f562", + ["fa-concierge-bell"] = "\\f562", + ["fa-cookie"] = "\\f563", + ["fa-cookie-bite"] = "\\f564", + ["fa-crop-simple"] = "\\f565", + ["fa-crop-alt"] = "\\f565", + ["fa-tachograph-digital"] = "\\f566", + ["fa-digital-tachograph"] = "\\f566", + ["fa-face-dizzy"] = "\\f567", + ["fa-dizzy"] = "\\f567", + ["fa-compass-drafting"] = "\\f568", + ["fa-drafting-compass"] = "\\f568", + ["fa-drum"] = "\\f569", + ["fa-drum-steelpan"] = "\\f56a", + ["fa-feather-pointed"] = "\\f56b", + ["fa-feather-alt"] = "\\f56b", + ["fa-file-contract"] = "\\f56c", + ["fa-file-arrow-down"] = "\\f56d", + ["fa-file-download"] = "\\f56d", + ["fa-file-export"] = "\\f56e", + ["fa-arrow-right-from-file"] = "\\f56e", + ["fa-file-import"] = "\\f56f", + ["fa-arrow-right-to-file"] = "\\f56f", + ["fa-file-invoice"] = "\\f570", + ["fa-file-invoice-dollar"] = "\\f571", + ["fa-file-prescription"] = "\\f572", + ["fa-file-signature"] = "\\f573", + ["fa-file-arrow-up"] = "\\f574", + ["fa-file-upload"] = "\\f574", + ["fa-fill"] = "\\f575", + ["fa-fill-drip"] = "\\f576", + ["fa-fingerprint"] = "\\f577", + ["fa-fish"] = "\\f578", + ["fa-face-flushed"] = "\\f579", + ["fa-flushed"] = "\\f579", + ["fa-face-frown-open"] = "\\f57a", + ["fa-frown-open"] = "\\f57a", + ["fa-martini-glass"] = "\\f57b", + ["fa-glass-martini-alt"] = "\\f57b", + ["fa-earth-africa"] = "\\f57c", + ["fa-globe-africa"] = "\\f57c", + ["fa-earth-americas"] = "\\f57d", + ["fa-earth"] = "\\f57d", + ["fa-earth-america"] = "\\f57d", + ["fa-globe-americas"] = "\\f57d", + ["fa-earth-asia"] = "\\f57e", + ["fa-globe-asia"] = "\\f57e", + ["fa-face-grimace"] = "\\f57f", + ["fa-grimace"] = "\\f57f", + ["fa-face-grin"] = "\\f580", + ["fa-grin"] = "\\f580", + ["fa-face-grin-wide"] = "\\f581", + ["fa-grin-alt"] = "\\f581", + ["fa-face-grin-beam"] = "\\f582", + ["fa-grin-beam"] = "\\f582", + ["fa-face-grin-beam-sweat"] = "\\f583", + ["fa-grin-beam-sweat"] = "\\f583", + ["fa-face-grin-hearts"] = "\\f584", + ["fa-grin-hearts"] = "\\f584", + ["fa-face-grin-squint"] = "\\f585", + ["fa-grin-squint"] = "\\f585", + ["fa-face-grin-squint-tears"] = "\\f586", + ["fa-grin-squint-tears"] = "\\f586", + ["fa-face-grin-stars"] = "\\f587", + ["fa-grin-stars"] = "\\f587", + ["fa-face-grin-tears"] = "\\f588", + ["fa-grin-tears"] = "\\f588", + ["fa-face-grin-tongue"] = "\\f589", + ["fa-grin-tongue"] = "\\f589", + ["fa-face-grin-tongue-squint"] = "\\f58a", + ["fa-grin-tongue-squint"] = "\\f58a", + ["fa-face-grin-tongue-wink"] = "\\f58b", + ["fa-grin-tongue-wink"] = "\\f58b", + ["fa-face-grin-wink"] = "\\f58c", + ["fa-grin-wink"] = "\\f58c", + ["fa-grip"] = "\\f58d", + ["fa-grid-horizontal"] = "\\f58d", + ["fa-grip-horizontal"] = "\\f58d", + ["fa-grip-vertical"] = "\\f58e", + ["fa-grid-vertical"] = "\\f58e", + ["fa-headset"] = "\\f590", + ["fa-highlighter"] = "\\f591", + ["fa-hot-tub-person"] = "\\f593", + ["fa-hot-tub"] = "\\f593", + ["fa-hotel"] = "\\f594", + ["fa-joint"] = "\\f595", + ["fa-face-kiss"] = "\\f596", + ["fa-kiss"] = "\\f596", + ["fa-face-kiss-beam"] = "\\f597", + ["fa-kiss-beam"] = "\\f597", + ["fa-face-kiss-wink-heart"] = "\\f598", + ["fa-kiss-wink-heart"] = "\\f598", + ["fa-face-laugh"] = "\\f599", + ["fa-laugh"] = "\\f599", + ["fa-face-laugh-beam"] = "\\f59a", + ["fa-laugh-beam"] = "\\f59a", + ["fa-face-laugh-squint"] = "\\f59b", + ["fa-laugh-squint"] = "\\f59b", + ["fa-face-laugh-wink"] = "\\f59c", + ["fa-laugh-wink"] = "\\f59c", + ["fa-cart-flatbed-suitcase"] = "\\f59d", + ["fa-luggage-cart"] = "\\f59d", + ["fa-map-location"] = "\\f59f", + ["fa-map-marked"] = "\\f59f", + ["fa-map-location-dot"] = "\\f5a0", + ["fa-map-marked-alt"] = "\\f5a0", + ["fa-marker"] = "\\f5a1", + ["fa-medal"] = "\\f5a2", + ["fa-face-meh-blank"] = "\\f5a4", + ["fa-meh-blank"] = "\\f5a4", + ["fa-face-rolling-eyes"] = "\\f5a5", + ["fa-meh-rolling-eyes"] = "\\f5a5", + ["fa-monument"] = "\\f5a6", + ["fa-mortar-pestle"] = "\\f5a7", + ["fa-paint-roller"] = "\\f5aa", + ["fa-passport"] = "\\f5ab", + ["fa-pen-fancy"] = "\\f5ac", + ["fa-pen-nib"] = "\\f5ad", + ["fa-pen-ruler"] = "\\f5ae", + ["fa-pencil-ruler"] = "\\f5ae", + ["fa-plane-arrival"] = "\\f5af", + ["fa-plane-departure"] = "\\f5b0", + ["fa-prescription"] = "\\f5b1", + ["fa-face-sad-cry"] = "\\f5b3", + ["fa-sad-cry"] = "\\f5b3", + ["fa-face-sad-tear"] = "\\f5b4", + ["fa-sad-tear"] = "\\f5b4", + ["fa-van-shuttle"] = "\\f5b6", + ["fa-shuttle-van"] = "\\f5b6", + ["fa-signature"] = "\\f5b7", + ["fa-face-smile-beam"] = "\\f5b8", + ["fa-smile-beam"] = "\\f5b8", + ["fa-solar-panel"] = "\\f5ba", + ["fa-spa"] = "\\f5bb", + ["fa-splotch"] = "\\f5bc", + ["fa-spray-can"] = "\\f5bd", + ["fa-stamp"] = "\\f5bf", + ["fa-star-half-stroke"] = "\\f5c0", + ["fa-star-half-alt"] = "\\f5c0", + ["fa-suitcase-rolling"] = "\\f5c1", + ["fa-face-surprise"] = "\\f5c2", + ["fa-surprise"] = "\\f5c2", + ["fa-swatchbook"] = "\\f5c3", + ["fa-person-swimming"] = "\\f5c4", + ["fa-swimmer"] = "\\f5c4", + ["fa-water-ladder"] = "\\f5c5", + ["fa-ladder-water"] = "\\f5c5", + ["fa-swimming-pool"] = "\\f5c5", + ["fa-droplet-slash"] = "\\f5c7", + ["fa-tint-slash"] = "\\f5c7", + ["fa-face-tired"] = "\\f5c8", + ["fa-tired"] = "\\f5c8", + ["fa-tooth"] = "\\f5c9", + ["fa-umbrella-beach"] = "\\f5ca", + ["fa-weight-hanging"] = "\\f5cd", + ["fa-wine-glass-empty"] = "\\f5ce", + ["fa-wine-glass-alt"] = "\\f5ce", + ["fa-spray-can-sparkles"] = "\\f5d0", + ["fa-air-freshener"] = "\\f5d0", + ["fa-apple-whole"] = "\\f5d1", + ["fa-apple-alt"] = "\\f5d1", + ["fa-atom"] = "\\f5d2", + ["fa-bone"] = "\\f5d7", + ["fa-book-open-reader"] = "\\f5da", + ["fa-book-reader"] = "\\f5da", + ["fa-brain"] = "\\f5dc", + ["fa-car-rear"] = "\\f5de", + ["fa-car-alt"] = "\\f5de", + ["fa-car-battery"] = "\\f5df", + ["fa-battery-car"] = "\\f5df", + ["fa-car-burst"] = "\\f5e1", + ["fa-car-crash"] = "\\f5e1", + ["fa-car-side"] = "\\f5e4", + ["fa-charging-station"] = "\\f5e7", + ["fa-diamond-turn-right"] = "\\f5eb", + ["fa-directions"] = "\\f5eb", + ["fa-draw-polygon"] = "\\f5ee", + ["fa-vector-polygon"] = "\\f5ee", + ["fa-laptop-code"] = "\\f5fc", + ["fa-layer-group"] = "\\f5fd", + ["fa-location-crosshairs"] = "\\f601", + ["fa-location"] = "\\f601", + ["fa-lungs"] = "\\f604", + ["fa-microscope"] = "\\f610", + ["fa-oil-can"] = "\\f613", + ["fa-poop"] = "\\f619", + ["fa-shapes"] = "\\f61f", + ["fa-triangle-circle-square"] = "\\f61f", + ["fa-star-of-life"] = "\\f621", + ["fa-gauge"] = "\\f624", + ["fa-dashboard"] = "\\f624", + ["fa-gauge-med"] = "\\f624", + ["fa-tachometer-alt-average"] = "\\f624", + ["fa-gauge-high"] = "\\f625", + ["fa-tachometer-alt"] = "\\f625", + ["fa-tachometer-alt-fast"] = "\\f625", + ["fa-gauge-simple"] = "\\f629", + ["fa-gauge-simple-med"] = "\\f629", + ["fa-tachometer-average"] = "\\f629", + ["fa-gauge-simple-high"] = "\\f62a", + ["fa-tachometer"] = "\\f62a", + ["fa-tachometer-fast"] = "\\f62a", + ["fa-teeth"] = "\\f62e", + ["fa-teeth-open"] = "\\f62f", + ["fa-masks-theater"] = "\\f630", + ["fa-theater-masks"] = "\\f630", + ["fa-traffic-light"] = "\\f637", + ["fa-truck-monster"] = "\\f63b", + ["fa-truck-pickup"] = "\\f63c", + ["fa-rectangle-ad"] = "\\f641", + ["fa-ad"] = "\\f641", + ["fa-ankh"] = "\\f644", + ["fa-book-bible"] = "\\f647", + ["fa-bible"] = "\\f647", + ["fa-business-time"] = "\\f64a", + ["fa-briefcase-clock"] = "\\f64a", + ["fa-city"] = "\\f64f", + ["fa-comment-dollar"] = "\\f651", + ["fa-comments-dollar"] = "\\f653", + ["fa-cross"] = "\\f654", + ["fa-dharmachakra"] = "\\f655", + ["fa-envelope-open-text"] = "\\f658", + ["fa-folder-minus"] = "\\f65d", + ["fa-folder-plus"] = "\\f65e", + ["fa-filter-circle-dollar"] = "\\f662", + ["fa-funnel-dollar"] = "\\f662", + ["fa-gopuram"] = "\\f664", + ["fa-hamsa"] = "\\f665", + ["fa-bahai"] = "\\f666", + ["fa-haykal"] = "\\f666", + ["fa-jedi"] = "\\f669", + ["fa-book-journal-whills"] = "\\f66a", + ["fa-journal-whills"] = "\\f66a", + ["fa-kaaba"] = "\\f66b", + ["fa-khanda"] = "\\f66d", + ["fa-landmark"] = "\\f66f", + ["fa-envelopes-bulk"] = "\\f674", + ["fa-mail-bulk"] = "\\f674", + ["fa-menorah"] = "\\f676", + ["fa-mosque"] = "\\f678", + ["fa-om"] = "\\f679", + ["fa-spaghetti-monster-flying"] = "\\f67b", + ["fa-pastafarianism"] = "\\f67b", + ["fa-peace"] = "\\f67c", + ["fa-place-of-worship"] = "\\f67f", + ["fa-square-poll-vertical"] = "\\f681", + ["fa-poll"] = "\\f681", + ["fa-square-poll-horizontal"] = "\\f682", + ["fa-poll-h"] = "\\f682", + ["fa-person-praying"] = "\\f683", + ["fa-pray"] = "\\f683", + ["fa-hands-praying"] = "\\f684", + ["fa-praying-hands"] = "\\f684", + ["fa-book-quran"] = "\\f687", + ["fa-quran"] = "\\f687", + ["fa-magnifying-glass-dollar"] = "\\f688", + ["fa-search-dollar"] = "\\f688", + ["fa-magnifying-glass-location"] = "\\f689", + ["fa-search-location"] = "\\f689", + ["fa-socks"] = "\\f696", + ["fa-square-root-variable"] = "\\f698", + ["fa-square-root-alt"] = "\\f698", + ["fa-star-and-crescent"] = "\\f699", + ["fa-star-of-david"] = "\\f69a", + ["fa-synagogue"] = "\\f69b", + ["fa-scroll-torah"] = "\\f6a0", + ["fa-torah"] = "\\f6a0", + ["fa-torii-gate"] = "\\f6a1", + ["fa-vihara"] = "\\f6a7", + ["fa-volume"] = "\\f6a8", + ["fa-volume-medium"] = "\\f6a8", + ["fa-volume-xmark"] = "\\f6a9", + ["fa-volume-mute"] = "\\f6a9", + ["fa-volume-times"] = "\\f6a9", + ["fa-yin-yang"] = "\\f6ad", + ["fa-blender-phone"] = "\\f6b6", + ["fa-book-skull"] = "\\f6b7", + ["fa-book-dead"] = "\\f6b7", + ["fa-campground"] = "\\f6bb", + ["fa-cat"] = "\\f6be", + ["fa-chair"] = "\\f6c0", + ["fa-cloud-moon"] = "\\f6c3", + ["fa-cloud-sun"] = "\\f6c4", + ["fa-cow"] = "\\f6c8", + ["fa-dice-d20"] = "\\f6cf", + ["fa-dice-d6"] = "\\f6d1", + ["fa-dog"] = "\\f6d3", + ["fa-dragon"] = "\\f6d5", + ["fa-drumstick-bite"] = "\\f6d7", + ["fa-dungeon"] = "\\f6d9", + ["fa-file-csv"] = "\\f6dd", + ["fa-hand-fist"] = "\\f6de", + ["fa-fist-raised"] = "\\f6de", + ["fa-ghost"] = "\\f6e2", + ["fa-hammer"] = "\\f6e3", + ["fa-hanukiah"] = "\\f6e6", + ["fa-hat-wizard"] = "\\f6e8", + ["fa-person-hiking"] = "\\f6ec", + ["fa-hiking"] = "\\f6ec", + ["fa-hippo"] = "\\f6ed", + ["fa-horse"] = "\\f6f0", + ["fa-house-chimney-crack"] = "\\f6f1", + ["fa-house-damage"] = "\\f6f1", + ["fa-hryvnia-sign"] = "\\f6f2", + ["fa-hryvnia"] = "\\f6f2", + ["fa-mask"] = "\\f6fa", + ["fa-mountain"] = "\\f6fc", + ["fa-network-wired"] = "\\f6ff", + ["fa-otter"] = "\\f700", + ["fa-ring"] = "\\f70b", + ["fa-person-running"] = "\\f70c", + ["fa-running"] = "\\f70c", + ["fa-scroll"] = "\\f70e", + ["fa-skull-crossbones"] = "\\f714", + ["fa-slash"] = "\\f715", + ["fa-spider"] = "\\f717", + ["fa-toilet-paper"] = "\\f71e", + ["fa-toilet-paper-alt"] = "\\f71e", + ["fa-toilet-paper-blank"] = "\\f71e", + ["fa-tractor"] = "\\f722", + ["fa-user-injured"] = "\\f728", + ["fa-vr-cardboard"] = "\\f729", + ["fa-wand-sparkles"] = "\\f72b", + ["fa-wind"] = "\\f72e", + ["fa-wine-bottle"] = "\\f72f", + ["fa-cloud-meatball"] = "\\f73b", + ["fa-cloud-moon-rain"] = "\\f73c", + ["fa-cloud-rain"] = "\\f73d", + ["fa-cloud-showers-heavy"] = "\\f740", + ["fa-cloud-sun-rain"] = "\\f743", + ["fa-democrat"] = "\\f747", + ["fa-flag-usa"] = "\\f74d", + ["fa-hurricane"] = "\\f751", + ["fa-landmark-dome"] = "\\f752", + ["fa-landmark-alt"] = "\\f752", + ["fa-meteor"] = "\\f753", + ["fa-person-booth"] = "\\f756", + ["fa-poo-storm"] = "\\f75a", + ["fa-poo-bolt"] = "\\f75a", + ["fa-rainbow"] = "\\f75b", + ["fa-republican"] = "\\f75e", + ["fa-smog"] = "\\f75f", + ["fa-temperature-high"] = "\\f769", + ["fa-temperature-low"] = "\\f76b", + ["fa-cloud-bolt"] = "\\f76c", + ["fa-thunderstorm"] = "\\f76c", + ["fa-tornado"] = "\\f76f", + ["fa-volcano"] = "\\f770", + ["fa-check-to-slot"] = "\\f772", + ["fa-vote-yea"] = "\\f772", + ["fa-water"] = "\\f773", + ["fa-baby"] = "\\f77c", + ["fa-baby-carriage"] = "\\f77d", + ["fa-carriage-baby"] = "\\f77d", + ["fa-biohazard"] = "\\f780", + ["fa-blog"] = "\\f781", + ["fa-calendar-day"] = "\\f783", + ["fa-calendar-week"] = "\\f784", + ["fa-candy-cane"] = "\\f786", + ["fa-carrot"] = "\\f787", + ["fa-cash-register"] = "\\f788", + ["fa-minimize"] = "\\f78c", + ["fa-compress-arrows-alt"] = "\\f78c", + ["fa-dumpster"] = "\\f793", + ["fa-dumpster-fire"] = "\\f794", + ["fa-ethernet"] = "\\f796", + ["fa-gifts"] = "\\f79c", + ["fa-champagne-glasses"] = "\\f79f", + ["fa-glass-cheers"] = "\\f79f", + ["fa-whiskey-glass"] = "\\f7a0", + ["fa-glass-whiskey"] = "\\f7a0", + ["fa-earth-europe"] = "\\f7a2", + ["fa-globe-europe"] = "\\f7a2", + ["fa-grip-lines"] = "\\f7a4", + ["fa-grip-lines-vertical"] = "\\f7a5", + ["fa-guitar"] = "\\f7a6", + ["fa-heart-crack"] = "\\f7a9", + ["fa-heart-broken"] = "\\f7a9", + ["fa-holly-berry"] = "\\f7aa", + ["fa-horse-head"] = "\\f7ab", + ["fa-icicles"] = "\\f7ad", + ["fa-igloo"] = "\\f7ae", + ["fa-mitten"] = "\\f7b5", + ["fa-mug-hot"] = "\\f7b6", + ["fa-radiation"] = "\\f7b9", + ["fa-circle-radiation"] = "\\f7ba", + ["fa-radiation-alt"] = "\\f7ba", + ["fa-restroom"] = "\\f7bd", + ["fa-satellite"] = "\\f7bf", + ["fa-satellite-dish"] = "\\f7c0", + ["fa-sd-card"] = "\\f7c2", + ["fa-sim-card"] = "\\f7c4", + ["fa-person-skating"] = "\\f7c5", + ["fa-skating"] = "\\f7c5", + ["fa-person-skiing"] = "\\f7c9", + ["fa-skiing"] = "\\f7c9", + ["fa-person-skiing-nordic"] = "\\f7ca", + ["fa-skiing-nordic"] = "\\f7ca", + ["fa-sleigh"] = "\\f7cc", + ["fa-comment-sms"] = "\\f7cd", + ["fa-sms"] = "\\f7cd", + ["fa-person-snowboarding"] = "\\f7ce", + ["fa-snowboarding"] = "\\f7ce", + ["fa-snowman"] = "\\f7d0", + ["fa-snowplow"] = "\\f7d2", + ["fa-tenge-sign"] = "\\f7d7", + ["fa-tenge"] = "\\f7d7", + ["fa-toilet"] = "\\f7d8", + ["fa-screwdriver-wrench"] = "\\f7d9", + ["fa-tools"] = "\\f7d9", + ["fa-cable-car"] = "\\f7da", + ["fa-tram"] = "\\f7da", + ["fa-fire-flame-curved"] = "\\f7e4", + ["fa-fire-alt"] = "\\f7e4", + ["fa-bacon"] = "\\f7e5", + ["fa-book-medical"] = "\\f7e6", + ["fa-bread-slice"] = "\\f7ec", + ["fa-cheese"] = "\\f7ef", + ["fa-house-chimney-medical"] = "\\f7f2", + ["fa-clinic-medical"] = "\\f7f2", + ["fa-clipboard-user"] = "\\f7f3", + ["fa-comment-medical"] = "\\f7f5", + ["fa-crutch"] = "\\f7f7", + ["fa-disease"] = "\\f7fa", + ["fa-egg"] = "\\f7fb", + ["fa-folder-tree"] = "\\f802", + ["fa-burger"] = "\\f805", + ["fa-hamburger"] = "\\f805", + ["fa-hand-middle-finger"] = "\\f806", + ["fa-helmet-safety"] = "\\f807", + ["fa-hard-hat"] = "\\f807", + ["fa-hat-hard"] = "\\f807", + ["fa-hospital-user"] = "\\f80d", + ["fa-hotdog"] = "\\f80f", + ["fa-ice-cream"] = "\\f810", + ["fa-laptop-medical"] = "\\f812", + ["fa-pager"] = "\\f815", + ["fa-pepper-hot"] = "\\f816", + ["fa-pizza-slice"] = "\\f818", + ["fa-sack-dollar"] = "\\f81d", + ["fa-book-tanakh"] = "\\f827", + ["fa-tanakh"] = "\\f827", + ["fa-bars-progress"] = "\\f828", + ["fa-tasks-alt"] = "\\f828", + ["fa-trash-arrow-up"] = "\\f829", + ["fa-trash-restore"] = "\\f829", + ["fa-trash-can-arrow-up"] = "\\f82a", + ["fa-trash-restore-alt"] = "\\f82a", + ["fa-user-nurse"] = "\\f82f", + ["fa-wave-square"] = "\\f83e", + ["fa-person-biking"] = "\\f84a", + ["fa-biking"] = "\\f84a", + ["fa-border-all"] = "\\f84c", + ["fa-border-none"] = "\\f850", + ["fa-border-top-left"] = "\\f853", + ["fa-border-style"] = "\\f853", + ["fa-person-digging"] = "\\f85e", + ["fa-digging"] = "\\f85e", + ["fa-fan"] = "\\f863", + ["fa-icons"] = "\\f86d", + ["fa-heart-music-camera-bolt"] = "\\f86d", + ["fa-phone-flip"] = "\\f879", + ["fa-phone-alt"] = "\\f879", + ["fa-square-phone-flip"] = "\\f87b", + ["fa-phone-square-alt"] = "\\f87b", + ["fa-photo-film"] = "\\f87c", + ["fa-photo-video"] = "\\f87c", + ["fa-text-slash"] = "\\f87d", + ["fa-remove-format"] = "\\f87d", + ["fa-arrow-down-z-a"] = "\\f881", + ["fa-sort-alpha-desc"] = "\\f881", + ["fa-sort-alpha-down-alt"] = "\\f881", + ["fa-arrow-up-z-a"] = "\\f882", + ["fa-sort-alpha-up-alt"] = "\\f882", + ["fa-arrow-down-short-wide"] = "\\f884", + ["fa-sort-amount-desc"] = "\\f884", + ["fa-sort-amount-down-alt"] = "\\f884", + ["fa-arrow-up-short-wide"] = "\\f885", + ["fa-sort-amount-up-alt"] = "\\f885", + ["fa-arrow-down-9-1"] = "\\f886", + ["fa-sort-numeric-desc"] = "\\f886", + ["fa-sort-numeric-down-alt"] = "\\f886", + ["fa-arrow-up-9-1"] = "\\f887", + ["fa-sort-numeric-up-alt"] = "\\f887", + ["fa-spell-check"] = "\\f891", + ["fa-voicemail"] = "\\f897", + ["fa-hat-cowboy"] = "\\f8c0", + ["fa-hat-cowboy-side"] = "\\f8c1", + ["fa-computer-mouse"] = "\\f8cc", + ["fa-mouse"] = "\\f8cc", + ["fa-radio"] = "\\f8d7", + ["fa-record-vinyl"] = "\\f8d9", + ["fa-walkie-talkie"] = "\\f8ef", + ["fa-caravan"] = "\\f8ff", + ["fa-firefox-browser"] = "\\e007", + ["fa-ideal"] = "\\e013", + ["fa-microblog"] = "\\e01a", + ["fa-square-pied-piper"] = "\\e01e", + ["fa-pied-piper-square"] = "\\e01e", + ["fa-unity"] = "\\e049", + ["fa-dailymotion"] = "\\e052", + ["fa-square-instagram"] = "\\e055", + ["fa-instagram-square"] = "\\e055", + ["fa-mixer"] = "\\e056", + ["fa-shopify"] = "\\e057", + ["fa-deezer"] = "\\e077", + ["fa-edge-legacy"] = "\\e078", + ["fa-google-pay"] = "\\e079", + ["fa-rust"] = "\\e07a", + ["fa-tiktok"] = "\\e07b", + ["fa-unsplash"] = "\\e07c", + ["fa-cloudflare"] = "\\e07d", + ["fa-guilded"] = "\\e07e", + ["fa-hive"] = "\\e07f", + ["fa-42-group"] = "\\e080", + ["fa-innosoft"] = "\\e080", + ["fa-instalod"] = "\\e081", + ["fa-octopus-deploy"] = "\\e082", + ["fa-perbyte"] = "\\e083", + ["fa-uncharted"] = "\\e084", + ["fa-watchman-monitoring"] = "\\e087", + ["fa-wodu"] = "\\e088", + ["fa-wirsindhandwerk"] = "\\e2d0", + ["fa-wsh"] = "\\e2d0", + ["fa-bots"] = "\\e340", + ["fa-cmplid"] = "\\e360", + ["fa-bilibili"] = "\\e3d9", + ["fa-golang"] = "\\e40f", + ["fa-pix"] = "\\e43a", + ["fa-sitrox"] = "\\e44a", + ["fa-hashnode"] = "\\e499", + ["fa-meta"] = "\\e49b", + ["fa-padlet"] = "\\e4a0", + ["fa-nfc-directional"] = "\\e530", + ["fa-nfc-symbol"] = "\\e531", + ["fa-screenpal"] = "\\e570", + ["fa-space-awesome"] = "\\e5ac", + ["fa-square-font-awesome"] = "\\e5ad", + ["fa-square-gitlab"] = "\\e5ae", + ["fa-gitlab-square"] = "\\e5ae", + ["fa-odysee"] = "\\e5c6", + ["fa-stubber"] = "\\e5c7", + ["fa-debian"] = "\\e60b", + ["fa-shoelace"] = "\\e60c", + ["fa-threads"] = "\\e618", + ["fa-square-threads"] = "\\e619", + ["fa-square-x-twitter"] = "\\e61a", + ["fa-x-twitter"] = "\\e61b", + ["fa-opensuse"] = "\\e62b", + ["fa-letterboxd"] = "\\e62d", + ["fa-square-letterboxd"] = "\\e62e", + ["fa-mintbit"] = "\\e62f", + ["fa-google-scholar"] = "\\e63b", + ["fa-brave"] = "\\e63c", + ["fa-brave-reverse"] = "\\e63d", + ["fa-pixiv"] = "\\e640", + ["fa-upwork"] = "\\e641", + ["fa-webflow"] = "\\e65c", + ["fa-signal-messenger"] = "\\e663", + ["fa-bluesky"] = "\\e671", + ["fa-jxl"] = "\\e67b", + ["fa-square-upwork"] = "\\e67c", + ["fa-square-web-awesome"] = "\\e683", + ["fa-square-web-awesome-stroke"] = "\\e684", + ["fa-dart-lang"] = "\\e693", + ["fa-flutter"] = "\\e694", + ["fa-files-pinwheel"] = "\\e69f", + ["fa-css"] = "\\e6a2", + ["fa-square-bluesky"] = "\\e6a3", + ["fa-openai"] = "\\e7cf", + ["fa-square-linkedin"] = "\\e7d0", + ["fa-cash-app"] = "\\e7d4", + ["fa-disqus"] = "\\e7d5", + ["fa-eleventy"] = "\\e7d6", + ["fa-11ty"] = "\\e7d6", + ["fa-kakao-talk"] = "\\e7d7", + ["fa-linktree"] = "\\e7d8", + ["fa-notion"] = "\\e7d9", + ["fa-pandora"] = "\\e7da", + ["fa-pixelfed"] = "\\e7db", + ["fa-tidal"] = "\\e7dc", + ["fa-vsco"] = "\\e7dd", + ["fa-w3c"] = "\\e7de", + ["fa-lumon"] = "\\e7e2", + ["fa-lumon-drop"] = "\\e7e3", + ["fa-square-figma"] = "\\e7e4", + ["fa-tex"] = "\\e7ff", + ["fa-duolingo"] = "\\e812", + ["fa-supportnow"] = "\\e833", + ["fa-tor-browser"] = "\\e838", + ["fa-typescript"] = "\\e840", + ["fa-square-deskpro"] = "\\e844", + ["fa-circle-zulip"] = "\\e851", + ["fa-julia"] = "\\e852", + ["fa-zulip"] = "\\e853", + ["fa-unison"] = "\\e854", + ["fa-board-game-geek"] = "\\e855", + ["fa-bgg"] = "\\e855", + ["fa-ko-fi"] = "\\e856", + ["fa-kubernetes"] = "\\e857", + ["fa-postgresql"] = "\\e858", + ["fa-scaleway"] = "\\e859", + ["fa-venmo"] = "\\e85a", + ["fa-venmo-v"] = "\\e85b", + ["fa-unreal-engine"] = "\\e85c", + ["fa-globaleaks"] = "\\e85d", + ["fa-solana"] = "\\e85e", + ["fa-threema"] = "\\e85f", + ["fa-forgejo"] = "\\e860", + ["fa-claude"] = "\\e861", + ["fa-gitee"] = "\\e863", + ["fa-xmpp"] = "\\e864", + ["fa-fediverse"] = "\\e865", + ["fa-tailwind-css"] = "\\e866", + ["fa-arch-linux"] = "\\e867", + ["fa-svelte"] = "\\e868", + ["fa-hugging-face"] = "\\e869", + ["fa-leetcode"] = "\\e86a", + ["fa-openstreetmap"] = "\\e86b", + ["fa-ultralytics"] = "\\e86d", + ["fa-ultralytics-hub"] = "\\e86e", + ["fa-ultralytics-yolo"] = "\\e86f", + ["fa-obsidian"] = "\\e879", + ["fa-zoom"] = "\\e87b", + ["fa-vim"] = "\\e88a", + ["fa-symfonycasts"] = "\\e8ab", + ["fa-square-twitter"] = "\\f081", + ["fa-twitter-square"] = "\\f081", + ["fa-square-facebook"] = "\\f082", + ["fa-facebook-square"] = "\\f082", + ["fa-linkedin"] = "\\f08c", + ["fa-square-github"] = "\\f092", + ["fa-github-square"] = "\\f092", + ["fa-twitter"] = "\\f099", + ["fa-facebook"] = "\\f09a", + ["fa-github"] = "\\f09b", + ["fa-pinterest"] = "\\f0d2", + ["fa-square-pinterest"] = "\\f0d3", + ["fa-pinterest-square"] = "\\f0d3", + ["fa-square-google-plus"] = "\\f0d4", + ["fa-google-plus-square"] = "\\f0d4", + ["fa-google-plus-g"] = "\\f0d5", + ["fa-linkedin-in"] = "\\f0e1", + ["fa-github-alt"] = "\\f113", + ["fa-maxcdn"] = "\\f136", + ["fa-html5"] = "\\f13b", + ["fa-css3"] = "\\f13c", + ["fa-btc"] = "\\f15a", + ["fa-youtube"] = "\\f167", + ["fa-xing"] = "\\f168", + ["fa-square-xing"] = "\\f169", + ["fa-xing-square"] = "\\f169", + ["fa-dropbox"] = "\\f16b", + ["fa-stack-overflow"] = "\\f16c", + ["fa-instagram"] = "\\f16d", + ["fa-flickr"] = "\\f16e", + ["fa-adn"] = "\\f170", + ["fa-bitbucket"] = "\\f171", + ["fa-tumblr"] = "\\f173", + ["fa-square-tumblr"] = "\\f174", + ["fa-tumblr-square"] = "\\f174", + ["fa-apple"] = "\\f179", + ["fa-windows"] = "\\f17a", + ["fa-android"] = "\\f17b", + ["fa-linux"] = "\\f17c", + ["fa-dribbble"] = "\\f17d", + ["fa-skype"] = "\\f17e", + ["fa-foursquare"] = "\\f180", + ["fa-trello"] = "\\f181", + ["fa-gratipay"] = "\\f184", + ["fa-vk"] = "\\f189", + ["fa-weibo"] = "\\f18a", + ["fa-renren"] = "\\f18b", + ["fa-pagelines"] = "\\f18c", + ["fa-stack-exchange"] = "\\f18d", + ["fa-square-vimeo"] = "\\f194", + ["fa-vimeo-square"] = "\\f194", + ["fa-slack"] = "\\f198", + ["fa-slack-hash"] = "\\f198", + ["fa-wordpress"] = "\\f19a", + ["fa-openid"] = "\\f19b", + ["fa-yahoo"] = "\\f19e", + ["fa-google"] = "\\f1a0", + ["fa-reddit"] = "\\f1a1", + ["fa-square-reddit"] = "\\f1a2", + ["fa-reddit-square"] = "\\f1a2", + ["fa-stumbleupon-circle"] = "\\f1a3", + ["fa-stumbleupon"] = "\\f1a4", + ["fa-delicious"] = "\\f1a5", + ["fa-digg"] = "\\f1a6", + ["fa-pied-piper-pp"] = "\\f1a7", + ["fa-pied-piper-alt"] = "\\f1a8", + ["fa-drupal"] = "\\f1a9", + ["fa-joomla"] = "\\f1aa", + ["fa-behance"] = "\\f1b4", + ["fa-square-behance"] = "\\f1b5", + ["fa-behance-square"] = "\\f1b5", + ["fa-steam"] = "\\f1b6", + ["fa-square-steam"] = "\\f1b7", + ["fa-steam-square"] = "\\f1b7", + ["fa-spotify"] = "\\f1bc", + ["fa-deviantart"] = "\\f1bd", + ["fa-soundcloud"] = "\\f1be", + ["fa-vine"] = "\\f1ca", + ["fa-codepen"] = "\\f1cb", + ["fa-jsfiddle"] = "\\f1cc", + ["fa-rebel"] = "\\f1d0", + ["fa-empire"] = "\\f1d1", + ["fa-square-git"] = "\\f1d2", + ["fa-git-square"] = "\\f1d2", + ["fa-git"] = "\\f1d3", + ["fa-hacker-news"] = "\\f1d4", + ["fa-tencent-weibo"] = "\\f1d5", + ["fa-qq"] = "\\f1d6", + ["fa-weixin"] = "\\f1d7", + ["fa-slideshare"] = "\\f1e7", + ["fa-twitch"] = "\\f1e8", + ["fa-yelp"] = "\\f1e9", + ["fa-paypal"] = "\\f1ed", + ["fa-google-wallet"] = "\\f1ee", + ["fa-cc-visa"] = "\\f1f0", + ["fa-cc-mastercard"] = "\\f1f1", + ["fa-cc-discover"] = "\\f1f2", + ["fa-cc-amex"] = "\\f1f3", + ["fa-cc-paypal"] = "\\f1f4", + ["fa-cc-stripe"] = "\\f1f5", + ["fa-lastfm"] = "\\f202", + ["fa-square-lastfm"] = "\\f203", + ["fa-lastfm-square"] = "\\f203", + ["fa-ioxhost"] = "\\f208", + ["fa-angellist"] = "\\f209", + ["fa-buysellads"] = "\\f20d", + ["fa-connectdevelop"] = "\\f20e", + ["fa-dashcube"] = "\\f210", + ["fa-forumbee"] = "\\f211", + ["fa-leanpub"] = "\\f212", + ["fa-sellsy"] = "\\f213", + ["fa-shirtsinbulk"] = "\\f214", + ["fa-simplybuilt"] = "\\f215", + ["fa-skyatlas"] = "\\f216", + ["fa-pinterest-p"] = "\\f231", + ["fa-whatsapp"] = "\\f232", + ["fa-viacoin"] = "\\f237", + ["fa-medium"] = "\\f23a", + ["fa-medium-m"] = "\\f23a", + ["fa-y-combinator"] = "\\f23b", + ["fa-optin-monster"] = "\\f23c", + ["fa-opencart"] = "\\f23d", + ["fa-expeditedssl"] = "\\f23e", + ["fa-cc-jcb"] = "\\f24b", + ["fa-cc-diners-club"] = "\\f24c", + ["fa-creative-commons"] = "\\f25e", + ["fa-gg"] = "\\f260", + ["fa-gg-circle"] = "\\f261", + ["fa-odnoklassniki"] = "\\f263", + ["fa-square-odnoklassniki"] = "\\f264", + ["fa-odnoklassniki-square"] = "\\f264", + ["fa-get-pocket"] = "\\f265", + ["fa-wikipedia-w"] = "\\f266", + ["fa-safari"] = "\\f267", + ["fa-chrome"] = "\\f268", + ["fa-firefox"] = "\\f269", + ["fa-opera"] = "\\f26a", + ["fa-internet-explorer"] = "\\f26b", + ["fa-contao"] = "\\f26d", + ["fa-500px"] = "\\f26e", + ["fa-amazon"] = "\\f270", + ["fa-houzz"] = "\\f27c", + ["fa-vimeo-v"] = "\\f27d", + ["fa-black-tie"] = "\\f27e", + ["fa-fonticons"] = "\\f280", + ["fa-reddit-alien"] = "\\f281", + ["fa-edge"] = "\\f282", + ["fa-codiepie"] = "\\f284", + ["fa-modx"] = "\\f285", + ["fa-fort-awesome"] = "\\f286", + ["fa-usb"] = "\\f287", + ["fa-product-hunt"] = "\\f288", + ["fa-mixcloud"] = "\\f289", + ["fa-scribd"] = "\\f28a", + ["fa-bluetooth"] = "\\f293", + ["fa-bluetooth-b"] = "\\f294", + ["fa-gitlab"] = "\\f296", + ["fa-wpbeginner"] = "\\f297", + ["fa-wpforms"] = "\\f298", + ["fa-envira"] = "\\f299", + ["fa-glide"] = "\\f2a5", + ["fa-glide-g"] = "\\f2a6", + ["fa-viadeo"] = "\\f2a9", + ["fa-square-viadeo"] = "\\f2aa", + ["fa-viadeo-square"] = "\\f2aa", + ["fa-snapchat"] = "\\f2ab", + ["fa-snapchat-ghost"] = "\\f2ab", + ["fa-square-snapchat"] = "\\f2ad", + ["fa-snapchat-square"] = "\\f2ad", + ["fa-pied-piper"] = "\\f2ae", + ["fa-first-order"] = "\\f2b0", + ["fa-yoast"] = "\\f2b1", + ["fa-themeisle"] = "\\f2b2", + ["fa-google-plus"] = "\\f2b3", + ["fa-linode"] = "\\f2b8", + ["fa-quora"] = "\\f2c4", + ["fa-free-code-camp"] = "\\f2c5", + ["fa-telegram"] = "\\f2c6", + ["fa-telegram-plane"] = "\\f2c6", + ["fa-bandcamp"] = "\\f2d5", + ["fa-grav"] = "\\f2d6", + ["fa-etsy"] = "\\f2d7", + ["fa-imdb"] = "\\f2d8", + ["fa-ravelry"] = "\\f2d9", + ["fa-sellcast"] = "\\f2da", + ["fa-superpowers"] = "\\f2dd", + ["fa-wpexplorer"] = "\\f2de", + ["fa-meetup"] = "\\f2e0", + ["fa-square-font-awesome-stroke"] = "\\f35c", + ["fa-font-awesome-alt"] = "\\f35c", + ["fa-accessible-icon"] = "\\f368", + ["fa-accusoft"] = "\\f369", + ["fa-adversal"] = "\\f36a", + ["fa-affiliatetheme"] = "\\f36b", + ["fa-algolia"] = "\\f36c", + ["fa-amilia"] = "\\f36d", + ["fa-angrycreative"] = "\\f36e", + ["fa-app-store"] = "\\f36f", + ["fa-app-store-ios"] = "\\f370", + ["fa-apper"] = "\\f371", + ["fa-asymmetrik"] = "\\f372", + ["fa-audible"] = "\\f373", + ["fa-avianex"] = "\\f374", + ["fa-aws"] = "\\f375", + ["fa-bimobject"] = "\\f378", + ["fa-bitcoin"] = "\\f379", + ["fa-bity"] = "\\f37a", + ["fa-blackberry"] = "\\f37b", + ["fa-blogger"] = "\\f37c", + ["fa-blogger-b"] = "\\f37d", + ["fa-buromobelexperte"] = "\\f37f", + ["fa-centercode"] = "\\f380", + ["fa-cloudscale"] = "\\f383", + ["fa-cloudsmith"] = "\\f384", + ["fa-cloudversify"] = "\\f385", + ["fa-cpanel"] = "\\f388", + ["fa-css3-alt"] = "\\f38b", + ["fa-cuttlefish"] = "\\f38c", + ["fa-d-and-d"] = "\\f38d", + ["fa-deploydog"] = "\\f38e", + ["fa-deskpro"] = "\\f38f", + ["fa-digital-ocean"] = "\\f391", + ["fa-discord"] = "\\f392", + ["fa-discourse"] = "\\f393", + ["fa-dochub"] = "\\f394", + ["fa-docker"] = "\\f395", + ["fa-draft2digital"] = "\\f396", + ["fa-square-dribbble"] = "\\f397", + ["fa-dribbble-square"] = "\\f397", + ["fa-dyalog"] = "\\f399", + ["fa-earlybirds"] = "\\f39a", + ["fa-erlang"] = "\\f39d", + ["fa-facebook-f"] = "\\f39e", + ["fa-facebook-messenger"] = "\\f39f", + ["fa-firstdraft"] = "\\f3a1", + ["fa-fonticons-fi"] = "\\f3a2", + ["fa-fort-awesome-alt"] = "\\f3a3", + ["fa-freebsd"] = "\\f3a4", + ["fa-gitkraken"] = "\\f3a6", + ["fa-gofore"] = "\\f3a7", + ["fa-goodreads"] = "\\f3a8", + ["fa-goodreads-g"] = "\\f3a9", + ["fa-google-drive"] = "\\f3aa", + ["fa-google-play"] = "\\f3ab", + ["fa-gripfire"] = "\\f3ac", + ["fa-grunt"] = "\\f3ad", + ["fa-gulp"] = "\\f3ae", + ["fa-square-hacker-news"] = "\\f3af", + ["fa-hacker-news-square"] = "\\f3af", + ["fa-hire-a-helper"] = "\\f3b0", + ["fa-hotjar"] = "\\f3b1", + ["fa-hubspot"] = "\\f3b2", + ["fa-itunes"] = "\\f3b4", + ["fa-itunes-note"] = "\\f3b5", + ["fa-jenkins"] = "\\f3b6", + ["fa-joget"] = "\\f3b7", + ["fa-js"] = "\\f3b8", + ["fa-square-js"] = "\\f3b9", + ["fa-js-square"] = "\\f3b9", + ["fa-keycdn"] = "\\f3ba", + ["fa-kickstarter"] = "\\f3bb", + ["fa-square-kickstarter"] = "\\f3bb", + ["fa-kickstarter-k"] = "\\f3bc", + ["fa-laravel"] = "\\f3bd", + ["fa-line"] = "\\f3c0", + ["fa-lyft"] = "\\f3c3", + ["fa-magento"] = "\\f3c4", + ["fa-medapps"] = "\\f3c6", + ["fa-medrt"] = "\\f3c8", + ["fa-microsoft"] = "\\f3ca", + ["fa-mix"] = "\\f3cb", + ["fa-mizuni"] = "\\f3cc", + ["fa-monero"] = "\\f3d0", + ["fa-napster"] = "\\f3d2", + ["fa-node-js"] = "\\f3d3", + ["fa-npm"] = "\\f3d4", + ["fa-ns8"] = "\\f3d5", + ["fa-nutritionix"] = "\\f3d6", + ["fa-page4"] = "\\f3d7", + ["fa-palfed"] = "\\f3d8", + ["fa-patreon"] = "\\f3d9", + ["fa-periscope"] = "\\f3da", + ["fa-phabricator"] = "\\f3db", + ["fa-phoenix-framework"] = "\\f3dc", + ["fa-playstation"] = "\\f3df", + ["fa-pushed"] = "\\f3e1", + ["fa-python"] = "\\f3e2", + ["fa-red-river"] = "\\f3e3", + ["fa-wpressr"] = "\\f3e4", + ["fa-rendact"] = "\\f3e4", + ["fa-replyd"] = "\\f3e6", + ["fa-resolving"] = "\\f3e7", + ["fa-rocketchat"] = "\\f3e8", + ["fa-rockrms"] = "\\f3e9", + ["fa-schlix"] = "\\f3ea", + ["fa-searchengin"] = "\\f3eb", + ["fa-servicestack"] = "\\f3ec", + ["fa-sistrix"] = "\\f3ee", + ["fa-speakap"] = "\\f3f3", + ["fa-staylinked"] = "\\f3f5", + ["fa-steam-symbol"] = "\\f3f6", + ["fa-sticker-mule"] = "\\f3f7", + ["fa-studiovinari"] = "\\f3f8", + ["fa-supple"] = "\\f3f9", + ["fa-uber"] = "\\f402", + ["fa-uikit"] = "\\f403", + ["fa-uniregistry"] = "\\f404", + ["fa-untappd"] = "\\f405", + ["fa-ussunnah"] = "\\f407", + ["fa-vaadin"] = "\\f408", + ["fa-viber"] = "\\f409", + ["fa-vimeo"] = "\\f40a", + ["fa-vnv"] = "\\f40b", + ["fa-square-whatsapp"] = "\\f40c", + ["fa-whatsapp-square"] = "\\f40c", + ["fa-whmcs"] = "\\f40d", + ["fa-wordpress-simple"] = "\\f411", + ["fa-xbox"] = "\\f412", + ["fa-yandex"] = "\\f413", + ["fa-yandex-international"] = "\\f414", + ["fa-apple-pay"] = "\\f415", + ["fa-cc-apple-pay"] = "\\f416", + ["fa-fly"] = "\\f417", + ["fa-node"] = "\\f419", + ["fa-osi"] = "\\f41a", + ["fa-react"] = "\\f41b", + ["fa-autoprefixer"] = "\\f41c", + ["fa-less"] = "\\f41d", + ["fa-sass"] = "\\f41e", + ["fa-vuejs"] = "\\f41f", + ["fa-angular"] = "\\f420", + ["fa-aviato"] = "\\f421", + ["fa-ember"] = "\\f423", + ["fa-gitter"] = "\\f426", + ["fa-hooli"] = "\\f427", + ["fa-strava"] = "\\f428", + ["fa-stripe"] = "\\f429", + ["fa-stripe-s"] = "\\f42a", + ["fa-typo3"] = "\\f42b", + ["fa-amazon-pay"] = "\\f42c", + ["fa-cc-amazon-pay"] = "\\f42d", + ["fa-ethereum"] = "\\f42e", + ["fa-korvue"] = "\\f42f", + ["fa-elementor"] = "\\f430", + ["fa-square-youtube"] = "\\f431", + ["fa-youtube-square"] = "\\f431", + ["fa-flipboard"] = "\\f44d", + ["fa-hips"] = "\\f452", + ["fa-php"] = "\\f457", + ["fa-quinscape"] = "\\f459", + ["fa-readme"] = "\\f4d5", + ["fa-java"] = "\\f4e4", + ["fa-pied-piper-hat"] = "\\f4e5", + ["fa-creative-commons-by"] = "\\f4e7", + ["fa-creative-commons-nc"] = "\\f4e8", + ["fa-creative-commons-nc-eu"] = "\\f4e9", + ["fa-creative-commons-nc-jp"] = "\\f4ea", + ["fa-creative-commons-nd"] = "\\f4eb", + ["fa-creative-commons-pd"] = "\\f4ec", + ["fa-creative-commons-pd-alt"] = "\\f4ed", + ["fa-creative-commons-remix"] = "\\f4ee", + ["fa-creative-commons-sa"] = "\\f4ef", + ["fa-creative-commons-sampling"] = "\\f4f0", + ["fa-creative-commons-sampling-plus"] = "\\f4f1", + ["fa-creative-commons-share"] = "\\f4f2", + ["fa-creative-commons-zero"] = "\\f4f3", + ["fa-ebay"] = "\\f4f4", + ["fa-keybase"] = "\\f4f5", + ["fa-mastodon"] = "\\f4f6", + ["fa-r-project"] = "\\f4f7", + ["fa-researchgate"] = "\\f4f8", + ["fa-teamspeak"] = "\\f4f9", + ["fa-first-order-alt"] = "\\f50a", + ["fa-fulcrum"] = "\\f50b", + ["fa-galactic-republic"] = "\\f50c", + ["fa-galactic-senate"] = "\\f50d", + ["fa-jedi-order"] = "\\f50e", + ["fa-mandalorian"] = "\\f50f", + ["fa-old-republic"] = "\\f510", + ["fa-phoenix-squadron"] = "\\f511", + ["fa-sith"] = "\\f512", + ["fa-trade-federation"] = "\\f513", + ["fa-wolf-pack-battalion"] = "\\f514", + ["fa-hornbill"] = "\\f592", + ["fa-mailchimp"] = "\\f59e", + ["fa-megaport"] = "\\f5a3", + ["fa-nimblr"] = "\\f5a8", + ["fa-rev"] = "\\f5b2", + ["fa-shopware"] = "\\f5b5", + ["fa-squarespace"] = "\\f5be", + ["fa-themeco"] = "\\f5c6", + ["fa-weebly"] = "\\f5cc", + ["fa-wix"] = "\\f5cf", + ["fa-ello"] = "\\f5f1", + ["fa-hackerrank"] = "\\f5f7", + ["fa-kaggle"] = "\\f5fa", + ["fa-markdown"] = "\\f60f", + ["fa-neos"] = "\\f612", + ["fa-zhihu"] = "\\f63f", + ["fa-alipay"] = "\\f642", + ["fa-the-red-yeti"] = "\\f69d", + ["fa-critical-role"] = "\\f6c9", + ["fa-d-and-d-beyond"] = "\\f6ca", + ["fa-dev"] = "\\f6cc", + ["fa-fantasy-flight-games"] = "\\f6dc", + ["fa-wizards-of-the-coast"] = "\\f730", + ["fa-think-peaks"] = "\\f731", + ["fa-reacteurope"] = "\\f75d", + ["fa-artstation"] = "\\f77a", + ["fa-atlassian"] = "\\f77b", + ["fa-canadian-maple-leaf"] = "\\f785", + ["fa-centos"] = "\\f789", + ["fa-confluence"] = "\\f78d", + ["fa-dhl"] = "\\f790", + ["fa-diaspora"] = "\\f791", + ["fa-fedex"] = "\\f797", + ["fa-fedora"] = "\\f798", + ["fa-figma"] = "\\f799", + ["fa-intercom"] = "\\f7af", + ["fa-invision"] = "\\f7b0", + ["fa-jira"] = "\\f7b1", + ["fa-mendeley"] = "\\f7b3", + ["fa-raspberry-pi"] = "\\f7bb", + ["fa-redhat"] = "\\f7bc", + ["fa-sketch"] = "\\f7c6", + ["fa-sourcetree"] = "\\f7d3", + ["fa-suse"] = "\\f7d6", + ["fa-ubuntu"] = "\\f7df", + ["fa-ups"] = "\\f7e0", + ["fa-usps"] = "\\f7e1", + ["fa-yarn"] = "\\f7e3", + ["fa-airbnb"] = "\\f834", + ["fa-battle-net"] = "\\f835", + ["fa-bootstrap"] = "\\f836", + ["fa-buffer"] = "\\f837", + ["fa-chromecast"] = "\\f838", + ["fa-evernote"] = "\\f839", + ["fa-itch-io"] = "\\f83a", + ["fa-salesforce"] = "\\f83b", + ["fa-speaker-deck"] = "\\f83c", + ["fa-symfony"] = "\\f83d", + ["fa-waze"] = "\\f83f", + ["fa-yammer"] = "\\f840", + ["fa-git-alt"] = "\\f841", + ["fa-stackpath"] = "\\f842", + ["fa-cotton-bureau"] = "\\f89e", + ["fa-buy-n-large"] = "\\f8a6", + ["fa-mdb"] = "\\f8ca", + ["fa-orcid"] = "\\f8d2", + ["fa-swift"] = "\\f8e1", + ["fa-umbraco"] = "\\f8e8", +} + +-- Function to get Unicode value for a FontAwesome icon name +local function fa_unicode(icon_name) + return fa_icons[icon_name] or nil +end + +return { + fa_unicode = fa_unicode +} diff --git a/quarto/_quarto.yml b/quarto/_quarto.yml index 2535ef9..b4a8dea 100644 --- a/quarto/_quarto.yml +++ b/quarto/_quarto.yml @@ -23,7 +23,7 @@ book: pinned: false sidebar: collapse-level: 1 - page-footer: "Copyright 2022-25, John Verzani" + page-footer: "Copyright 2022-26, John Verzani" chapters: - index.qmd - part: basics.qmd @@ -52,9 +52,9 @@ book: chapters: - limits/limits.qmd - limits/limits_extensions.qmd - - limits/sequences_series.qmd - limits/continuity.qmd - limits/intermediate_value_theorem.qmd + - limits/sequences_series.qmd - part: derivatives.qmd chapters: @@ -77,15 +77,16 @@ book: chapters: - integrals/area.qmd - integrals/ftc.qmd + - integrals/numeric_integrals.qmd - integrals/substitution.qmd - integrals/integration_by_parts.qmd - integrals/partial_fractions.qmd + - integrals/area_between_curves.qmd - integrals/improper_integrals.qmd - integrals/mean_value_theorem.qmd - - integrals/area_between_curves.qmd - integrals/center_of_mass.qmd - - integrals/volumes_slice.qmd - integrals/arc_length.qmd + - integrals/volumes_slice.qmd - integrals/surface_area.qmd - integrals/orthogonal_polynomials.qmd - integrals/twelve-qs.qmd @@ -97,24 +98,24 @@ book: - ODEs/solve.qmd - ODEs/differential_equations.qmd - - part: differentiable_vector_calculus.qmd - chapters: - - differentiable_vector_calculus/polar_coordinates.qmd - - differentiable_vector_calculus/vectors.qmd - - differentiable_vector_calculus/vector_valued_functions.qmd - - differentiable_vector_calculus/scalar_functions.qmd - - differentiable_vector_calculus/scalar_functions_applications.qmd - - differentiable_vector_calculus/vector_fields.qmd - - differentiable_vector_calculus/matrix_calculus_notes.qmd - - differentiable_vector_calculus/plots_plotting.qmd + # - part: differentiable_vector_calculus.qmd + # chapters: + # - differentiable_vector_calculus/polar_coordinates.qmd + # - differentiable_vector_calculus/vectors.qmd + # - differentiable_vector_calculus/vector_valued_functions.qmd + # - differentiable_vector_calculus/scalar_functions.qmd + # - differentiable_vector_calculus/scalar_functions_applications.qmd + # - differentiable_vector_calculus/vector_fields.qmd + # - differentiable_vector_calculus/matrix_calculus_notes.qmd + # - differentiable_vector_calculus/plots_plotting.qmd - - part: integral_vector_calculus.qmd - chapters: - - integral_vector_calculus/double_triple_integrals.qmd - - integral_vector_calculus/line_integrals.qmd - - integral_vector_calculus/div_grad_curl.qmd - - integral_vector_calculus/stokes_theorem.qmd - - integral_vector_calculus/review.qmd + # - part: integral_vector_calculus.qmd + # chapters: + # - integral_vector_calculus/double_triple_integrals.qmd + # - integral_vector_calculus/line_integrals.qmd + # - integral_vector_calculus/div_grad_curl.qmd + # - integral_vector_calculus/stokes_theorem.qmd + # - integral_vector_calculus/review.qmd - part: alternatives.qmd chapters: @@ -145,11 +146,12 @@ website: format: html: theme: - light: lux # spacelab # lux # sketchy # cosmo # https://quarto.org/docs/output-formats/html-themes.html + light: spacelab # cosmo #flatly #lux # spacelab # lux # sketchy # cosmo # https://quarto.org/docs/output-formats/html-themes.html dark: darkly number-depth: 3 toc-depth: 3 link-external-newwindow: true + css: styles.css # pdf: # documentclass: scrbook # classoption: [oneside] @@ -163,7 +165,24 @@ format: execute: error: false -# freeze: false freeze: auto -# cache: false -# enabled: true + cache: true + #freeze: false + #cache: false + enabled: true + +custom-callout: + theorem: + title: "Theorem" + icon: false + icon-symbol: "fa-splotch" + color: "#800000" + definition: + title: "Definition" + icon-symbol: "fa-book-open" + color: "#008000" + relationship: + icon-symbol: "fa-certificate" + color: "#008000" +filters: +- custom-callout diff --git a/quarto/alternatives.qmd b/quarto/alternatives.qmd index d59ba08..0b3f2b2 100644 --- a/quarto/alternatives.qmd +++ b/quarto/alternatives.qmd @@ -2,6 +2,8 @@ These notes use a particular selection of packages. This selection could have been different. For example: +* Symbolic math is provided by `SymPy`. [Giac](./alternatives/giac.html) and +[Symbolics](./alternatives/symbolics.html) (along with `SymbolicUtils` and `ModelingToolkit`) provide alternatives. * The finding of zeros of scalar-valued, univariate functions is done with `Roots`. The [NonlinearSolve](./alternatives/SciML.html#nonlinearsolve) package provides an alternative for univariate and multi-variate functions. diff --git a/quarto/alternatives/Project.toml b/quarto/alternatives/Project.toml index 1e24643..b538d7a 100644 --- a/quarto/alternatives/Project.toml +++ b/quarto/alternatives/Project.toml @@ -1,10 +1,12 @@ [deps] BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf" +CairoMakie = "13f3f980-e62b-5c42-98c6-ff1f3baf88f0" CalculusWithJulia = "a2e0e22d-7d4c-5312-9169-8b992201a882" ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210" GLMakie = "e9467ef8-e4e7-5192-8a1a-b1aee30e663a" GeometryBasics = "5c1252a2-5f33-56bf-86c9-59e7332b4326" IJulia = "7073ff75-c697-5162-941a-fcdaad2a7d2a" +Implicit3DPlotting = "d997a800-832a-4a4c-b340-7dddf3c1ad50" Integrals = "de52edbc-65ea-441a-8357-d3a637375a31" LaTeXStrings = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f" LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e" @@ -26,6 +28,7 @@ Roots = "f2b01f46-fcfa-551c-844a-d8ac1e96c665" SplitApplyCombine = "03a91e81-4c3e-53e1-a0a4-9c0c8f19dd66" StaticArrays = "90137ffa-7385-5640-81b9-e52037218182" SymPy = "24249f21-da20-56a4-8eb1-6a02cf4ae2e6" +SymbolicIntegration = "315ce56f-eed0-411d-ab8a-2fbdf9327b51" SymbolicLimits = "19f23fe9-fdab-4a78-91af-e7b7767979c3" SymbolicNumericIntegration = "78aadeae-fbc0-11eb-17b6-c7ec0477ba9e" Symbolics = "0c5d862f-8b57-4792-8d23-62f2024744c7" diff --git a/quarto/alternatives/SciML.qmd b/quarto/alternatives/SciML.qmd index a6e23c1..c68626c 100644 --- a/quarto/alternatives/SciML.qmd +++ b/quarto/alternatives/SciML.qmd @@ -1,7 +1,6 @@ # The SciML suite of packages - The `Julia` ecosystem advances rapidly. For much of it, the driving force is the [SciML](https://github.com/SciML) organization (Scientific Machine Learning). @@ -17,6 +16,27 @@ These packages are in a process of rapid development and change to them is expec ::: +This section uses these packages from `SciML`: + +```{julia} +using NonlinearSolve +using ModelingToolkit +using Optimization +using OptimizationOptimJL +using Integrals +``` + +In addition, we use these packages: + +```{julia} +using ForwardDiff +using QuadGK +using Plots +using StaticArrays +using BenchmarkTools +``` + + ## Symbolic math (`Symbolics`) @@ -26,7 +46,7 @@ The `Symbolics`, `SymbolicUtils`, and `ModelingToolkit` packages are provided by ## Solving equations -Solving one or more equations (simultaneously) is different in the linear case (where solutions are readily found – though performance can distinguish approaches) and the nonlinear case – where for most situations, numeric approaches are required. +Solving one or more equations (simultaneously) is different in the linear case (where solutions are readily found---though performance can distinguish approaches) and the nonlinear case---where for most situations, numeric approaches are required. ### `LinearSolve` @@ -69,7 +89,9 @@ However, it is more performant and not much more work to allow for a vector of s f(u, p) = @. (u^5 - u - 1) ``` -The function definition expects a container for the "`x`" variables and allows the passing of a container to hold parameters. We could have used the dotted operations for the power and each subtraction to allow vectorization of these basic math operations, as `u` is a container of values. The `@.` macro makes adding the "dots" quite easy, as illustrated above. It converts "every function call or operator in expr into a `dot call`." +The function definition expects a container for the "`x`" variables and allows the passing of a container to hold parameters. + +We could have used the dotted operations for the power and each subtraction to allow vectorization of these basic math operations, as `u` is a container of values. The `@.` macro makes adding the "dots" quite easy, as illustrated above. It converts "every function call or operator in expr into a `dot call`." A problem is set up with this function and an initial guess. The `@SVector` specification for the guess is for performance purposes and is provided by the `StaticArrays` package. @@ -90,7 +112,7 @@ soln = solve(prob, NewtonRaphson()) Again, the derivative of `f` is computed automatically. -The basic interface for retrieving the numeric solution from the solution object is to use indexing: +The basic interface for retrieving the numeric solution from the solution object is to use indexing notation with an empty index: ```{julia} @@ -101,7 +123,7 @@ soln[] :::{.callout-note} ## Note -This interface is more performant than `Roots`, though it isn't an apples to oranges comparison, as different stopping criteria are used by the two. In order to compare, we help out the call to `NonlinearProblem` to indicate the problem is non-mutating by adding a "`false`", as follows: +This interface is more performant than `Roots`, though it isn't like comparing apples to oranges, as different stopping criteria are used by the two. In order to compare, we help out the call to `NonlinearProblem` to indicate the problem is non-mutating by adding a "`false`", as follows: ```{julia} @@ -198,7 +220,7 @@ The gradient can be computed different ways within `Julia`, but here we use the ```{julia} -∇peaks(x, p=nothing) = NonlinearSolve.ForwardDiff.gradient(peaks, x) +∇peaks(x, p=nothing) = ForwardDiff.gradient(peaks, x) u0 = @SVector[1.0, 1.0] prob = NonlinearProblem(∇peaks, u0) u = solve(prob, NewtonRaphson()) @@ -234,29 +256,29 @@ The extra step is to specify a "`NonlinearSystem`." It is a system, as in practi ```{julia} -ns = NonlinearSystem([eq], [x], [α], name=:ns); -ns = complete(ns) +@mtkcompile sys = NonlinearSystem([eq], [x], [α]) ``` -The `name` argument is special. The name of the object (`ns`) is assigned through `=`, but the system must also know this same name. However, the name on the left is not known when the name on the right is needed, so it is up to the user to keep them synchronized. The `@named` macro handles this behind the scenes by simply rewriting the syntax of the assignment: - - -```{julia} -@named ns = NonlinearSystem([eq], [x], [α]); -ns = complete(ns) -``` +The name of the object (`sys`) is assigned through `=`, but the system must also know this same name. However, the name on the left is not known when the name on the right is needed, the`@mtkcompile` macro does this bookkeeping (an alternate is to use `@named`) and builds and compiles the system. With the system defined, we can pass this to `NonlinearProblem`, as was done with a function. The parameter is specified here, and in this case is `α => 1.0`. The initial guess is `[1.0]`: +A system has unknowns, parameters, and observables: ```{julia} -prob = NonlinearProblem(mtkcompile(ns), [1.0], Dict(α => 1.0)) +[:u=>unknowns(sys), :p=>parameters(sys), :o=>observables(sys)] ``` -The problem is solved as before: - +The initial condition and parameters are given as a dictionary mapping the variables to values, as with: ```{julia} +op = Dict(x => 1.5, α => 1.0) +``` + +Then the problem may be setup and solved as before: + +```{julia} +prob = NonlinearProblem(sys, op) solve(prob, NewtonRaphson()) ``` @@ -295,11 +317,11 @@ The minus sign is needed here as optimization routines find *minimums*, not maxi -Next, we define an optimization function with information on how its derivatives will be taken. The following uses `ForwardDiff`, which is a good choice in the typical calculus setting, where there are a small number of inputs (just $1$ here.) +Next, we define an optimization function with information on how its derivatives will be taken. T ```{julia} -F = OptimizationFunction(A, Optimization.AutoForwardDiff()) +F = OptimizationFunction(A) x0 = [4.0] prob = OptimizationProblem(F, x0) ``` @@ -313,11 +335,11 @@ soln = solve(prob, NelderMead()) :::{.callout-note} ## Note -We use the method `Newton` and not `NewtonRaphson`, as above. Both methods are similar, but they come from different packages – the latter for solving non-linear equation(s), the former for solving optimization problems. +We use the method `NelderMead` above, and not the more performant `Newton` and not `NewtonRaphson`, as above. Both methods are similar, but they come from different packages---the latter for solving non-linear equation(s), the former for solving optimization problems. ::: -The solution is an object containing the identified answer and more. To get the value, use index notation: +The solution is an object containing the identified answer and more. To get the value, again use index notation with an empty index: ```{julia} @@ -334,9 +356,8 @@ height(xstar), A(xstar) The `minimum` property also holds the identified minimum: - ```{julia} -soln.minimum # compare with A(soln[], nothing) +soln.objective # compare with A(soln[], nothing) ``` The package is a wrapper around other packages. The output of the underlying package is presented in the `original` property: @@ -352,7 +373,7 @@ soln.original This problem can also be approached symbolically, using `ModelingToolkit`. -For example, we set up the problem with: +For example, we load the package and set up the problem with: ```{julia} @@ -363,35 +384,29 @@ y = (P - 2x)/2 Area = - x*y ``` -The above should be self explanatory. To put into a form to pass to `solve` we define a "system" by specifying our objective function, the variables, and the parameters. +The above should be self explanatory. To put into a form to pass to `solve` we define a "system" by specifying our objective function, the variables, and the parameters and then "compile" it through `mtkcompile`, in this case its macro form, as before: ```{julia} -@named sys = OptimizationSystem(Area, [x], [P]); -sys = complete(sys) +@mtkcompile sys = OptimizationSystem(Area, [x], [P]); ``` -(This step is different, as before an `OptimizationFunction` was defined; we use `@named`, as above, to ensure the system has the same name as the identifier, `sys`.) - - -This system is passed to `OptimizationProblem` along with a specification of the initial condition ($x=4$) and the perimeter ($P=25$). A vector of pairs is used below: +This system is passed to `OptimizationProblem` along with a specification of the initial condition ($x=4$) and the perimeter ($P=25$). A dictionary is used below: ```{julia} -prob = OptimizationProblem(sys, [x => 4.0], [P => 25.0]; grad=true, hess=true) +op = Dict(x => 4.0, P => 25.0) +prob = OptimizationProblem(sys, op) ``` -The keywords `grad=true` and `hess=true` instruct for automatic derivatives to be taken as needed. These are needed in the choice of method, `Newton`, below. - - -Solving this problem then follows the same pattern as before, again with `Newton` we have: +Solving this problem then follows the same pattern as before: ```{julia} -solve(prob, Newton()) +solve(prob, NelderMead()) ``` -(A derivative-free method like `NelderMead()` could be used and then the `grad` and `hess` keywords above would be unnecessary, though not harmful.) +We used the derivative-free method `NelderMead()`. Other methods are more performant, but this one did not require a conversation about auto differentiation, as `Newton()` would have. --- @@ -412,13 +427,11 @@ could be similarly approached: @variables x y = Area/x # from A = xy P = 2x + 2y -@named sys = OptimizationSystem(P, [x], [Area]); -sys = structural_simplify(sys) +@mtkcompile sys = OptimizationSystem(P, [x], [Area]); -u0 = [x => 4.0] -p = [Area => 25.0] +op = Dict(x => 4.0, Area => 25.0) -prob = OptimizationProblem(sys, u0, p; grad=true, hess=true) +prob = OptimizationProblem(sys, op; grad=true, hess=true) soln = solve(prob, LBFGS()) ``` @@ -506,7 +519,6 @@ The package follows the same `problem-algorithm-solve` interface, as already see The interface is designed for $1$-and-higher dimensional integrals. - The package is loaded with @@ -519,7 +531,7 @@ For a simple definite integral, such as $\int_0^\pi \sin(x)dx$, we have: ```{julia} f(x, p) = sin(x) -prob = IntegralProblem(f, 0.0, 1pi) +prob = IntegralProblem(f, (0.0, 1pi)) soln = solve(prob, QuadGKJL()) ``` @@ -550,7 +562,7 @@ The `Integrals` solution is a bit more verbose, but it is more flexible. For exa ```{julia} f(x, p) = sin.(x) -prob = IntegralProblem(f, [0.0], [1pi]) +prob = IntegralProblem(f, (0.0, 1pi)) soln = solve(prob, HCubatureJL()) ``` @@ -588,14 +600,13 @@ Using `Integrals` with `QuadGK` we have: ```{julia} f(x, p) = sin(p*x) function ∫sinpx(p) - prob = IntegralProblem(f, 0.0, 1pi, p) + prob = IntegralProblem(f, (0.0, 1pi), p) solve(prob, QuadGKJL()) end ``` We can compute values at both $p=1$ and $p=2$: - ```{julia} ∫sinpx(1), ∫sinpx(2) ``` @@ -624,13 +635,13 @@ The area under a surface generated by $z=f(x,y)$ over a rectangular region $[a,b For example, the area under the function $f(x,y) = 1 + x^2 + 2y^2$ over $[-1/2, 1/2] \times [-1,1]$ is computed by: - +XXX ```{julia} f(x, y) = 1 + x^2 + 2y^2 # match math fxp(x, p) = f(x[1], x[2]) # prepare for IntegralProblem ls = [-1/2, -1] # left endpoints rs = [1/2, 1] # right endpoints -prob = IntegralProblem(fxp, ls, rs) +prob = IntegralProblem(fxp, (ls, rs)) # tuple of (ls, rs) soln = solve(prob, HCubatureJL()) ``` @@ -702,6 +713,7 @@ So we have $\iint_{G(R)} x^2 dA$ is computed by the following with $\alpha=\pi/4 ```{julia} +#| eval: false import LinearAlgebra: det 𝑓(uv) = uv[1]^2 @@ -735,6 +747,6 @@ For a trivial example, we have: ```{julia} f(x, p) = [x[1], x[2]^2] -prob = IntegralProblem(f, [0,0],[3,4]) +prob = IntegralProblem(f, ([0,0],[3,4])) solve(prob, HCubatureJL()) ``` diff --git a/quarto/alternatives/makie_plotting.qmd b/quarto/alternatives/makie_plotting.qmd index 4434a49..50669d9 100644 --- a/quarto/alternatives/makie_plotting.qmd +++ b/quarto/alternatives/makie_plotting.qmd @@ -1,15 +1,12 @@ # Calculus plots with Makie -{{< include ../_common_code.qmd >}} The [Makie.jl webpage](https://github.com/JuliaPlots/Makie.jl) says > From the Japanese word Maki-e, which is a technique to sprinkle lacquer with gold and silver powder. Data is basically the gold and silver of our age, so let's spread it out beautifully on the screen! - - -`Makie` itself is a metapackage for a rich ecosystem. We show how to use the interface provided by the `GLMakie` backend to produce the familiar graphics of calculus. +`Makie` itself is a metapackage for a rich ecosystem. We show how to use the interface provided by the `CairoMakie` backend to produce the familiar graphics of calculus. :::{.callout-note} @@ -21,139 +18,94 @@ The [Makie.jl webpage](https://github.com/JuliaPlots/Makie.jl) says ## Figures -Makie draws graphics onto a canvas termed a "scene" in the Makie documentation. A scene is an implementation detail, the basic (non-mutating) plotting commands described below return a `FigureAxisPlot` object, a compound object that combines a figure, an axes, and a plot object. The `show` method for these objects display the figure. +Makie draws graphics onto a canvas termed a "scene" in the Makie documentation. A scene is an implementation detail, the basic (non-mutating) plotting commands described below return a `FigureAxisPlot` object, a compound object that combines a figure, an axes, and a plot object. We also briefly show the details of constructing a separate figure and axis. +The `show` method for figures displays the resulting graphic. -For `Makie` there are the `GLMakie`, `WGLMakie`, and `CairoMakie` backends for different types of canvases. In the following, we have used `GLMakie`. `WGLMakie` is useful for incorporating `Makie` plots into web-based technologies. +For `Makie` there are the different backends for different types of canvases. In the following, we have used `CairoMakie`. We begin by loading the main package and the `norm` function from the standard `LinearAlgebra` package: ```{julia} -using GLMakie +using CairoMakie import LinearAlgebra: norm ``` -The package load time as of recent version of `Makie` is quite reasonable for a complicated project. (The time to first plot is under 3 seconds on a typical machine.) +```{julia} +#| echo: false +set_theme!(Figure = (size = (width, height),)) +nothing +``` + +The package load time as of recent version of `Makie` is quite reasonable for a complicated project. (The time to first plot is around five seconds on a typical machine.) ## Points (`scatter`) +The task of plotting the points, say $(1,2)$, $(2,3)$, $(3,2)$ can be done different ways. -The task of plotting the points, say $(1,2)$, $(2,3)$, $(3,2)$ can be done different ways. Most plotting packages, and `Makie` is no exception, allow the following: form vectors of the $x$ and $y$ values then plot those with `scatter`: +* We can plot two vectors holding the `x` and `y` coordinates, e.g. `[1,2,3]` and `[2,3,2]`. +* Using a tuple to represent a point, we can plot a vector of tuples, e.g. `[(1,2), (2,3), (3,2)]`. +* More idiomatically, using a `Point2` object to represent a point, a vector of point objects can be plotted, e.g. `[Point2(1,2), Point2(2,3), Point2(3,2)]`. +The `Point2` function also accepts a vector or tuple for input to describe the point. There is also `Point3` for 3-d plotting. `Makie` uses a GPU, when present, to accelerate the graphic rendering. GPUs employ 32-bit numbers. Julia uses an `f0` to indicate 32-bit floating points. Hence the alternate types `Point2f0` to store 2D points as 32-bit numbers and `Points3f0` to store 3D points as 32-bit numbers are seen in the documentation for Makie. + + +It is not so difficult to convert between these storage formats. For example starting with ```{julia} -xs = [1,2,3] -ys = [2,3,2] -scatter(xs, ys) +xs = [1, 2, 3] +ys = [2, 3, 2] ``` -The `scatter` function creates and returns an object, which when displayed shows the plot. - - -### `Point2`, `Point3` - - -When learning about points on the Cartesian plane, a "`t`"-chart is often produced: - - -``` -x | y ------ -1 | 2 -2 | 3 -3 | 2 -``` - -The `scatter` usage above used the columns. The rows are associated with the points, and these too can be used to produce the same graphic. Rather than make vectors of $x$ and $y$ (and optionally $z$) coordinates, it is more idiomatic to create a vector of "points." `Makie` utilizes a `Point` type to store a 2 or 3 dimensional point. The `Point2` and `Point3` constructors will be utilized. - - -`Makie` uses a GPU, when present, to accelerate the graphic rendering. GPUs employ 32-bit numbers. Julia uses an `f0` to indicate 32-bit floating points. Hence the alternate types `Point2f0` to store 2D points as 32-bit numbers and `Points3f0` to store 3D points as 32-bit numbers are seen in the documentation for Makie. - - -We can plot a vector of points in as direct manner as vectors of their coordinates: - +We can convert to a vector of tuples using broadcasting or `zip`: ```{julia} -pts = [Point2(1,2), Point2(2,3), Point2(3,2)] -scatter(pts) +tuple.(xs, ys), collect(zip(xs, ys)) ``` -A typical usage is to generate points from some vector-valued function. Say we have a parameterized function `r` taking $R$ into $R^2$ defined by: - +A vector of points can be generated similarly: ```{julia} -r(t) = [sin(t), cos(t)] +Point2.(xs, ys), Point2.(zip(xs, ys)) ``` -Then broadcasting values gives a vector of vectors, each identified with a point: - +The `unzip` function from the `CalculusWithJulia` package can reverse this direction: ```{julia} -ts = [1,2,3] -r.(ts) +using CalculusWithJulia: unzip # just SplitApplyCombine.invert +unzip(Point2.(xs, ys)) ``` -We can broadcast `Point2` over this to create a vector of `Point` objects: - +We illustrate in @fig-makie-plot-of-points-in-2-and-3 where we generate a vector of points using these two functions: ```{julia} -pts = Point2.(r.(ts)) +r(t) = [cos(t), sin(t)] +h(t) = (cos(t), sin(t), t) +ts = range(0, 2pi, 25) ``` -These then can be plotted directly: - +::: {#fig-makie-plot-of-points-in-2-and-3 layout-ncol=2} +```{julia} +scatter(Point2.(r.(ts))) +``` ```{julia} -scatter(pts) +scatter(Point3.(h.(ts))) ``` -The plotting of points in three dimensions is essentially the same, save the use of `Point3` instead of `Point2`. - - -```{julia} -r(t) = [sin(t), cos(t), t] -ts = range(0, 4pi, length=100) -pts = Point3.(r.(ts)) -scatter(pts; markersize=5) -``` - ---- - - -To plot points generated in terms of vectors of coordinates, the component vectors must be created. The "`t`"-table shows how, simply loop over each column and add the corresponding $x$ or $y$ (or $z$) value. This utility function does exactly that, returning the vectors in a tuple. - - -```{julia} -unzip(vs) = Tuple([vs[j][i] for j in eachindex(vs)] for i in eachindex(vs[1])) -``` - -:::{.callout-note} -## Note -In the `CalculusWithJulia` package, `unzip` is implemented using `SplitApplyCombine.invert`. +A scatter plot in 2d and 3d generated by broadscasting `Point2` over a vector of 2 component vectors and `Point3` over a vector of 3 component tuples. ::: -We might have then: + +### Attributes of a marker -```{julia} -scatter(unzip(r.(ts))...; markersize=5) -``` - -where splatting is used to specify the `xs`, `ys`, and `zs` to `scatter`. - - -(Compare to `scatter(Point3.(r.(ts)))` or `scatter(Point3∘r).(ts))`.) - - -### Attributes - - -A point is drawn with a "marker" with a certain size and color. These attributes can be adjusted, as in the following: - +A point is drawn with a "marker" with a certain size and color. These attributes can be adjusted, as in @fig-makie-scatter-with-marker-attributes. +::: {#fig-makie-scatter-with-marker-attributes} ```{julia} scatter(xs, ys; marker=[:x,:cross, :circle], @@ -161,26 +113,27 @@ scatter(xs, ys; color=:blue) ``` +Different marker attributes +::: + + Marker attributes include - * `marker` a symbol, shape. - * `marker_offset` offset coordinates - * `markersize` size (radius pixels) of marker - +* `marker` a symbol, shape +* `markersize` size (radius pixels) of marker +* `marker_offset` offset coordinates +* `color` to adjust color A single value will be repeated. A vector of values of a matching size will specify the attribute on a per point basis. ## Curves - -A visualization of a curve in calculus is comprised of line segments. The `lines` command of `Makie` will render a curve by connecting a series of points with straight-line segments. By taking a sufficient number of points the connect-the-dot figure can appear curved. - +A visualized curve in calculus is comprised of line segments. The `lines` command of `Makie` will render a curve by connecting a series of points with straight-line segments. By taking a sufficient number of points the connect-the-dot figure can appear curved. ### Plots of univariate functions - The basic plot of univariate calculus is the graph of a function $f$ over an interval $[a,b]$. This is implemented using a familiar strategy: produce a series of representative values between $a$ and $b$; produce the corresponding $f(x)$ values; plot these as points and connect the points with straight lines. @@ -189,7 +142,7 @@ To create regular values between `a` and `b` typically the `range` function or t For example: - +::: {#fig-makie-basic-f-a-b-plot} ```{julia} f(x) = sin(x) a, b = 0, 2pi @@ -197,55 +150,51 @@ xs = range(a, b, length=250) lines(xs, f.(xs)) ``` -`Makie` also will read the interval notation of `IntervalSets` and select its own set of intermediate points: +Use of `lines` to generate a curve +::: + +Makie has a recipe that allows the `y` position to be just the function---`lines(xs, f)` would have generated @fig-makie-basic-f-a-b-plot as well. -```{julia} -lines(a..b, f) -``` +`Makie` also will read the interval notation of `IntervalSets` and select its own set of intermediate points, so this command would also render the same plot as that of @fig-makie-basic-f-a-b-plot: `lines(a..b, f)`. + + +As with `scatter`, `lines` can also be drawn using a vector of points. (Though the advantage isn't clear here, this will be useful when the points are generated in different manners.) In this case, this approach would involve a command like `lines(Point2.(xs, f.(xs)))`. As with `scatter`, `lines` returns an object that produces a graphic when displayed. - -As with `scatter`, `lines` can also be drawn using a vector of points: - - -```{julia} -pts = [Point2(x, f(x)) for x ∈ xs] -lines(pts) -``` - -(Though the advantage isn't clear here, this will be useful when the points are generated in different manners.) - - When a `y` value is `NaN` or infinite, the connecting lines are not drawn: - +::: {#fig-makie-nan-disrupts-plotting} ```{julia} xs = 1:5 ys = [1,2,NaN, 4, 5] lines(xs, ys) ``` -As with other plotting packages, this is useful to represent discontinuous functions, such as what occurs at a vertical asymptote or a step function. +An `NaN` value in the `y` vector has no connecting line segment +::: +As with other plotting packages, this is useful to represent discontinuous functions, such as what occurs at a vertical asymptote or a step function. #### Adding to a figure (`lines!`, `scatter!`, ...) - -To *add* or *modify* a scene can be done using a mutating version of a plotting primitive, such as `lines!` or `scatter!`. The names follow `Julia`'s convention of using an `!` to indicate that a function modifies an argument, in this case the underlying figure. +To *add* or *modify* a figure can be done using a mutating version of a plotting primitive, such as `lines!` or `scatter!`. The names follow `Julia`'s convention of using an `!` to indicate that a function modifies an argument, in this case the underlying figure. Here is one way to show two plots at once: - +::: {#fig-makie-example-adding-a-layer} ```{julia} xs = range(0, 2pi, length=100) -lines(xs, sin.(xs)) -lines!(xs, cos.(xs)) +lines(xs, sin) # use function, not values from sin.(xs) +lines!(xs, cos) current_figure() ``` +Using `lines!` to add a layer +::: + :::{.callout-note} ## Current figure The `current_figure` call is needed to have the figure display, as the returned value of `lines!` is not a figure object. (Figure objects display when shown as the output of a cell.) @@ -255,37 +204,43 @@ The `current_figure` call is needed to have the figure display, as the returned We will see soon how to modify the line attributes so that the curves can be distinguished. -The following shows the construction details in the graphic: - +@fig-makie-example-adding-a-layer-scatter shows the construction details to produce the graphic. +::: {#fig-makie-example-adding-a-layer-scatter} ```{julia} xs = range(0, 2pi, length=10) -lines(xs, sin.(xs)) -scatter!(xs, sin.(xs); +lines(xs, sin) +scatter!(xs, sin; markersize=10) current_figure() ``` -As an example, this shows how to add the tangent line to a graph. The slope of the tangent line being computed by `ForwardDiff.derivative`. +Using `scatter!` to show the points used when creating a dot-to-dot plot +::: +As an example, @fig-makie-example-adding-a-layer-tangent-line shows how to add the tangent line to a graph. The slope of the tangent line being computed by `ForwardDiff.derivative`. +::: {#fig-makie-example-adding-a-layer-tangent-line} ```{julia} -import ForwardDiff +using ForwardDiff: derivative f(x) = x^x a, b= 0, 2 c = 0.5 xs = range(a, b, length=200) -tl(x) = f(c) + ForwardDiff.derivative(f, c) * (x-c) +tl(x) = f(c) + derivative(f, c) * (x-c) -lines(xs, f.(xs)) -lines!(xs, tl.(xs), color=:blue) +lines(xs, f) +lines!(xs, tl, color=:blue) current_figure() ``` -This example, modified from a [discourse](https://discourse.julialang.org/t/how-to-plot-step-functions-x-correctly-in-julia/84087/5) post by user `@rafael.guerra`, shows how to plot a step function (`floor`) using `NaN`s to create line breaks. The marker colors set for `scatter!` use `:white` to match the background color. +Adding a tangent line to a curve at a point +::: +This example^[This example is modified from a [discourse](https://discourse.julialang.org/t/how-to-plot-step-functions-x-correctly-in-julia/84087/5) post by user `@rafael.guerra`.] shows how to plot a step function (`floor`) using `NaN`s to create line breaks. The marker colors set for `scatter!` use `:white` to match the background color. +::: {#fig-makie-example-adding-a-layer-step-function} ```{julia} x = -5:5 δ = 5eps() # for rounding purposes; our interval is [i,i+1) ≈ [i, i+1-δ] @@ -300,15 +255,17 @@ scatter!(xx, yy, color=repeat([:black, :white, :white], length(xx)÷3)) current_figure() ``` +Using layers to plot a step function +::: + ### Text (`annotations`) Text can be placed at a point, as a marker is. To place text, the desired text and a position need to be specified along with any adjustments to the default attributes. +For example @fig-makie-example-adding-a-layer-annotations show an annotation and the use of `fontsize` to adjust the displayed text size. -For example: - - +::: {#fig-makie-example-adding-a-layer-annotations} ```{julia} xs = 1:5 pts = Point2.(xs, xs) @@ -320,26 +277,32 @@ annotation!(pts; current_figure() ``` -The graphic shows that `fontsize` adjusts the displayed size. +Using `annotation!` to add text within a graphic +::: Attributes for `text`, among many others, include: - * `align` Specify the text alignment through `(:pos, :pos)`, where `:pos` can be `:left`, `:center`, or `:right`. - * `fontsize` the font point size for the text - * `font` to indicate the desired font +* `align` Specify the text alignment through `(:pos, :pos)`, where `:pos` can be `:left`, `:center`, or `:right`. + +* `fontsize` the font point size for the text + +* `font` to indicate the desired font -Annotations with an arrow can be useful to highlight a feature of a graph. This example is modified from the documentation and utilizes some interval functions to draw an arrow with an arc: +Annotations with an arrow can be useful to highlight a feature of a graph. The code to produce @fig-makie-example-adding-a-layer-annotations-arrows is modified from the documentation of Makie; it utilizes some interval functions to draw an arrow with an arc.^[This example annotates the underlying `Axis` object, extracted using tuple destructuring. A more direct approach of creating a `Figure` object and then an `Axis` object will be illustrated later.] +::: {#fig-makie-example-adding-a-layer-annotations-arrows} ```{julia} g(x) = cos(6x) * exp(x) xs = 0:0.01:4 -_, ax, _ = lines(xs, g.(xs); axis = (; xgridvisible = false, ygridvisible = false)) +# this next line gets the Axis object from the FigureAxisPlot object +_, ax, _ = lines(xs, g; + axis = (; xgridvisible = false, ygridvisible = false)) -annotation!(ax, 1, 20, 2.1, g(2.1), +annotation!(ax, 1, 20, 2.1, g(2.1), # annotate an Axis object text = "A relative maximum", path = Ann.Paths.Arc(0.3), style = Ann.Styles.LineArrow(), @@ -349,21 +312,25 @@ annotation!(ax, 1, 20, 2.1, g(2.1), current_figure() ``` +Use of `path` and `style` attributes of `annotation!` to draw an arrow +::: #### Line attributes - In a previous example, we added the argument `color=:blue` to the `lines!` call. This was to set an attribute for the line being drawn. Lines have other attributes that allow different ones to be distinguished, as above where colors indicate the different graphs. - Other attributes can be seen from the help page for `lines`, and include: - * `color` set with a symbol, as above, or a string - * `label` a label for the line to display in a legend - * `linestyle` available styles are set by a symbol, one of `:dash`, `:dot`, `:dashdot`, or `:dashdotdot`. - * `linewidth` width of line - * `transparency` the `alpha` value, a number between $0$ and $1$, smaller numbers for more transparent. +* `color` set with a symbol, as above, or a string + +* `label` a label for the line to display in a legend + +* `linestyle` available styles are set by a symbol, one of `:dash`, `:dot`, `:dashdot`, or `:dashdotdot`. + +* `linewidth` width of line + +* `transparency` the `alpha` value, a number between $0$ and $1$, smaller numbers for more transparent. #### Simple legends @@ -392,7 +359,7 @@ The basic plots we have seen are of type `FigureAxisPlot`. The "axis" part contr For example: - +::: {#fig-makie-title-axis-label-xlabel-ylabel} ```{julia} xs = 0..2pi lines(xs, sin; @@ -400,9 +367,14 @@ lines(xs, sin; ) ``` +Passing `title`, `xlabel`, and `ylabel` values to the underlying axis +::: + + To access the `axis` element of a plot **after** the plot is constructed, values can be assigned to the `axis` property of the `FigureAxisPlot` object. For example: +::: {#fig-makie-title-axis-label-xlabel-ylabel-after-construction} ```{julia} xs = 0..2pi p = lines(xs, sin; @@ -413,52 +385,58 @@ p.axis.xticks = MultiplesTicks(5, pi, "π") # label 5 times using `pi` current_figure() ``` -The ticks are most easily set as a collection of values. Above, the `MultiplesTicks` function was used to label with multiples of $\pi$. +One way to access the underlying axis after construction. (The more systematic way is to produce a `Figure` and construct an `Axis` object to modify.) +::: +The ticks are most easily set as a collection of values. Above, the `MultiplesTicks` function was used to label with multiples of $\pi$. Later we will discuss how `Makie` allows for subsequent modification of several parts of the plot (not just the ticks) including the data. +#### Figure size, $x$ and $y$ limits -#### Figure resolution, $x$ and $y$ limits - - -As just mentioned, the basic plots we have seen are of type `FigureAxisPlot`. The "figure" part can be used to adjust the background color or the resolution. As with attributes for the axis, these too can be passed to a simple constructor: - +As just mentioned, the basic plots we have seen are of type `FigureAxisPlot`. The "figure" part can be used to adjust the background color or the size. As with attributes for the axis, these too can be passed to a simple constructor: +::: {#fig-makie-title-axis-figure-size} ```{julia} lines(xs, sin; axis=(title="Plot of sin(x)", xlabel="x", ylabel="sin(x)"), - figure=(;resolution=(300, 300)) + figure=(;size=(300, 300)) ) ``` -The `;` in the tuple passed to `figure` is one way to create a *named* tuple with a single element. Alternatively, `(resolution=(300,300), )` – with a trailing comma – could have been used. +Adjust `size` value for the enclosing `Figure` object +::: + +The `;` in the tuple passed to `figure` is one way to create a *named* tuple with a single element. -To set the limits of the graph there are shorthand functions `xlims!`, `ylims!`, and `zlims!`. This might prove useful if vertical asymptotes are encountered, as in this example: +To set the limits of the graph there are shorthand functions `xlims!`, `ylims!`, and `zlims!`. This might prove useful if vertical asymptotes are encountered, as in the code to produce @fig-makie-xlims-ylims. +::: {#fig-makie-xlims-ylims} ```{julia} f(x) = 1/x a,b = -1, 1 xs = range(-1, 1, length=200) -lines(xs, f.(xs)) +lines(xs, f) ylims!(-10, 10) current_figure() ``` -This still leaves the artifact due to the vertical asymptote at $0$ having different values from the left and the right. +Adjusting the range of possible `y` values with `ylims!` +::: +Adjusting the `y` limits still leaves an artifact due to the vertical asymptote at $0$ having different values from the left and the right. ### Plots of parametric functions - A space curve is a plot of a function $f:R^2 \rightarrow R$ or $f:R^3 \rightarrow R$. To construct a curve from a set of points, we have a similar pattern in both $2$ and $3$ dimensions: +::: {#fig-makie-parametric-plot-examples layout-ncol=2} ```{julia} r(t) = [sin(2t), cos(3t)] @@ -467,9 +445,6 @@ pts = Point2.(r.(ts)) # or (Point2∘r).(ts) lines(pts) ``` -Or - - ```{julia} r(t) = [sin(2t), cos(3t), t] ts = range(0, 2pi, length=200) @@ -477,32 +452,34 @@ pts = Point3.(r.(ts)) lines(pts) ``` -Alternatively, vectors of the $x$, $y$, and $z$ components can be produced and then plotted using the pattern `lines(xs, ys)` or `lines(xs, ys, zs)`. For example, using `unzip`, as above, we might have done the prior example with: +Two and three dimensional parametric plots +::: - -```{julia} -xs, ys, zs = unzip(r.(ts)) -lines(xs, ys, zs) -``` +Alternatively, vectors of the $x$, $y$, and $z$ components can be produced and then plotted using the pattern `lines(xs, ys)` or `lines(xs, ys, zs)`. For example, using `unzip`, as above, we might have done the prior example with `lines(unzip(r.(ts))...)`. #### Aspect ratio -A simple plot of a parametrically defined circle will show an ellipse, as the aspect ratio of the $x$ and $y$ axis is not $1$. To enforce this, we can pass a value of `aspect=1` to the underlying "Axis" object. For example: - +A simple plot of a parametrically defined circle will show an ellipse, as the aspect ratio of the $x$ and $y$ axis is not $1$. To enforce this, we can pass a value of `aspect=1` to the underlying "Axis" object. @fig-makie-aspect-ratio-1 provides an example. +::: {#fig-makie-aspect-ratio-1} ```{julia} ts = range(0, 2pi, length=100) -xs, ys = sin.(ts), cos.(ts) -lines(xs, ys; axis=(; aspect = 1)) +lines(sin.(ts), cos.(ts); + axis=(; aspect = 1)) ``` +Passing `aspect=1` to the axis make the `x` and `y`-axis scales equal +::: + #### Tangent vectors (`arrows`) -A tangent vector along a curve can be drawn quite easily using the `arrows` function. There are different interfaces for `arrows`, but we show the one which uses a vector of positions and a vector of "vectors". For the latter, we utilize the `derivative` function from `ForwardDiff`: +A tangent vector along a curve can be drawn quite easily using the `arrows` function. There are different interfaces for `arrows`, but we show the one which uses a vector of positions and a vector of "vectors". For the latter, we utilize the `derivative` function from `ForwardDiff`. In 3 dimensions the differences are minor, as seen in the code to produce @fig-makie-parametric-plot-tangent-vectors. + +::: {#fig-makie-parametric-plot-tangent-vectors layout-ncol=2} ```{julia} r(t) = [sin(t), cos(t)] # vector, not tuple ts = range(0, 4pi, length=200) @@ -510,15 +487,14 @@ lines(Point2.(r.(ts))) nts = 0:pi/4:2pi us = r.(nts) -dus = ForwardDiff.derivative.(r, nts) +dus = derivative.(r, nts) -arrows!(Point2.(us), Point2.(dus)) +arrows2d!(Point2.(us), Point2.(dus)) current_figure() ``` -In 3 dimensions the differences are minor: - +and ```{julia} r(t) = [sin(t), cos(t), t] # vector, not tuple @@ -527,38 +503,40 @@ lines(Point3.(r.(ts))) nts = 0:pi/2:(4pi-pi/2) us = r.(nts) -dus = ForwardDiff.derivative.(r, nts) +dus = derivative.(r, nts) -arrows!(Point3.(us), Point3.(dus)) +arrows3d!(Point3.(us), Point3.(dus)) current_figure() ``` -#### Arrow attributes +Plot of tangent lines in both two and three dimenstions +::: +#### Arrow attributes + Attributes for `arrows` include +* `arrowsize` to adjust the size - * `arrowsize` to adjust the size - * `lengthscale` to scale the size - * `arrowcolor` to set the color - * `arrowhead` to adjust the head - * `arrowtail` to adjust the tail +* `lengthscale` to scale the size + +* `arrowcolor` to set the color + +* `arrowhead` to adjust the head + +* `arrowtail` to adjust the tail ## Surfaces - -Plots of surfaces in $3$ dimensions are useful to help understand the behavior of multivariate functions. - +Plots of surfaces in $3$ dimensions are useful to help understand the behavior of multivariate functions. There are a few common visualizations. #### Surfaces defined through $z=f(x,y)$ - The "`peaks`" function defined below has a few prominent peaks: - ```{julia} function peaks(x, y) p = 3*(1-x)^2*exp(-x^2 - (y+1)^2) @@ -568,52 +546,72 @@ function peaks(x, y) end ``` -Here we see how `peaks` can be visualized over the region $[-5,5]\times[-5,5]$: - +@fig-makie-surface-peaks shows how `peaks` can be visualized over the region $[-5,5]\times[-5,5]$: +::: {#fig-makie-surface-peaks} ```{julia} xs = ys = range(-5, 5, length=25) surface(xs, ys, peaks) ``` -The calling pattern `surface(xs, ys, f)` implies a rectangular grid over the $x$-$y$ plane defined by `xs` and `ys` with $z$ values given by $f(x,y)$. +Surface plot produced by `surface(xs, ys, f)` +::: +The calling pattern `surface(xs, ys, f)` implies a rectangular grid over the $x$-$y$ plane defined by `xs` and `ys` with $z$ values given by $f(x,y)$. Alternatively a "matrix" of $z$ values can be specified. For a function `f`, this is conveniently generated by the pattern `f.(xs, ys')`, the `'` being important to get a matrix of all $x$-$y$ pairs through `Julia`'s broadcasting syntax. +::: {#fig-makie-surface-using-broadcasting} ```{julia} zs = peaks.(xs, ys') -surface(xs, ys, zs) +surface(xs, ys, zs); ``` -To see how this graph is constructed, the points $(x,y,f(x,y))$ are plotted over the grid and displayed. +Surface generated by `surface(xs, ys, zs)` where `zs` is a matrix of values +::: -Here we downsample to illustrate: +##### Example: surface graph construction + +To see how a surface graph is constructed, the points $(x,y,f(x,y))$ are plotted over the grid and displayed. +In this examplpe, we downsample to illustrate. + +::: {#fig-makie-surface-plot-scatter} ```{julia} xs = ys = range(-5, 5, length=5) pts = [Point3(x, y, peaks(x,y)) for x in xs for y in ys] scatter(pts, markersize=25) ``` -These points are then connected. The `wireframe` function illustrates just the frame: +Points used in downsampled graphic +::: +The points in @fig-makie-surface-plot-scatter are then connected. The `wireframe` function illustrates just the frame in @fig-makie-surface-plot-wireframe. + +::: {#fig-makie-surface-plot-wireframe} ```{julia} wireframe(xs, ys, peaks.(xs, ys'); linewidth=5) ``` -The `surface` call triangulates the frame and fills in the shading: +Wireframe used in downsampled graphic +::: +The `surface` call triangulates the frame and fills in the shading. + +::: {#fig-makie-surface-plot-surface} ```{julia} surface!(xs, ys, peaks.(xs, ys')) current_figure() ``` +The `surface` shading used in the downsampled graphic +::: + #### Parametrically defined surfaces @@ -641,10 +639,8 @@ end With the data suitably massaged, we can directly plot either a `surface` or `wireframe` plot. - --- - As an aside, The above can be done more campactly with nested list comprehensions: @@ -662,11 +658,12 @@ xs, ys, zs = unzip(r.(us, vs')) --- -For example, a sphere can be parameterized by $r(u,v) = (\sin(u)\cos(v), \sin(u)\sin(v), \cos(u))$ and visualized through: - +For example, a sphere can be parameterized by $r(u,v) = (\sin(u)\cos(v), \sin(u)\sin(v), \cos(u))$ and visualized through these commands to produce @fig-makie-parametric-sphere. +::: {#fig-makie-parametric-sphere} ```{julia} r(u,v) = [sin(u)*cos(v), sin(u)*sin(v), cos(u)] + us = range(0, pi, length=25) vs = range(0, pi/2, length=25) xs, ys, zs = parametric_grid(us, vs, r) @@ -676,8 +673,12 @@ wireframe!(xs, ys, zs) current_figure() ``` -A surface of revolution for $g(u)$ revolved about the $z$ axis can be visualized through: +Part of sphere plotted using a parametric description of the data +::: +A surface of revolution for $g(u)$ revolved about the $z$ axis can be visualized through the commands to produce @fig-makie-parametric-surface-revolution. + +::: {#fig-makie-parametric-surface-revolution} ```{julia} g(u) = u^2 * exp(-u) @@ -692,9 +693,12 @@ wireframe!(xs, ys, zs) current_figure() ``` -A torus with big radius $2$ and inner radius $1/2$ can be visualized as follows +Surface of revolution formed by revolving `g` around the `z` axis. +::: +A torus with big radius $2$ and inner radius $1/2$ is visualized in @fig-makie-parametric-surface-torus. +::: {#fig-makie-parametric-surface-torus} ```{julia} r1, r2 = 2, 1/2 r(u,v) = ((r1 + r2*cos(v))*cos(u), (r1 + r2*cos(v))*sin(u), r2*sin(v)) @@ -706,15 +710,17 @@ surface(xs, ys, zs) wireframe!(xs, ys, zs) current_figure() ``` +Surface of torus plotted as a parametrically defined surface +::: -A Möbius strip can be produced with: - +A Möbius strip is produced in @fig-makie-parametric-surface-mobius-strip. +::: {#fig-makie-parametric-surface-mobius-strip} ```{julia} -ws = range(-1/4, 1/4, length=8) -thetas = range(0, 2pi, length=30) r(w, θ) = ((1+w*cos(θ/2))*cos(θ), (1+w*cos(θ/2))*sin(θ), w*sin(θ/2)) +ws = range(-1/4, 1/4, length=8) +thetas = range(0, 2pi, length=30) xs, ys, zs = parametric_grid(ws, thetas, r) surface(xs, ys, zs) @@ -722,6 +728,10 @@ wireframe!(xs, ys, zs) current_figure() ``` +A Möbius strip can be parameterized and displayed +::: + + ## Contour plots (`contour`, `contourf`, `heatmap`) @@ -731,43 +741,54 @@ For a function $z = f(x,y)$ an alternative to a surface plot, is a contour plot. For a function $f(x,y)$, the syntax for generating a contour plot follows that for `surface`. -For example, using the `peaks` function, previously defined, we have a contour plot over the region $[-5,5]\times[-5,5]$ is generated through: - +For example, using the `peaks` function, previously defined, we have a contour plot over the region $[-5,5]\times[-5,5]$ is generated through `contour(xs, ys, peaks)`. A figure is shown in @fig-makie-contour-peaks`. +::: {#fig-makie-contour-peaks} ```{julia} xs = ys = range(-5, 5, length=100) contour(xs, ys, peaks) ``` -The default of $5$ levels can be adjusted using the `levels` keyword: +Contour plot of `peaks` +::: + +The default of $5$ levels can be adjusted using the contour function's `levels` keyword. @fig-makie-contour-peaks-levels show the `peaks` function with `levels = 20`. The `levels` argument can also specify precisely what levels are to be drawn. + +::: {#fig-makie-contour-peaks-levels} ```{julia} contour(xs, ys, peaks; levels = 20) ``` -The `levels` argument can also specify precisely what levels are to be drawn. +Contour plot of `peaks` with the display of 20 levels specified +::: The contour graph makes identification of peaks and valleys easy as the limits of patterns of nested contour lines. +A *filled* contour plot is produced by `contourf`, as seen in @fig-makie-contour-peaks-filled. -A *filled* contour plot is produced by `contourf`: - - +::: {#fig-makie-contour-peaks-filled} ```{julia} contourf(xs, ys, peaks) ``` -A related, but alternative visualization, using color to represent magnitude is a heatmap, produced by the `heatmap` function. The calling syntax is similar to `contour` and `surface`: +The `contourf` command produces filled contour plots +::: + +A related, but alternative visualization, using color to represent magnitude is a heatmap, produced by the `heatmap` function. The calling syntax is similar to `contour` and `surface`. @fig-makie-heatmatp-peaks shows peaks and valleys through "hotspots" on the graph. +::: {#fig-makie-heatmatp-peaks} ```{julia} heatmap(xs, ys, peaks) ``` -This graph shows peaks and valleys through "hotspots" on the graph. +Heatmap of `peaks` function +::: +##### Example The `MakieGallery` package includes an example of a surface plot with both a wireframe and 2D contour graph added. It is replicated here using the `peaks` function scaled by $5$. @@ -783,33 +804,44 @@ zs = peaks.(xs, ys') / 5; The `zs` were generated, as `wireframe` does not provide the interface for passing a function. -The `surface` and `wireframe` are produced as follows. Here we manually create the figure and axis object so that we can set the viewing angle through the `elevation` argument to the axis object: - +The `surface` and `wireframe` graphics are produced as follows. In the following we manually create the figure and axis object (using `Figure` and `Axis` as shown). We do this to set the viewing angle through the `elevation` argument to the axis object. We plot onto this axis in producing @fig-makie-peaks-surface-wireframe and then display the `Figure` object. +::: {#fig-makie-peaks-surface-wireframe} ```{julia} fig = Figure() -ax3 = Axis3(fig[1,1]; +ax3 = Axis3(fig[1,1]; # upper left of `fig` elevation=pi/9, azimuth=pi/16) + surface!(ax3, xs, ys, zs) wireframe!(ax3, xs, ys, zs; overdraw = true, transparency = true, color = (:black, 0.1)) -current_figure() + +fig # Figure object displays graphic ``` -To add the contour, a simple call via `contour!(scene, xs, ys, zs)` will place the contour at the $z=0$ level which will make it hard to read. Rather, placing at the "bottom" of the figure is desirable. To identify that the minimum value, is identified (and rounded) and the argument `transformation = (:xy, zmin)` is passed to `contour!`: +Surface and wireframe for the `peaks` function +::: +Next, we add a contour graph to @fig-makie-peaks-surface-wireframe to produce @fig-makie-peaks-surface-wireframe-contour. A simple call via `contour!(scene, xs, ys, zs)` will place the contour at the $z=0$ level which will make it hard to read. Rather, placing at the "bottom" of the figure is desirable. +To identify that the minimum value, is identified (and rounded) and the argument `transformation = (:xy, zmin)` is passed to `contour!`: + +::: {#fig-makie-peaks-surface-wireframe-contour} ```{julia} -ezs = extrema(zs) -zmin, zmax = floor(first(ezs)), ceil(last(ezs)) +zmin, zmax = extrema(zs) +zmin, zmax = floor(zmin), ceil(zmax) # round down/up contour!(ax3, xs, ys, zs; levels = 15, linewidth = 2, transformation = (:xy, zmin)) -zlims!(zmin, zmax) -current_figure() +zlims!(ax3, zmin, zmax) + +fig ``` +Surface, wireframe, and contour for the `peaks` function +::: + The `transformation` plot attribute sets the "plane" (one of `:xy`, `:yz`, or `:xz`) at a location, in this example `zmin`. @@ -819,9 +851,9 @@ The manual construction of a figure and an axis object will be further discussed ### Three dimensional contour plots -The `contour` function can also plot $3$-dimensional contour plots. Concentric spheres, contours of $x^2 + y^2 + z^2 = c$ for $c > 0$ are presented by the following: - +The `contour` function can also plot $3$-dimensional contour plots. Concentric spheres, contours of $x^2 + y^2 + z^2 = c$ for $c > 0$ are presented in @fig-makie-countour-three-d. +::: {#fig-makie-countour-three-d} ```{julia} f(x,y,z) = x^2 + y^2 + z^2 xs = ys = zs = range(-3, 3, length=100) @@ -829,6 +861,9 @@ xs = ys = zs = range(-3, 3, length=100) contour(xs, ys, zs, f) ``` +Three dimensional contour plot +::: + ### Implicitly defined curves and surfaces @@ -843,14 +878,16 @@ The graph of an equation is the collection of all $(x,y)$ values satisfying the The contour graph can produce these graphs by setting the `levels` argument to `[0]`. - +::: {#fig-makie-implicit-plot} ```{julia} f(x,y) = x^3 + x^2 + x + 1 - x*y # solve x^3 + x^2 + x + 1 = x*y xs = range(-5, 5, length=100) ys = range(-10, 10, length=100) -contour(xs, ys, f.(xs, ys'); levels=[0]) +contour(xs, ys, f; levels=[0]) ``` +Using `contour` to graph an implicitly defined function +::: The `implicitPlots.jl` function uses the `Contour` package along with a `Plots` recipe to plot such graphs. Here we see how to use `Makie` in a similar manner: @@ -888,58 +925,91 @@ To plot the equation $F(x,y,z)=0$, for $F$ a scalar-valued function, again the i With `Makie`, many implicitly defined surfaces can be adequately represented using `contour` with the attribute `levels=[0]`. We will illustrate this technique. -The `Implicit3DPlotting` package takes an approach like `ImplicitPlots` to represent these surfaces. It replaces the `Contour` package computation with a $3$-dimensional alternative provided through the `Meshing` and `GeometryBasics` packages. This package has a `plot_implicit_surface` function that does something similar to below. We don't illustrate it, as it *currently* doesn't work with the latest version of `Makie`. - -To begin, we plot a sphere implicitly as a solution to $F(x,y,z) = x^2 + y^2 + z^2 - 1 = 0$> +::: {.callout-note} +## GLMakie +The `CairoMakie` backend does not handle these next few plots, so we use `GLMakie.contour` in the following after importing with: ```{julia} -f(x,y,z) = x^2 + y^2 + z^2 - 1 +import GLMakie +``` +::: + +To begin, we plot a sphere implicitly as a solution to $F(x,y,z) = x^2 + y^2 + z^2 - 1 = 0$ in @fig-makie-implicit-plot-3d. + +::: {#fig-makie-implicit-plot-3d} +```{julia} +f(x) = norm(x)^2 - 1 +ϕ(x,y,z) = (x,y,z) xs = ys = zs = range(-3/2, 3/2, 100) -contour(xs, ys, zs, f; levels=[0]) +GLMakie.contour(xs, ys, zs, f∘ϕ; levels=[0], colormap=:RdBu) ``` +Three dimensional implicitly defined surface plotted with `contour` +::: -Here we visualize an intersection of a sphere with another figure: - +@fig-makie-implicit-plot-3d-intersection visualizes an intersection of a sphere with another figure. To show the different surfaces, different colormaps are chosen. +::: {#fig-makie-implicit-plot-3d-intersection} ```{julia} r₂(x) = sum(x.^2) - 2 # a sphere r₄(x) = sum(x.^4) - 1 ϕ(x,y,z) = (x,y,z) xs = ys = zs = range(-2, 2, 100) -contour(xs, ys, zs, r₂∘ϕ; levels = [0], colormap=:RdBu) -contour!(xs, ys, zs, r₄∘ϕ; levels = [0], colormap=:viridis) +GLMakie.contour(xs, ys, zs, r₂∘ϕ; levels = [0], colormap=:RdBu) +GLMakie.contour!(xs, ys, zs, r₄∘ϕ; levels = [0], colormap=:viridis) current_figure() ``` -This example comes from [Wikipedia](https://en.wikipedia.org/wiki/Implicit_surface) showing an implicit surface of genus $2$: +Two implicitly defined surfaces in three dimensions disambiguated through different colormaps +::: +@fig-makie-implicit-genus-2-function presents an example from [Wikipedia](https://en.wikipedia.org/wiki/Implicit_surface) showing an implicit surface of genus $2$. +::: {#fig-makie-implicit-genus-2-function} ```{julia} f(x,y,z) = 2y*(y^2 -3x^2)*(1-z^2) + (x^2 +y^2)^2 - (9z^2-1)*(1-z^2) xs = ys = zs = range(-5/2, 5/2, 100) -contour(xs, ys, zs, f; levels=[0], colormap=:RdBu) - +GLMakie.contour(xs, ys, zs, f; levels=[0], colormap=:RdBu) ``` -(This figure does not render well though, as the hole is not shown.) +Implicit surface of a genus 2 function. This figure does not render well though, as the hole is not displayed. +::: -For one last example from Wikipedia, we have the Cassini oval which "can be defined as the point set for which the *product* of the distances to $n$ given points is constant." That is: +The `Implicit3DPlotting` package takes an approach like `ImplicitPlots` to represent these surfaces. It replaces the `Contour` package computation with a $3$-dimensional alternative provided through the `Meshing` and `GeometryBasics` packages. This package has a `plot_implicit_surface` function that does something similar as just illustrated, but handles the "hole" not shown in @fig-makie-implicit-genus-2-function. +The `plot_implicit_surface` takes a function of a single argument, so we wrap `f` within `splat` which takes that single argument and "splats" them so `f` can be used. The result appears in @fig-makie-implicit-genus-2-function-Implicit3DPlotting. + +::: {#fig-makie-implicit-genus-2-function-Implicit3DPlotting} +```{julia} +using Implicit3DPlotting +plot_implicit_surface(splat(f); xlims=(-5/2, 5/2), ylims=(-5/2, 5/2)) +``` + +Implicit surface of a genus 2 function. This surface, rendered with the `Implicit3DPlotting` package, shows the holes present in the surface. +::: + + + +For one last example from Wikipedia, we have the Cassini oval which "can be defined as the point set for which the *product* of the distances to $n$ given points is constant." + +::: {#fig-makie-implicit-cassini-oval} ```{julia} -function cassini(λ, ps = ((1,0,0), (-1, 0, 0))) +function cassini(λ, ps = ((1,0,0), (-1, 0, 0))) # cassini returns a function n = length(ps) x -> prod(norm(x .- p) for p ∈ ps) - λ^n end -xs = ys = zs = range(-2, 2, 100) -contour(xs, ys, zs, cassini(0.80) ∘ ϕ; levels=[0], colormap=:RdBu) +xs = ys = zs = range(-3/2, 3/2, 100) +GLMakie.contour(xs, ys, zs, cassini(0.80) ∘ ϕ; levels=[0], colormap=:RdBu) ``` +Cassini oval implicitly defined +::: + ## Vector fields. Visualizations of $f:R^2 \rightarrow R^2$ @@ -962,20 +1032,26 @@ Broadcasting over `(xs, ys')` ensures each pair of possible values is encountere Calling `arrows` on the prepared data produces the graphic: - +::: {#fig-makie-arrows-vector-field} ```{julia} -arrows(pts, dus) +arrows2d(pts, dus) ``` +Vectorfield plot generated by `arrows2d`. Modification is needed. +::: + The grid seems rotated at first glance; but is also confusing. This is due to the length of the vectors as the $(x,y)$ values get farther from the origin. Plotting the *normalized* values (each will have length $1$) can be done easily using `norm` (which is found in the standard `LinearAlgebra` library): - +::: {#fig-makie-arrows-vector-field-modified} ```{julia} dvs = dus ./ norm.(dus) -arrows(pts, dvs) +arrows2d(pts, dvs) ``` -The rotational pattern becomes much clearer now. +Vectorfield plot generated by `arrows2d` after modification +::: + +The rotational pattern in @fig-makie-arrows-vector-field-modified is much clearer than from @fig-makie-arrows-vector-field. The `streamplot` function also illustrates this phenomenon. This implements an "algorithm [that] puts an arrow somewhere and extends the streamline in both directions from there. Then, it chooses a new position (from the remaining ones), repeating the exercise until the streamline gets blocked, from which on a new starting point, the process repeats." @@ -983,7 +1059,7 @@ The `streamplot` function also illustrates this phenomenon. This implements an " The `streamplot` function expects a `Point` not a pair of values, so we adjust `f` slightly and call the function using the pattern `streamplot(g, xs, ys)`: - +::: {#fig-makie-arrows-vector-field-streamplot} ```{julia} f(x, y) = [y, -x] g(xs) = Point2(f(xs...)) @@ -991,6 +1067,9 @@ g(xs) = Point2(f(xs...)) streamplot(g, -5..5, -5..5) ``` +Graph produced by `streamplot` +::: + (We used interval notation to set the viewing range, a range could also be used.) @@ -1000,10 +1079,25 @@ The calling pattern of `streamplot` is different than other functions, such as ` ::: -## Layoutables and Observables +## Layoutables +The `FigureAxisPlot` comprises an enclosing figure and one or more axes. These can be constructed directly through a pattern like the following: -### Layoutables +::: {#fig-makie-figure-axis} +```{julia} +F = Figure() # can pass size=(w,h) +ax = Axis(F[1,1]) # can pass title, xlabel, ylabel, ... +ylims!(ax, (-5, 5)) # can limit viewing window size for an axis + +xs = range(-8, 8, 100) +ys = xs .+ sin.(xs) +lines!(ax, xs, ys) # layer on the axis, not the figure + +F # display figure +``` + +Basic pattern to construct a figure and an axis to layer on +::: `Makie` makes it really easy to piece together figures from individual plots. To illustrate, we create a graphic consisting of a plot of a function, its derivative, and its second derivative. In our graphic, we also leave space for a label. @@ -1015,14 +1109,17 @@ The Layout [Tutorial](https://makie.juliaplots.org/stable/tutorials/layout-tutor ::: -The basic plotting commands, like `lines`, return a `FigureAxisPlot` object. For laying out our own graphic, we manage the figure and axes manually. The commands below create a figure, then assign axes to portions of the figure: +For laying out our own composite graphic, we manage the figure and axes manually. The commands below create a figure, then assign axes to certain portions of the figure: ```{julia} F = Figure() -af = F[2,1:2] = Axis(F) -afp = F[3,1:end] = Axis(F) -afpp = F[4,:] = Axis(F) +#af = F[2,1:2] = Axis(F) +#afp = F[3,1:end] = Axis(F) +#afpp = F[4,:] = Axis(F) +af = Axis(F[2,1:2]) +afp = Axis(F[3,1:end]) +afpp = Axis(F[4,:]) ``` The axes are named `af`, `afp` and `afpp`, as they will hold the respective graphs. The key here is the use of matrix notation to layout the graphic in a grid. The first one is row 2 and columns 1 through 2; the second row 3 and again all columns, the third is row 4 and all columns. @@ -1054,7 +1151,7 @@ lines!(afpp, xs, fpp) lines!(afpp, xs, zero, color=:blue); ``` -We can give title information to each axis: +We can give title information to each axis on construction or after construction, through commands like: ```{julia} @@ -1075,14 +1172,24 @@ is zero, the function f has an inflection point. """); ``` -Finally we display the figure: - +We display the figure in @fig-makie-layoutables +::: {#fig-makie-layoutables} ```{julia} F ``` -### Observables +Figure containing multiple axes +::: + + +## Observables + +::: {.callout-note} +## This needs updating + +As over `v"0.24"` of Makie, there is an alternative to using `Observables`. These notes need updating to reflect that change. +::: The basic components of a plot in `Makie` can be updated [interactively](https://makie.juliaplots.org/stable/documentation/nodes/index.html#observables_interaction). Historically `Makie` used the `Observables` package which allows complicated interactions to be modeled quite naturally. In the following we give a simple example, though newer versions of `Makie` rely on a different mechanism. @@ -1093,14 +1200,15 @@ In Makie, an `Observable` is a structure that allows its value to be updated, si This simple example shows how an observable `h` can be used to create a collection of points representing a secant line. The figure shows the value for `h=3/2`. - +::: {#fig-makie-observable} ```{julia} +begin f(x) = sqrt(x) c = 1 xs = 0..3 -h = Observable(3/2) +h′ = Observable(3/2) -points = lift(h) do h +points = lift(h′) do h xs = [0,c,c+h,3] tl = x -> f(c) + (f(c+h)-f(c))/h * (x-c) [Point2(x, tl(x)) for x ∈ xs] @@ -1109,23 +1217,31 @@ end lines(xs, f) lines!(points) current_figure() +end ``` -We can update the value of `h` using `setindex!` notation (square brackets). For example, to see that the secant line is a good approximation to the tangent line as $h \rightarrow 0$ we can set `h` to be `1/4` and replot: +Illustration of using an `Observable` +::: +We can update the value of `h` using `setindex!` notation (square brackets). For example, to see that the secant line is a good approximation to the tangent line as $h \rightarrow 0$ we can set `h` to be `1/4` and replot in @fig-makie-observable-updated. +::: {#fig-makie-observable-updated} ```{julia} -h[] = 1/4 +h′[] = 1/4 current_figure() ``` +Same plot as in @fig-makie-observable with `h'` value updated +::: + The line `h[] = 1/4` updated `h` which then updated `points` (a points is lifted up from `h`) which updated the graphic. (In these notes, we replot to see the change, but in an interactive session, the current *displayed* figure would be updated; no replotting would be necessary.) Finally, this example shows how to add a slider to adjust the value of `h` with a mouse. The slider object is positioned along with a label using the grid reference, as before. - +::: {#fig-makie=slider-added} ```{julia} +let f(x) = sqrt(x) c = 1 xs = 0..3 @@ -1145,7 +1261,11 @@ end lines!(ax, xs, f) lines!(ax, points) scatter!(ax, points; markersize=10) -current_figure() + current_figure() +end ``` +A slider added in position `F[2,2]` and connected through `lift` +::: + The slider value is "lifted" by its `value` component, as shown. Otherwise, the above is fairly similar to just using an observable for `h`. diff --git a/quarto/alternatives/symbolics.qmd b/quarto/alternatives/symbolics.qmd index cac5003..26b45bd 100644 --- a/quarto/alternatives/symbolics.qmd +++ b/quarto/alternatives/symbolics.qmd @@ -1,9 +1,5 @@ # Symbolics.jl -XXX This needs updating! XXX - -XXX add https://docs.sciml.ai/SymbolicIntegration/stable/ XXX - There are a few options in `Julia` for symbolic math, for example, the `SymPy` package which wraps a Python library. This section describes a collection of native `Julia` packages providing many features of symbolic math. @@ -386,7 +382,7 @@ ex = x^5 - x - 1 Roots.find_zero(λ, (1, 2)) ``` -### Plotting +## Plotting Using `Plots`, the plotting of symbolic expressions is similar to the plotting of a function, as there is a plot recipe that converts the expression into a function via `build_function`. @@ -425,7 +421,7 @@ The ordering of the variables is determined by `Symbolics.get_variables`: Symbolics.get_variables(ex) ``` -### Polynomial manipulations +## Polynomial manipulations There are some facilities for manipulating polynomial expressions in `Symbolics`. A polynomial, mathematically, is an expression involving one or more symbols with coefficients from a collection that has, at a minimum, addition and multiplication defined. The basic building blocks of polynomials are *monomials*, which are comprised of products of powers of the symbols. Mathematically, monomials are often allowed to have a multiplying coefficient and may be just a coefficient (if each symbol is taken to the power $0$), but here we consider just expressions of the type $x_1^{a_1} \cdot x_2^{a_2} \cdots \cdot x_k^{a_k}$ with the $a_i > 0$ as monomials. @@ -560,7 +556,7 @@ m,n = degree.(nd(ex)) m > n ? "limit is infinite" : m < n ? "limit is 0" : "limit is a constant" ``` -### Vectors and matrices +## Vectors and matrices Symbolic vectors and matrices can be created with a specified size: @@ -616,7 +612,7 @@ R \ b collect(R \ b) ``` -### Algebraically solving equations +## Algebraically solving equations The `~` operator creates a symbolic equation. For example @@ -660,7 +656,7 @@ eqs = R*X .~ b Symbolics.symbolic_linear_solve(eqs, [x,y]) ``` -### Limits +## Limits Many symbolic limits involving exponentials and logarithms can be computed in Symbolics, as of recent versions. The underlying package @@ -683,7 +679,7 @@ limit(F(𝑥), 𝑥, Inf) ``` -### Derivatives +## Derivatives `Symbolics` provides the `derivative` function to compute the derivative of a function with respect to a variable: @@ -764,164 +760,76 @@ eqs = [ x^2 - y^2, 2x*y] Symbolics.jacobian(eqs, [x,y]) ``` -### Integration +## Integration -The `SymbolicNumericIntegration` package provides a means to integrate *univariate* expressions through its `integrate` function. +The `SymbolicIntegration` package provides two means to integration *univariate* functions using either the Risch alogorithm or a rules-based approach. + +```{julia} +using SymbolicIntegration, Symbolics +``` + +The main entry point is the function `integrate`. + +This is a substitution test: + +```{julia} +@variables x a b +integrate(x*exp(-x^2), x) +``` + +This is an integration by parts example: + +```{julia} +integrate(x * sin(x), x) +``` + +As is this. We + +```{julia} +integrate(a * log(b*x), x) +``` -Symbolic integration can be approached in different ways. SymPy implements part of the Risch algorithm in addition to other algorithms. Rules-based algorithms could also be implemented. - - -For a trivial example, here is a rule that could be used to integrate a single integral +The integration of rational functions (ratios of polynomials) can be done algorithmically, provided the underlying factorizations can be identified. ```{julia} -@syms x ∫(x) - -is_var(x) = (xs = Symbolics.get_variables(x); length(xs) == 1 && xs[1] === x) -r = @rule ∫(~x::is_var) => x^2/2 - -r(∫(x)) +integrate((x-1)/((x-2)^3*(x-4)), x) ``` -The `SymbolicNumericIntegration` package includes many more predicates for doing rules-based integration, but it primarily approaches the task in a different manner. +#### Method selection +A third argument to `integrate` can specify the method, as in `integrate(expr, var, method)`. -If $f(x)$ is to be integrated, a set of *candidate* answers is generated. The following is **proposed** as an answer: $\sum q_i \Theta_i(x)$. Differentiating the proposed answer leads to a *linear system of equations* that can be solved. - - -The example in the [paper](https://arxiv.org/pdf/2201.12468v2.pdf) describing the method is with $f(x) = x \sin(x)$ and the candidate thetas are ${x, \sin(x), \cos(x), x\sin(x), x\cos(x)}$ so that the proposed answer is: - - -$$ -\int f(x) dx = q_1 x + q_2 \sin(x) + q_3 \cos(x) + q_4 x \sin(x) + q_5 x \cos(x) -$$ - -We differentiate the right hand side: - +This function does not get solved by the Risch method: ```{julia} -@variables q[1:5] x -ΣqᵢΘᵢ = dot(collect(q), (x, sin(x), cos(x), x*sin(x), x*cos(x))) -simplify(Symbolics.derivative(ΣqᵢΘᵢ, x)) +ex = cos(5x)*sin(x) +integrate(ex, x, RischMethod()) ``` -This must match $x\sin(x)$ so we have by equating coefficients of the respective terms: - - -$$ -q_2 + q_5 = 0, \quad q_4 = 0, \quad q_1 = 0, \quad q_3 = 0, \quad q_5 = -1 -$$ - -That is $q_2=1$, $q_5=-1$, and the other coefficients are $0$, giving an answer computed with: - +However, it can be done with the rules-based method: ```{julia} -d = Dict(q[i] => v for (i,v) ∈ enumerate((0,1,0,0,-1))) -substitute(ΣqᵢΘᵢ, d) +integrate(ex, x, RuleBasedMethod()) ``` -The package provides an algorithm for the creation of candidates and the means to solve when possible. The `integrate` function is the main entry point. It returns three values: `solved`, `unsolved`, and `err`. The `unsolved` is the part of the integrand which can not be solved through this package. It is `0` for a given problem when `integrate` is successful in identifying an antiderivative, in which case `solved` is the answer. The value of `err` is a bound on the numerical error introduced by the algorithm. + +The example of integrating `(x-1)/((x-2)^3*(x-4))`, done above, is one where the `RischMethod` works, but not the `RuleBasedMethod`. -To see, we have: - +Each method has different keyword arguments. For `RuleBasedMethod` the `verbose=true` argument will show which rules were applied. In this case, there is a single one: ```{julia} -using SymbolicNumericIntegration -@variables x - -integrate(x * sin(x)) +integrate(ex, x, RuleBasedMethod(verbose=true)) ``` -The second term is `0`, as this integrand has an identified antiderivative. - +This example, shows two rules are applied: ```{julia} -#| eval: false -integrate(exp(x^2) + sin(x)) +integrate(a * log(b*x), x, RuleBasedMethod(verbose=true)) ``` -This returns `exp(x^2)` for the unsolved part, as this function has no simple antiderivative. +To read the rules, there are different predicate functions involved. The commonly used `contains_var` predicate checks through pattern matching if the last variable is contained in any of the rest of the specified variables. In the pattern `c * x ^ n` the match is `c` is `b` and `n` is `1`. Neither depends on `x`, so this rule is applied to integrate `log(b*x)`. - -Powers of trig functions have antiderivatives, as can be deduced using integration by parts. When the fifth power is used, there is a numeric aspect to the algorithm that is seen: - - -```{julia} -u,v,w = integrate(sin(x)^5) -``` - -The derivative of `u` matches up to some numeric tolerance: - - -```{julia} -Symbolics.derivative(u, x) - sin(x)^5 -``` - ---- - - -The integration of rational functions (ratios of polynomials) can be done algorithmically, provided the underlying factorizations can be identified. The `SymbolicNumericIntegration` package has a function `factor_rational` that can identify factorizations. - - -```{julia} -#| eval: false -import SymbolicNumericIntegration: factor_rational -@variables x -u = (1 + x + x^2)/ (x^2 -2x + 1) -v = factor_rational(u) -``` - -The summands in `v` are each integrable. We can see that `v` is a reexpression through - - -```{julia} -#| eval: false -simplify(u - v) -``` - -The algorithm is numeric, not symbolic. This can be seen in these two factorizations: - - -```{julia} -#| eval: false -u = 1 / expand((x^2-1)*(x-2)^2) -v = factor_rational(u) -``` - -or - - -```{julia} -#| eval: false -u = 1 / expand((x^2+1)*(x-2)^2) -v = factor_rational(u) -``` - -As such, the integrals have numeric differences from their mathematical counterparts: - -::: {.callout-note} -#### Errors ahead - -These last commands are note being executed, as there are errors. -::: - - -```{julia} -#| eval: false -a,b,c = integrate(u) # not -``` - -We can see a bit of how much through the following, which needs a tolerance set to identify the rational numbers of the mathematical factorization correctly: - - -```{julia} -#| eval: false -cs = [first(arguments(term)) for term ∈ arguments(a)] # pick off coefficients -``` - -```{julia} -#| eval: false -rationalize.(cs[2:end]; tol=1e-8) -``` +The `contains_var(c, n, x)` call above checks if the constant `b` (matching `c` in the pattern) depends on `x` and if `1` depends on `x` (matching the power `n`, using a default for the variable, in the pattern). The rules comes from [`Rubi`](https://rulebasedintegration.org/) which has some 7000 rules, many of which are implemented in `SymbolicIntegration`. diff --git a/quarto/basics/Project.toml b/quarto/basics/Project.toml index 1c8e033..64b3c6c 100644 --- a/quarto/basics/Project.toml +++ b/quarto/basics/Project.toml @@ -1,4 +1,5 @@ [deps] +AbbreviatedStackTraces = "ac637c84-cc71-43bf-9c33-c1b4316be3d4" CalculusWithJulia = "a2e0e22d-7d4c-5312-9169-8b992201a882" DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" LaTeXStrings = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f" diff --git a/quarto/basics/calculator.qmd b/quarto/basics/calculator.qmd index 780149f..a4676bb 100644 --- a/quarto/basics/calculator.qmd +++ b/quarto/basics/calculator.qmd @@ -12,22 +12,16 @@ nothing Let us consider a basic calculator with buttons to add, subtract, multiply, divide, and take square roots. Using such a simple thing is certainly familiar for any reader of these notes. Indeed, a familiarity with a *graphing* calculator is expected. `Julia` makes these familiar tasks just as easy, offering numerous conveniences along the way. In this section we describe how. -The following image is the calculator that Google presents upon searching for "calculator." +::: {#fig-google-calculator-screenshot-1} +![](figures/calculator.png) + +Screenshot of a calculator provided by the Google search engine +::: -```{julia} -#| echo: false -# -imgfile = "figures/calculator.png" -caption = "Screenshot of a calculator provided by the Google search engine." -# ImageFile(:precalc, imgfile, caption) -nothing -``` - -![Screenshot of a calculator provided by the Google search engine.](./figures/calculator.png) -This calculator should have a familiar appearance with a keypad of numbers, a set of buttons for arithmetic operations, a set of buttons for some common mathematical functions, a degree/radian switch, and buttons for interacting with the calculator: `Ans`, `AC` (also `CE`), and `=`. +@fig-google-calculator-screenshot-1 is a screenshot of a calculator that Google presented upon searching for "calculator". This calculator should have a familiar appearance with a keypad of numbers, a set of buttons for arithmetic operations, a set of buttons for some common mathematical functions, a degree/radian switch, and buttons for interacting with the calculator: `Ans`, `AC` (also `CE`), and `=`. The goal here is to see the counterparts within `Julia` to these features. @@ -60,17 +54,16 @@ Performing a simple computation on the calculator typically involves hitting but 1 + 2 ``` -Sending an expression to `Julia`'s interpreter - the equivalent of pressing the "`=`" key on a calculator - is done at the command line by pressing the `Enter` or `Return` key, and in `Pluto`, also using the "play" icon, or the keyboard shortcut `Shift-Enter`. If the current expression is complete, then `Julia` evaluates it and shows any output. If the expression is not complete, `Julia`'s response depends on how it is being called. Within `Pluto`, a message about "`premature end of input`" is given. If the expression raises an error, this will be noted. +Sending an expression to `Julia`'s interpreter---the equivalent of pressing the "`=`" key on a calculator---is done at the command line by pressing the `Enter` or `Return` key. If the current expression is complete, then `Julia` evaluates it and shows any output. If the expression is not complete, `Julia`'s response at the command line pauses for more input. Other interfaces may have variations of the above. - -The basic arithmetic operations on a calculator are "+", "-", "×", "÷", and "$xʸ$". These have parallels in `Julia` through the *binary* operators: `+`, `-`, `*`, `/`, and `^`: +The basic arithmetic operations on a calculator are +, -, ×, ÷ and xʸ. These have parallels in `Julia` through the *binary* operators: `+`, `-`, `*`, `/`, and `^`: ```{julia} 1 + 2, 2 - 3, 3 * 4, 4 / 5, 5 ^ 6 ``` -On some calculators, there is a distinction between minus signs - the binary minus sign and the unary minus sign to create values such as $-1$. +On some calculators, there is a distinction between minus signs---the binary minus sign and the unary minus sign to create values such as $-1$. In `Julia`, the same symbol, "`-`", is used for each: @@ -89,7 +82,7 @@ An expression like $6 - -3$, subtracting minus three from six, must be handled w (If no space is included, the value "`--`" is parsed like a different, invalid, operation.) -:::{.callout-warning} +::: {.callout-warning} ## Warning `Julia` only uses one symbol for minus, but web pages may not! Copying and pasting an expression with a minus sign can lead to hard to @@ -101,13 +94,10 @@ the typeset math (e.g., $1 - \pi$) than for the code within cells ::: -### Examples +##### Example: Celsius and Fahrenheit -##### Example - - -For everyday temperatures, the conversion from Celsius to Fahrenheit ($9/5 C + 32$) is well approximated by simply doubling and adding $30$. Compare these values for an average room temperature, $C=20$, and for a relatively chilly day, $C=5$: +For everyday temperatures, the conversion from Celsius to Fahrenheit ($9/5\cdot C + 32$) is well approximated by simply doubling and adding $30$. Compare these values for an average room temperature, $C=20$, and for a relatively chilly day, $C=5$: For $C=20$: @@ -188,7 +178,7 @@ A right triangle has sides $a=11$ and $b=12$. Find the length of the hypotenus 11^2 + 12^2 ``` -##### Example +##### Example: How many ants? An overview of a research paper published in [theconversation.com](https://theconversation.com/earth-harbours-20-000-000-000-000-000-ants-and-they-weigh-more-than-wild-birds-and-mammals-combined-190831) reviews six authors' work on estimating the number of ants currently on earth. This was covered in an article in the [Washington Post](https://www.washingtonpost.com/climate-environment/2022/09/19/ants-population-20-quadrillion/). @@ -244,7 +234,7 @@ With the Google Calculator, typing `1 + 2 x 3 =` will give the value $7$, but *i In `Julia`, the entire expression is typed in before being evaluated, so the usual conventions of mathematics related to the order of operations may be used. These are colloquially summarized by the acronym [PEMDAS](http://en.wikipedia.org/wiki/Order_of_operations). -> **PEMDAS**. This acronym stands for Parentheses, Exponents, Multiplication, Division, Addition, Subtraction. The order indicates which operation has higher precedence, or should happen first. This isn't exactly the case, as "M" and "D" have the same precedence, as do "A" and "S". In the case of two operations with equal precedence, *associativity* is used to decide which to do. For the operations `-`, `/` the associativity is left to right, as in the left one is done first, then the right. However, `^` has right associativity, so `4^3^2` is `4^(3^2)` and not `(4^3)^2` (Be warned that some calculators - and spread sheets, such as Excel - will treat this expression with left associativity). But, `+` and `*` don't have associativity, so `1+2+3` can be `(1+2)+3` or `1+(2+3)`. +> **PEMDAS**. This acronym stands for Parentheses, Exponents, Multiplication, Division, Addition, Subtraction. The order indicates which operation has higher precedence, or should happen first. This isn't exactly the case, as "M" and "D" have the same precedence, as do "A" and "S". In the case of two operations with equal precedence, *associativity* is used to decide which to do. For the operations `-`, `/` the associativity is left to right, as in the left one is done first, then the right. However, `^` has right associativity, so `4^3^2` is `4^(3^2)` and not `(4^3)^2` (Be warned that some calculators---and spread sheets, such as Excel---will treat this expression with left associativity). The operators `+` and `*` are called associative, which means the order doesn't matter, so `1+2+3` can be `(1+2)+3` or `1+(2+3)`.^[Well, mathematically, floating point math has well known cases for `+` where associativity does not hold, such as `(0.1 + 0.2) + 0.3` not begin exactly equal to `0.1 + (0.2 + 0.3)`] @@ -269,8 +259,6 @@ If different parentheses are used, the answer will likely be different. For exam (1 + ((2 - 3) * 4)) / (5 ^ 6) ``` -### Examples - ##### Example @@ -284,6 +272,20 @@ The percentage error in $x$ if $y$ is the correct value is $(x-y)/y \cdot 100$. ##### Example +The *percentage decrease* of an item is $(o-n)/o \cdot 100$ with $o$ being the original price and $n$ being the new one. If a drug that cost 60 dollars is not 55 dollars what is the percentage decrease? + +```{julia} +(60 - 50) / 60 * 100 +``` + +The "different way of doing math" would compute this as $(o-n)/n \cdot 100$. With that unusual style, this nonmathematical value is: + +```{julia} +(60 - 50)/50 * 100 +``` + +##### Example + The marginal cost of producing one unit can be computed by finding the cost for $n+1$ units and subtracting the cost for $n$ units. If the cost of $n$ units is $n^2 + 10$, find the marginal cost when $n=100$. @@ -312,7 +314,7 @@ The slope of the line through two points is $m=(y_1 - y_0) / (x_1 - x_0)$. For t (4 - 2) / (3 - 1) ``` -### Two ways to write division - and they are not the same +### Two ways to write division The expression $a + b / c + d$ is equivalent to $a + (b/c) + d$ due to the order of operations. It will generally have a different answer than $(a + b) / (c + d)$. @@ -325,7 +327,7 @@ $$ \frac{1 + 2}{3 + 4}? $$ -It would have to be computed through $(1 + 2) / (3 + 4)$. This is because unlike `/`, the implied order of operation in the mathematical notation with the *horizontal division symbol* (the [vinculum](http://tinyurl.com/y9tj6udl)) is to compute the top and the bottom and then divide. That is, the vincula is a grouping notation like parentheses, only implicitly so. Thus the above expression really represents the more verbose: +It would have to be computed through $(1 + 2) / (3 + 4)$. This is because unlike `/`, the implied order of operations in the mathematical notation with the *horizontal division symbol* (the [vinculum](http://tinyurl.com/y9tj6udl)) is to compute the top and the bottom and then divide. That is, the vincula is a grouping notation like parentheses, only implicitly so. Thus the above expression really represents the more verbose: $$ @@ -346,7 +348,7 @@ To emphasize, this is not the same as the value without the parentheses: 1 + 2 / 3 + 4 ``` -:::{.callout-warning} +::: {.callout-warning} ## Warning The vinculum also indicates grouping when used with the square root (the top bar), and complex conjugation. That usage is often clear enough, but the usage of the vinculum in division often leads to confusion. The example above is one where the parentheses are often, erroneously, omitted. However, more confusion can arise when there is more than one vincula. An expression such as $a/b/c$ written inline has no confusion, it is: $(a/b) / c$ as left association is used; but when written with a pair of vincula there is often the typographical convention of a slightly longer vincula to indicate which is to be considered first. In the absence of that, then top to bottom association is often implied. @@ -406,7 +408,7 @@ In most cases. There are occasional (basically rare) spots where using `pi` by i ### Numeric literals -For some special cases, Julia parses *multiplication* without a multiplication symbol. One case is when the value on the left is a number, as in `2pi`, which has an equivalent value to `2*pi`. *However* the two are not equivalent, in that multiplication with *numeric literals* does not have the same precedence as regular multiplication - it is higher. This has practical importance when used in division or powers. For instance, these two expressions are **not** the same: +For some special cases, Julia parses *multiplication* without a multiplication symbol. One case is when the value on the left is a number, as in `2pi`, which has an equivalent value to `2*pi`. *However* the two are not equivalent, in that multiplication with *numeric literals* does not have the same precedence as regular multiplication---**it is higher**. This has practical importance when used in division or powers. For instance, these two expressions are **not** the same: ```{julia} @@ -439,44 +441,28 @@ This follows usual mathematical convention, but is a source of potential confusi On the Google calculator, the square root button has a single purpose: for the current value find a square root if possible, and if not signal an error (such as what happens if the value is negative). For more general powers, the $x^y$ key can be used. -In `Julia`, functions are used to perform the actions that a specialized button may do on the calculator. `Julia` provides many standard mathematical functions - more than there could be buttons on a calculator - and allows the user to easily define their own functions. For example, `Julia` provides the same set of functions as on Google's calculator, though with different names. For logarithms, $\ln$ becomes `log` and $\log$ is `log10` (computer programs almost exclusively reserve `log` for the natural log); for factorials, $x!$, there is `factorial`; for powers $\sqrt{...}$ becomes `sqrt`, $EXP$ becomes `exp`, and $x^y$ is computed with the infix operator `^`. For the trigonometric functions, the basic names are similar: `sin`, `cos`, `tan`. These expect radians. For angles in degrees, the convenience functions `sind`, `cosd`, and `tand` are provided. On the calculator, inverse functions like $\sin^{-1}(x)$ are done by combining $Inv$ with $\sin$. With `Julia`, the function name is `asin`, an abbreviation for "arcsine." (Which is a good thing, as the notation using a power of $-1$ is often a source of confusion and is not supported by `Julia` without work.) Similarly, there are `asind`, `acos`, `acosd`, `atan`, and `atand` functions available to the `Julia` user. +In `Julia`, functions are used to perform the actions that a specialized button may do on the calculator. `Julia` provides many standard mathematical functions---more than there could be buttons on a calculator---and allows the user to easily define their own functions. For example, `Julia` provides the same set of functions as on Google's calculator, though with different names. For logarithms, $\ln$ becomes `log` and $\log$ is `log10` (computer programs almost exclusively reserve `log` for the natural log); for factorials, $x!$, there is `factorial`; for powers $\sqrt{...}$ becomes `sqrt`, $EXP$ becomes `exp`, and $x^y$ is computed with the infix operator `^`. For the trigonometric functions, the basic names are similar: `sin`, `cos`, `tan`. These expect radians. For angles in degrees, the convenience functions `sind`, `cosd`, and `tand` are provided. On the calculator, inverse functions like $\sin^{-1}(x)$ are done by combining $Inv$ with $\sin$. With `Julia`, the function name is `asin`, an abbreviation for "arcsine." (Which is a good thing, as the notation using a power of $-1$ is often a source of confusion.) Similarly, there are `asind`, `acos`, `acosd`, `atan`, and `atand` functions available to the `Julia` user. +@tbl-calculator-and-julia-counterparts shows the counterparts between a calculator's dedicated buttons and `Julia` functions. -The following table summarizes the above: +::: {#tbl-calculator-and-julia-counterparts .striped .hover} +| Calculator | Julia | +| ------------------------------------:| ----------------------------------------------:| +| $+$, $-$, $\times$, $\div$ | `+`, `-`, `*`, `/` | +| $x^y$ | `^` | +| $\sqrt{...}$, $\sqrt[3]{...}$ | `sqrt`, `cbrt` | +| $EXP$, $e^x$ | `exp` | +| $\ln$, $\log$ | `log`, `log10` | +| $\sin, \cos, \tan, \sec, \csc, \cot$ | `sin`, `cos`, `tan`, `sec`, `csc`, `cot` | +| In degrees, not radians | `sind`, `cosd`, `tand`, `secd`, `cscd`, `cotd` | +| $\sin^{-1}, \cos^{-1}, \tan^{-1}$ | `asin`, `acos`, `atan` | +| $n!$ | `factorial` | -```{julia} -#| echo: false -using DataFrames -calc = [ -L" $+$, $-$, $\times$, $\div$", -L"x^y", -L"\sqrt{...}, \sqrt[3]{...}", -L"e^x", -L" $\ln$, $\log$", -L"\sin, \cos, \tan, \sec, \csc, \cot", -"In degrees, not radians", -L"\sin^{-1}, \cos^{-1}, \tan^{-1}", -L"n!", -] +: Table matching up common buttons on a scientific calculator with `Julia` functions +::: -julia = [ -"`+`, `-`, `*`, `/`", -"`^`", -"`sqrt`, `cbrt`", -"`exp`", -"`log`, `log10`", -"`sin`, `cos`, `tan`, `sec`, `csc`, `cot`", -"`sind`, `cosd`, `tand`, `secd`, `cscd`, `cotd`", -"`asin`, `acos`, `atan`", -"`factorial`" -] - -d = DataFrame(Calculator=calc, Julia=julia) -Table(d) -``` - Using a function is very straightforward. A function is called using parentheses, in a manner visually similar to how a function is called mathematically. So if we consider the `sqrt` function, we have: @@ -498,7 +484,7 @@ exp(2), log(10), sqrt(100), 10^(1/2) :::{.callout-note} ## Note -Parentheses have many roles. We've just seen that parentheses may be used for grouping, and now we see they are used to indicate a function is being called. These are familiar from their parallel usage in traditional math notation. In `Julia`, a third usage is common, the making of a "tuple," or a container of different objects, for example `(1, sqrt(2), pi)`. In these notes, the output of multiple commands separated by commas is a printed tuple. +Parentheses have many roles. We've just seen that parentheses may be used for grouping, and now we see they are used to indicate a function is being called. These are familiar from their parallel usage in traditional math notation. In `Julia`, a third usage is common, the making of a "tuple," or a container of different objects, for example `(1, sqrt(2), pi)`. In these notes, the output of multiple commands separated by commas is a printed tuple, as with the previous output. ::: @@ -518,8 +504,6 @@ log(e), log(2, e), log(10, e), log(e, 2) In `Julia`, the "generic" function `log` not only has different implementations for different types of arguments (real or complex), but also a different implementation depending on the number of arguments. -### Examples - ##### Example @@ -567,7 +551,10 @@ The formula to compute the resistance of two resistors in parallel is given by: Not all computations on a calculator are valid. For example, the Google calculator will display `Error` as the output of $0/0$ or $\sqrt{-1}$. These are also errors mathematically, though the second is not if the complex numbers are considered. -In `Julia`, there is a richer set of error types. The value `0/0` will in fact not be an error, but rather a value `NaN`. This is a special floating point value indicating "not a number" and is the result for various operations. The output of $\sqrt{-1}$ (computed via `sqrt(-1)`) will indicate a domain error: +In `Julia`, there is a richer set of error types. The value `0/0` will in fact not be an error, but rather a value `NaN`. This is a special floating point value indicating "not a number" and is the result for various operations. + + +The output of $\sqrt{-1}$ (computed via `sqrt(-1)`) will indicate a domain error: ```{julia} #| error: true @@ -592,7 +579,7 @@ On a machine with $64$-bit integers, the first of these two values is correct, t Wrong is in quotes, as though they are mathematically incorrect, computationally they are correct. The last two are due to overflow. The cost of checking is considered too high, so no error is thrown and the values represent what happens at the machine level. -The user is expected to have a sense that they need to be careful when their values are quite large. But the better recommendation is that the user use floating point numbers, which as easy as typing `2.0^63`. Though not always exact, floating point values can represent a much bigger range values and are exact for a reasonably wide range of integer values. +The user is expected to have a sense that they need to be careful when their values are quite large. But the better recommendation is that the user use floating point numbers, which is as easy as typing `2.0^63`. Though not always exact, floating point values can represent a much bigger range values and are exact for a reasonably wide range of integer values. ::: {.callout-note} @@ -600,7 +587,7 @@ The user is expected to have a sense that they need to be careful when their val We can see in the following, using the smaller 8-bit type, what goes on internally with successive powers of `2`: the bit pattern is found by shifting the previous one over to the left, consistent with what happens at the bit level when multiplying by `2`: -``` +```{julia} [bitstring(Int8(2)^i) for i in 1:8] ``` @@ -1131,7 +1118,7 @@ val = 8/2*(2+2) numericq(val) ``` -Does this expression return the *correct* answer using proper order of operations? +Does this expression return the *correct* answer using the proper order of operations? ```{julia} @@ -1189,8 +1176,8 @@ $$ Attempting to compute this, we have: ```{julia} -c = 299_792_458; # the speed of light -G = 6.67430e-11; # Gravitational constant +c = 299_792_458; # the speed of light +G = 6.67430e-11; # Gravitational constant h = 6.62607015e-34; # Planck's constant h_bar = h / (2*pi); planck_length = sqrt(h_bar * G / c^3) @@ -1209,3 +1196,40 @@ Yes, this is computed incorrectly. The value `c^3` *overflows* as the actual val """ buttonq(choices, 3; explanation=explanation) ``` + +## Appendix + +The order of operations for common operators are summarized in `Julia`'s [manual](https://docs.julialang.org/en/v1/manual/mathematical-operations/) in the "Operator Precedence and Associativity" section. @tbl-precedence-associativity is an edited version of one presented there. + +::: {#tbl-precedence-associativity} + +| Category | Operators | Associativity | +|:---------------|:--------------------------------|:---------------| +|Syntax | `.` followed by `::` | Left | +|*Exponentiation*| `^` | Right | +|Unary | `+`, `-` `!`, `~`, `√` | Right | +|Bit shifts | `<<`, `>>`, `>>>` | Left | +|Fractions | `//` | Left | +|*Multiplication*| `*`, `/`, `%`, `÷`, `&` | Left | +|*Addition* | `+`,`-`, `|` | Left | +|Syntax | `:`, `..` | Left | +|Syntax | `|>` | Right | +|Comparisons | `<|`, `>`, `<`, `>=`, `<=`, `==`, `===`, `!=`, `!--` | Non-associative | +|Control flow | `&&` followed by `||` followed by `?` | Right | +|Pair | `=>` | Right | +|Assignment | `=` `+-`, `-=`, `*=`, `/= | Right | + +: Table highlighting the precedence and associativity of different types of operators. Adapted from the Julia manual. +::: + + +Parentheses force evaluation, so aren't listed in the table. Otherwise we highlighted `⋅EM⋅A⋅` with italics. The `DS` are in the list of operators. + +The low precedence of the comparison, pair, and assignment operators ensure the left and right hand sides are parsed and evaluated before these binary operations, reducing the number of necessary parentheses. + +The low precedence of `?` is helpful. This symbol is used in the `ternary` operation and this low precedence requires fewer parentheses for most uses. + +The high precedence of the **unary** operations allows natural syntax for powers, where `2^-3` evaluates `-3` and then takes the power. However, note `2^1/2` will first find `2^1` and then divide by `2`. + + +The code that defines the order is in [Julia Syntax](https://github.com/JuliaLang/JuliaSyntax.jl/blob/main/src/julia/kinds.jl). A perusal will show that there are **numerous** unicode operations not listed in the @tbl-precedence-associativity above. diff --git a/quarto/basics/logical_expressions.qmd b/quarto/basics/logical_expressions.qmd index 74c4422..b707e7b 100644 --- a/quarto/basics/logical_expressions.qmd +++ b/quarto/basics/logical_expressions.qmd @@ -18,7 +18,7 @@ plotly() nothing ``` -## Boolean values +## Boolean values, comparison operators In mathematics it is common to test if an expression is true or false. For example, is the point $(1,2)$ inside the disc $x^2 + y^2 \leq 1$? We would check this by substituting $1$ for $x$ and $2$ for $y$, evaluating both sides of the inequality and then assessing if the relationship is true or false. In this case, we end up with a comparison of $5 \leq 1$, which we of course know is false. @@ -36,12 +36,12 @@ x^2 + y^2 <= 1 The response is `false`, as expected. `Julia` provides [Boolean](http://en.wikipedia.org/wiki/Boolean_data_type) values `true` and `false` for such questions. The same process is followed as was described mathematically. -The set of numeric comparisons is nearly the same as the mathematical counterparts: `<`, `<=`, `==`, `>=`, `>`. The syntax for less than or equal can also be represented with the Unicode `≤` (generated by `\le[tab]`). Similarly, for greater than or equal, there is `\ge[tab]`. +The set of comparison operators is nearly the same as the mathematical counterparts: `<`, `<=`, `==`, `>=`, `>`. The syntax for less than or equal can also be represented with the Unicode `≤` (generated by `\le[tab]`). Similarly, for greater than or equal, there is `\ge[tab]`. :::{.callout-warning} ## Warning -The use of `==` is necessary, as `=` is used for assignment and mutation. +The use of `==` is necessary, as `=` is used for assignment and mutation and `===` is used for *identicalness* from a hardware standpoint. ::: @@ -54,6 +54,13 @@ The `!` operator takes a boolean value and negates it. It uses prefix notation: For convenience, `a != b` can be used in place of `!(a == b)`. +---- + +The function `isapprox` has a unicode counterpart ` ≈` (typed through `\approx[tab]`) that allows numeric comparisons up to a tolerance. This is needed when comparing floating point values. For example, due to the inability to exactly represent the numbers in floating point the first of these comparisons is false: + +```{julia} +1/10 + 2/10 == 3/10, 1/10 + 2/10 ≈ 3/10 +``` ## Algebra of inequalities @@ -136,10 +143,13 @@ a < b, 1/a > 1/b In summary we investigated numerically that the following hold: - * `a < b` if and only if `a + c < b + c` for all finite `a`, `b`, and `c`. - * `a < b` if and only if `c*a < c*b` for all finite `a` and `b`, and finite, positive `c`. - * `a < b` if and only if `-a > -b` for all finite `a` and `b`. - * `a < b` if and only if `1/a > 1/b` for all finite, positive `a` and `b`. +* `a < b` if and only if `a + c < b + c` for all finite `a`, `b`, and `c` in $[0,1]$. + +* `a < b` if and only if `c*a < c*b` for all finite `a` and `b`, and finite, positive `c`, all in $[0,1]$. + +* `a < b` if and only if `-a > -b` for all finite `a` and `b` in $[0,1]$. + +* `a < b` if and only if `1/a > 1/b` for all finite, positive `a` and `b` in $[0,1]$. ### Examples @@ -228,28 +238,42 @@ This is to be expected, but we could also have written: Read aloud this would be "minus $7$ is less than $x$ minus $5$ **and** $x$ minus $5$ is less than $7$". -The "and" equations can be combined as above with a natural notation. However, an equation like $\lvert x - 5\rvert > 7$ would emphasize an **or** and be "$x$ minus $5$ less than minus $7$ **or** $x$ minus $5$ greater than $7$". Expressing this requires some new notation. +::: {.callout-note} +## Chaining expressions +The above (`-7 < x - 5 < 7`)---where two logical comparisons are used, is referred to as chaining. It is parsed differently than might be expected and is not read in as `(-7 < x - 5) < 7` which would compare a boolean to `7`. Chaining is neither left- or right-associative. +::: -The *boolean shortcut operators* `&&` and `||` implement "and" and "or". (There are also *bitwise* boolean operators `&` and `|`, but we only describe the former.) +## Short-circuit operators +The "and" equations can be combined as above with a natural notation through chaining. However, an equation like $\lvert x - 5\rvert > 7$ would emphasize an **or** and be "$x$ minus $5$ less than minus $7$ **or** $x$ minus $5$ greater than $7$". Expressing this requires some new notation. + + +The *boolean short-circuiting operators* `&&` and `||` implement "and" and "or". Thus we could write $-7 < x-5 < 7$ as ```{julia} -(-7 < x - 5) && (x - 5 < 7) +(-7 < x - 5) && (x - 5 < 7) # and ``` and could write $\lvert x-5\rvert > 7$ as ```{julia} -(x - 5 < -7) || (x - 5 > 7) +(x - 5 < -7) || (x - 5 > 7) # or ``` -(The first expression is false for $x=18$ and the second expression true, so the "or"ed result is `true` and the "and" result is `false`.) +The first expression is false for $x=18$ and the second expression true, so the "and" result is `false` and the "or"ed result is `true`. +These operators are called *short-circuiting* operators. + +The operation `x && y` evaluates `x` and if true returns the evaluated value of `y`. If `x` is false, `y` is not evaluated (it is short-circuited). + +The operation `x || y` evaluates `x` and if true return `true` and if false the value of `y` is evaluated and returned. + +Both are idiomatically used for control flow (if `x` then `y` or if *not* `x` then `y` statements). ##### Example @@ -262,6 +286,13 @@ A,B = true, false ## also true, true; false, true; and false, false !(A && B) == !A || !B ``` +::: {.callout-note} +## Bitwise `&` and `|` + +Both the short-circuiting operators and the bitwise operators can evaluate truth tables---where Boolean values are compared---but the short-circuiting values *don't* need the second argument to evaluate to a Boolean. The bitwise operators also have a different precedence; the same as `*` and `/` and not lower precedence than the arithmetic operations. +::: + + ## Precedence @@ -281,7 +312,6 @@ x - 5 < -7 || x - 5 > 7 So no, they were not in this case. - An operator (such as `<`, `>`, `||` above) has an associated associativity and precedence. The associativity is whether an expression like `a - b - c` is `(a-b) - c` or `a - (b-c)`. The former being left associative, the latter right. Of issue here is *precedence*, as in with two or more different operations, which happens first, second, $\dots$. @@ -295,7 +325,7 @@ The table in the manual on [operator precedence and associativity](https://docs. (This is different than the precedence of the bitwise boolean operators, which have `&` with "Multiplication" and `|` with "Addition", so `x-5 < 7 | x - 5 > 7` would need parentheses.) -A thorough understanding of the precedence rules can help eliminate unnecessary parentheses, but in most cases it is easier just to put them in. +A thorough understanding of the precedence rules can help eliminate unnecessary parentheses, but if in doubt, just put them in. ## Arithmetic with @@ -308,7 +338,7 @@ For convenience, basic arithmetic can be performed with Boolean values, `false` true + true + false, false * 1000 ``` -The first example shows a common means used to count the number of `true` values in a collection of Boolean values - just add them. +The first example shows a common means used to count the number of `true` values in a collection of Boolean values---just add them. This can be cleverly exploited. For example, the following expression returns `x` when it is positive and $0$ otherwise: @@ -328,7 +358,15 @@ This expression returns `x` if it is between $-10$ and $10$ and otherwise $-10$ (x < -10)*(-10) + (x >= -10)*(x < 10) * x + (x>=10)*10 ``` -The `clamp(x, a, b)` performs this task more generally, and is used as in `clamp(x, -10, 10)`. +The `clamp(x, a, b)` performs this task more generally. + +### Example + +The value of `im`, representing $i$ the imaginary number with $i^2 = -1$ is internally modeled using a Boolean `false` for the real part of a complex number and a value `true` for the imaginary part. + +```{julia} +dump(im) +``` ## Questions @@ -398,6 +436,27 @@ answ = 3 radioq(choices, answ) ``` +###### Question + +Compute the following: + +```{julia} +#| eval: false +u = typemax(1) +u < u + 1 +``` + +Is the output as expected? + +```{julia} +#| echo: false +explanation = """ +No, for integer numbers in math, it is always the case that `u < u + 1`, but for this particular `u` there is *no larger* integer, so `u+1` "wraps around" to give a negative number which on comparison is less than `u` +""" +buttonq(["Yes", "No"], 2; explanation) +``` + + ###### Question diff --git a/quarto/basics/numbers-types-orig.qmd b/quarto/basics/numbers-types-orig.qmd new file mode 100644 index 0000000..67223d9 --- /dev/null +++ b/quarto/basics/numbers-types-orig.qmd @@ -0,0 +1,685 @@ +# Number systems + + +{{< include ../_common_code.qmd >}} + +```{julia} +#| echo: false +#| results: "hidden" +using CalculusWithJulia + +nothing +``` + +In mathematics, there are many different number systems in common use. For example by the end of pre-calculus, all of the following have been introduced: + + + * The integers, $\{\dots, -3, -2, -1, 0, 1, 2, 3, \dots\}$; + * The rational numbers, $\{p/q: p, q \text{ are integers}, q \neq 0\}$; + * The real numbers, $\{x: -\infty < x < \infty\}$; + * The complex numbers, $\{a + bi: a,b \text{ are real numbers and } i^2=-1\}$. + + +On top of these, we have special subsets, such as the natural numbers $\{1, 2, \dots\}$ (sometimes including $0$), the even numbers, the odd numbers, the positive numbers, the non-negative numbers, etc. + + +Mathematically, these number systems are naturally nested within each other as integers are rational numbers which are real numbers, which can be viewed as part of the complex numbers. + + +Calculators typically have just one type of number - floating point values. These model the real numbers. `Julia`, on the other hand, has a rich type system, and within that has several different number types. There are types that model each of the four main systems above, and within each type, specializations for how these values are stored. + + +Most of the details will not be of interest to all, and will be described later. + + +For now, let's consider the number $1$. It can be viewed as either an integer, rational, real, or complex number. To construct "$1$" in each type within `Julia` we have these different styles: + + +```{julia} +1, 1.0, 1//1, 1 + 0im +``` + +The basic number types in `Julia` are `Int`, `Float64`, `Rational` and `Complex`, though in fact there are many more, and the last two aren't even *concrete* types. This distinction is important, as the type of number dictates how it will be stored and how precisely the stored value can be expected to be to the mathematical value it models. + + +Though there are explicit constructors for these types, these notes avoid them unless necessary, as `Julia`'s parser can distinguish these types through an easy to understand syntax: + + + * integers have no decimal point; + * floating point numbers have a decimal point (or are in scientific notation); + * rationals are constructed from integers using the double division operator, `//`; and + * complex numbers are formed by including a term with the imaginary unit, `im`. + + +:::{.callout-note} +## Warning +Heads up, the difference between `1` and `1.0` is subtle. Even more so, as `1.` will parse as `1.0`. This means some expressions, such as `2.*3`, are ambiguous, as the `.` might be part of the `2` (as in `2. * 3`) or the operation `*` (as in `2 .* 3`). + +::: + +Similarly, each type is printed slightly differently. + + +The key distinction is between integers and floating points. While floating point values include integers, and so can be used exclusively on the calculator, the difference is that an integer is guaranteed to be an exact value, whereas a floating point value, while often an exact representation of a number is also often just an *approximate* value. This can be an advantage – floating point values can model a much wider range of numbers. + +In nearly all cases the differences are not noticeable. Take for instance this simple calculation involving mixed types. + + +```{julia} +1 + 1.25 + 3//2 +``` + +The sum of an integer, a floating point number and rational number returns a floating point number without a complaint. + + +This is because behind the scenes, `Julia` will often "promote" a type to match, so for example to compute `1 + 1.25` the integer `1` will be promoted to a floating point value and the two values are then added. Similarly, with `2.25 + 3//2`, where the fraction is promoted to the floating point value `1.5` and addition is carried out. + + +As floating point numbers may be approximations, some values are not quite what they would be mathematically: + + +```{julia} +sqrt(2) * sqrt(2) - 2, sin(1pi), 1/10 + 1/5 - 3/10 +``` + +These values are *very* small numbers, but not exactly $0$, as they are mathematically. + + +--- + + +The only common issue is with powers. We saw this previously when discussing a distinction between `2^64` and `2.0^64`. `Julia` tries to keep a predictable output from the input types (not their values). Here are the two main cases that arise where this can cause unexpected results: + + +* integer bases and integer exponents can *easily* overflow. Not only `m^n` is always an integer, it is always an integer with a fixed storage size computed from the sizes of `m` and `n`. So the powers can quickly get too big. This can be especially noticeable on older $32$-bit machines, where too big is $2^{32} = 4,294,967,296$. On $64$-bit machines, this limit is present but much bigger. + + +Rather than give an error though, `Julia` gives seemingly arbitrary answers, as can be seen in this example on a $64$-bit machine: + + +```{julia} +2^62, 2^63 +``` + +(They aren't arbitrary, as explained previously.) + + +This could be worked around, as it is with some programming languages, but it isn't, as it would slow down this basic computation. So, it is up to the user to be aware of cases where their integer values can grow to big. The suggestion is to use floating point numbers in this domain, as they have more room, at the cost of sometimes being approximate values for fairly large values. + + +* the `sqrt` function will give a domain error for negative values: + + +```{julia} +#| error: true +sqrt(-1.0) +``` + +This is because for real-valued inputs `Julia` expects to return a real-valued output. Of course, this is true in mathematics until the complex numbers are introduced. Similarly in `Julia` - to take square roots of negative numbers, start with complex numbers: + + +```{julia} +sqrt(-1.0 + 0im) +``` + + * At one point, `Julia` had an issue with a third type of power: + + +integer bases and negative integer exponents. For example `2^(-1)`. This is now special cased, though only for numeric literals. If `z=-1`, `2^z` will throw a `DomainError`. Historically, the desire to keep a predictable type for the output (integer) led to defining this case as a domain error, but its usefulness led to special casing. + + +## Additional details. + + +What follows is only needed for those seeking more background. + + +Julia has *abstract* number types `Integer`, `Real`, and `Number`. All four types described above are of type `Number`, but `Complex` is not of type `Real`. + + +However, a specific value is an instance of a *concrete* type. A concrete type will also include information about how the value is stored. For example, the *integer* `1` could be stored using $64$ bits as a signed integers, or, should storage be a concern, as an $8$ bits signed or even unsigned integer, etc.. If storage isn't an issue, but exactness at all scales is, then it can be stored in a manner that allows for the storage to grow using "big" numbers. + + +These distinctions can be seen in how `Julia` parses these three values: + + + * `1234567890` will be a $64$-bit integer (on newer machines), `Int64` + * `12345678901234567890` will be a $128$ bit integer, `Int128` + * `1234567890123456789012345678901234567890` will be a big integer, `BigInt` + + +Having abstract types allows programmers to write functions that will work over a wide range of input values that are similar, but have different implementation details. + + +### Integers + + +Integers are often used casually, as they come about from parsing. As with a calculator, floating point numbers *could* be used for integers, but in `Julia` - and other languages - it proves useful to have numbers known to have *exact* values. In `Julia` there are built-in number types for integers stored in $8$, $16$, $32$, $64$, and $128$ bits and `BigInt`s if the previous aren't large enough. ($8$ bits can hold $8$ binary values representing $1$ of $256=2^8$ possibilities, whereas the larger $128$ bit can hold one of $2^{128}$ possibilities.) Smaller values can be more efficiently used, and this is leveraged at the system level, but not a necessary distinction with calculus where the default size along with an occasional usage of `BigInt` suffice. + + +### Floating point numbers + + +[Floating point](http://en.wikipedia.org/wiki/Floating_point) numbers are a computational model for the real numbers. For floating point numbers, $64$ bits are used by default for both $32$- and $64$-bit systems, though other storage sizes can be requested. This gives a large range - but still finite - set of real numbers that can be represented. However, there are infinitely many real numbers just between $0$ and $1$, so there is no chance that all can be represented exactly on the computer with a floating point value. Floating point then is *necessarily* an approximation for all but a subset of the real numbers. Floating point values can be viewed in normalized [scientific notation](http://en.wikipedia.org/wiki/Scientific_notation) as $a\cdot 2^b$ where $a$ is the *significand* and $b$ is the *exponent*. Save for special values, the significand $a$ is normalized to satisfy $1 \leq \lvert a\rvert < 2$, the exponent can be taken to be an integer, possibly negative. + + +As per IEEE Standard 754, the `Float64` type gives 52 bits to the precision (with an additional implied one), 11 bits to the exponent and the other bit is used to represent the sign. Positive, finite, floating point numbers have a range approximately between $10^{-308}$ and $10^{308}$, as 308 is about $\log_{10} 2^{1023}$. The numbers are not evenly spread out over this range, but, rather, are much more concentrated closer to $0$. + +The use of 32-bit floating point values is common, as some widely used computer chips expect this. These values have a narrower range of possible values. + +:::{.callout-warning} +## More on floating point numbers +You can discover more about the range of floating point values provided by calling a few different functions. + + * `typemax(0.0)` gives the largest value for the type (`Inf` in this case). + * `prevfloat(Inf)` gives the largest finite one, in general `prevfloat` is the next smallest floating point value. + + * `nextfloat(-Inf)`, similarly, gives the smallest finite floating point value, and in general returns the next largest floating point value. + * `nextfloat(0.0)` gives the closest positive value to 0. + * `eps()` gives the distance to the next floating point number bigger than `1.0`. This is sometimes referred to as machine precision. + +::: + + +#### Scientific notation + + +Floating point numbers may print in a familiar manner: + + +```{julia} +x = 1.23 +``` + +or may be represented in scientific notation: + + +```{julia} +6.022 * 10.0^23 +``` + +The special coding `aeb` (or if the exponent is negative `ae-b`) is used to represent the number $a \cdot 10^b$ ($1 \leq a < 10$). This notation can be used directly to specify a floating point value: + + +```{julia} +avogadro = 6.022e23 +``` + +::: {.callout-note} +## Not `e` +Here `e` is decidedly *not* the Euler number, rather **syntax** to separate the exponent from the mantissa. +::: + +The first way of representing this number required using `10.0` and not `10` as the integer power will return an integer and even for 64-bit systems is only valid up to `10^18`. Using scientific notation avoids having to concentrate on such limitations. + + +##### Example + + +Floating point values in scientific notation will always be normalized. This is easy for the computer to do, but tedious to do by hand. Here we see: + + +```{julia} +4e30 * 3e40 +``` + +```{julia} +3e40 / 4e30 +``` + +The power in the first is $71$, not $70 = 30+40$, as the product of $3$ and $4$ as $12$ or `1.2e^1`. (We also see the artifact of `1.2` not being exactly representable in floating point.) + + +##### Example: 32-bit floating point + + +In some uses, such as using a GPU, $32$-bit floating point (single precision) is also common. These values may be specified with an `f` in place of the `e` in scientific notation: + + +```{julia} +1.23f0 +``` + +As with the use of `e`, some exponent is needed after the `f`, even if it is `0`. + + +#### Special values: Inf, -Inf, NaN + + +The coding of floating point numbers also allows for the special values of `Inf`, `-Inf` to represent positive and negative infinity. As well, a special value `NaN` ("not a number") is used to represent a value that arises when an operation is not closed (e.g., $0.0/0.0$ yields `NaN`). (Technically `NaN` has several possible "values," a point ignored here.) Except for negative bases, the floating point numbers with the addition of `Inf` and `NaN` are closed under the operations `+`, `-`, `*`, `/`, and `^`. Here are some computations that produce `NaN`: + + +```{julia} +0/0, Inf/Inf, Inf - Inf, 0 * Inf +``` + +Whereas, these produce an infinity + + +```{julia} +1/0, Inf + Inf, 1 * Inf +``` + +Finally, these are mathematically undefined, but still yield a finite value with `Julia`: + + +```{julia} +0^0, Inf^0 +``` + +#### Floating point numbers and real numbers + + +Floating point numbers are an abstraction for the real numbers. For the most part this abstraction works in the background, though there are cases where one needs to have it in mind. Here are a few: + + + * For real and rational numbers, between any two numbers $a < b$, there is another real number in between. This is not so for floating point numbers which have a finite precision. (Julia has some functions for working with this distinction.) + * Floating point numbers are approximations for most values, even simple rational ones like $1/3$. This leads to oddities such as this value not being $0$: + + +```{julia} +sqrt(2)*sqrt(2) - 2 +``` + +It is no surprise that an irrational number, like $\sqrt{2}$, can't be represented **exactly** within floating point, but it is perhaps surprising that simple numbers can not be, so $1/3$, $1/5$, $\dots$ are approximated. Here is a surprising-at-first consequence: + + +```{julia} +1/10 + 2/10 == 3/10 +``` + +That is adding `1/10` and `2/10` is not exactly `3/10`, as expected mathematically. Such differences are usually very small and are generally attributed to rounding error. The user needs to be mindful when testing for equality, as is done above with the `==` operator. + + + * Floating point addition is not necessarily associative, that is the property $a + (b+c) = (a+b) + c$ may not hold exactly. For example: + + +```{julia} +1/10 + (2/10 + 3/10) == (1/10 + 2/10) + 3/10 +``` + + * Mathematically, for real numbers, subtraction of similar-sized numbers is not exceptional, for example $1 - \cos(x)$ is positive if $0 < x < \pi/2$, say. This will not be the case for floating point values. If $x$ is close enough to $0$, then $\cos(x)$ and $1$ will be so close, that they will be represented by the same floating point value, `1.0`, so the difference will be zero: + + +```{julia} +1.0 - cos(1e-8) +``` + +### Rational numbers + + +Rational numbers can be used when the exactness of the number is more important than the speed or wider range of values offered by floating point numbers. In `Julia` a rational number is comprised of a numerator and a denominator, each an integer of the same type, and reduced to lowest terms. The operations of addition, subtraction, multiplication, and division will keep their answers as rational numbers. As well, raising a rational number to an integer value will produce a rational number. + + +As mentioned, these are constructed using double slashes: + + +```{julia} +1//2, 2//1, 6//4 +``` + +Rational numbers are exact, so the following are identical to their mathematical counterparts: + + +```{julia} +1//10 + 2//10 == 3//10 +``` + +and associativity: + + +```{julia} +(1//10 + 2//10) + 3//10 == 1//10 + (2//10 + 3//10) +``` + +Here we see that the type is preserved under the basic operations: + + +```{julia} +(1//2 + 1//3 * 1//4 / 1//5) ^ 6 +``` + +For powers, a non-integer exponent is converted to floating point, so this operation is defined, though will always return a floating point value: + + +```{julia} +(1//2)^(1//2) # the first parentheses are necessary as `^` will be evaluated before `//`. +``` + +##### Example: different types of real numbers + + +This table shows what attributes are implemented for the different types. + + +```{julia} +#| echo: false +using DataFrames +attributes = ["construction", "exact", "wide range", "has infinity", "has `-0`", "fast", "closed under"] +integer = [q"1", "true", "false", "false", "false", "true", "`+`, `-`, `*`, `^` (non-negative exponent)"] +rational = ["`1//1`", "true", "false", "false", "false", "false", "`+`, `-`, `*`, `/` (non zero denominator),`^` (integer power)"] +float = [q"1.0", "not usually", "true", "true", "true", "true", "`+`, `-`, `*`, `/` (possibly `NaN`, `Inf`),`^` (non-negative base)"] +d = DataFrame(Attributes=attributes, Integer=integer, Rational=rational, FloatingPoint=float) +table(d) +``` + +### Complex numbers + + +Complex numbers in `Julia` are stored as two numbers, a real and imaginary part, each some type of `Real` number. The special constant `im` is used to represent $i=\sqrt{-1}$. This makes the construction of complex numbers fairly standard: + + +```{julia} +1 + 2im, 3 + 4.0im +``` + +(These two aren't exactly the same, the `3` is promoted from an integer to a float to match the `4.0`. Each of the components must be of the same type of number.) + + +Mathematically, complex numbers are needed so that certain equations can be satisfied. For example $x^2 = -2$ has solutions $-\sqrt{2}i$ and $\sqrt{2}i$ over the complex numbers. Finding this in `Julia` requires some attention, as we have both `sqrt(-2)` and `sqrt(-2.0)` throwing a `DomainError`, as the `sqrt` function expects non-negative real arguments. However first creating a complex number does work: + + +```{julia} +sqrt(-2 + 0im) +``` + +For complex arguments, the `sqrt` function will return complex values (even if the answer is a real number). + + +This means, if you wanted to perform the quadratic equation for any real inputs, your computations might involve something like the following: + + +```{julia} +a,b,c = 1,2,3 ## x^2 + 2x + 3 +discr = b^2 - 4a*c +(-b + sqrt(discr + 0im))/(2a), (-b - sqrt(discr + 0im))/(2a) +``` + +When learning calculus, the only common usage of complex numbers arises when solving polynomial equations for roots, or zeros, though they are very important for subsequent work using the concepts of calculus. + + +:::{.callout-note} +## Note +Though complex numbers are stored as pairs of numbers, the imaginary unit, `im`, is of type `Complex{Bool}`, a type that can be promoted to more specific types when `im` is used with different number types. + +::: + +### Strings and symbols + +For text, `Julia` has a `String` type. When double quotes are used to specify a string, the parser creates this type: + +```{julia} +x = "The quick brown fox jumped over the lazy dog" +typeof(x) +``` + +Values can be inserted into a string through *interpolation* using a dollar sign. + +```{julia} +animal = "lion" +x = "The quick brown $(animal) jumped over the lazy dog" +``` + +The use of parentheses allows more complicated expressions; it isn't always necessary. + +Longer strings can be produced using *triple* quotes: + +```{julia} +lincoln = """ +Four score and seven years ago our fathers brought forth, upon this continent, a new nation, conceived in Liberty, and dedicated to the proposition that all men are created equal. +""" +``` + +Strings are comprised of *characters* which can be produced directly using *single* quotes: + +```{julia} +'c' +``` + +We won't use these. + +Finally, `Julia` has *symbols* which are *interned* strings which are used as identifiers. Symbols are used for advanced programming techniques; we will only see them as shortcuts to specify plotting arguments. + +## Type stability + + +One design priority of `Julia` is that it should be fast. How can `Julia` do this? In a simple model, `Julia` is an interface between the user and the computer's processor(s). Processors consume a set of instructions, the user issues a set of commands. `Julia` is in charge of the translation between the two. Ultimately `Julia` calls a compiler to create the instructions. A basic premise is the shorter the instructions, the faster they are to process. Shorter instructions can come about by being more explicit about what types of values the instructions concern. Explicitness means, there is no need to reason about what a value can be. When `Julia` can reason about the type of value involved without having to reason about the values themselves, it can work with the compiler to produce shorter lists of instructions. + + +So knowing the type of the output of a function based only on the type of the inputs can be a big advantage. In `Julia` this is known as *type stability*. In the standard `Julia` library, this is a primary design consideration. + + +##### Example: closure + + +To motivate this a bit, we discuss how mathematics can be shaped by a desire to stick to simple ideas. A desirable algebraic property of a set of numbers and an operation is *closure*. That is, if one takes an operation like `+` and then uses it to add two numbers in a set, will that result also be in the set? If this is so for any pair of numbers, then the set is closed with respect to the operation addition. + + +Lets suppose we start with the *natural numbers*: $1,2, \dots$. Natural, in that we can easily represent small values in terms of fingers. This set is closed under addition - as a child learns when counting using their fingers. However, if we started with the odd natural numbers, this set would *not* be closed under addition - $3+3=6$. + + +The natural numbers are not all the numbers we need, as once a desire for subtraction is included, we find the set isn't closed. There isn't a $0$, needed as $n-n=0$ and there aren't negative numbers. The set of integers are needed for closure under addition and subtraction. + + +The integers are also closed under multiplication, which for integer values can be seen as just regrouping into longer additions. + + +However, the integers are not closed under division - even if you put aside the pesky issue of dividing by $0$. For that, the rational numbers must be introduced. So aside from division by $0$, the rationals are closed under addition, subtraction, multiplication, and division. There is one more fundamental operation though, powers. + + +Powers are defined for positive integers in a simple enough manner + + +$$ +a^n=a \cdot a \cdot a \cdots a \text{ (n times); } a, n \text{ are integers } n \text{ is positive}. +$$ + +We can define $a^0$ to be $1$, except for the special case of $0^0$, which is left undefined mathematically (though it is also defined as `1` within `Julia`). We can extend the above to include negative values of $a$, but what about negative values of $n$? We can't say the integers are closed under powers, as the definition consistent with the rules that $a^{(-n)} = 1/a^n$ requires rational numbers to be defined. + + +Well, in the above `a` could be a rational number, is `a^n` closed for rational numbers? No again. Though it is fine for $n$ as an integer (save the odd case of $0$, simple definitions like $2^{1/2}$ are not answered within the rationals. For this, we need to introduce the *real* numbers. It is mentioned that [Aristotle](http://tinyurl.com/bpqbkap) hinted at the irrationality of the square root of $2$. To define terms like $a^{1/n}$ for integer values $a,n > 0$ a reference to a solution to an equation $x^n-a$ is used. Such solutions require the irrational numbers to have solutions in general. Hence the need for the real numbers (well, algebraic numbers at least, though once the exponent is no longer a rational number, the full set of real numbers are needed.) + + +So, save the pesky cases, the real numbers will be closed under addition, subtraction, multiplication, division, and powers - provided the base is non-negative. + + +Finally for that last case, the complex numbers are introduced to give an answer to $\sqrt{-1}$. + + +--- + + +How does this apply with `Julia`? + + +The point is, if we restrict our set of inputs, we can get more precise values for the output of basic operations, but to get more general inputs we need to have bigger output sets. + + +A similar thing happens in `Julia`. For addition say, the addition of two integers of the same type will be an integer of that type. This speed consideration is not solely for type stability, but also to avoid checking for overflow. + + +Another example, the division of two integers will always be a number of the same type - floating point, as that is the only type that ensures the answer will always fit within. (The explicit use of rationals notwithstanding.) So even if two integers are the input and their answer *could* be an integer, in `Julia` it will be a floating point number, (cf. `2/1`). + + +Hopefully this helps explain the subtle issues around powers: in `Julia` an integer raised to an integer should be an integer, for speed, though certain cases are special cased, like `2^(-1)`. However since a real number raised to a real number makes sense always when the base is non-negative, as long as real numbers are used as outputs, the expressions `2.0^(-1)` and `2^(-1.0)` are computed and real numbers (floating points) are returned. For type stability, even though $2.0^1$ could be an integer, a floating point answer is returned. + + +As for negative bases, `Julia` could always return complex numbers, but in addition to this being slower, it would be irksome to users. So user's must opt in. Hence `sqrt(-1.0)` will be an error, but the more explicit - but mathematically equivalent - `sqrt(-1.0 + 0im)` will not be a domain error, but rather a complex value will be returned. + + +## Questions + + +```{julia} +#| echo: false +choices = ["Integer", "Rational", "Floating point", "Complex", "None, an error occurs"] +nothing +``` + +###### Question + + +The number created by `pi/2` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 3 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The number created by `2/2` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 3 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The number created by `2//2` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 2 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The number created by `1 + 1//2 + 1/3` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 3 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The number created by `2^3` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 1 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The number created by `sqrt(im)` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 4 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The number created by `2^(-1)` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 3 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The "number" created by `1/0` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 3 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +Is `(2 + 6) + 7` equal to `2 + (6 + 7)`? + + +```{julia} +#| hold: true +#| echo: false +yesnoq(true) +``` + +###### Question + + +Is `(2/10 + 6/10) + 7/10` equal to `2/10 + (6/10 + 7/10)`? + + +```{julia} +#| hold: true +#| echo: false +yesnoq(false) +``` + +###### Question + + +The following *should* compute `2^(-1)`, which if entered directly will return `0.5`. Does it? + + +```{julia} +#| eval: false +a, b = 2, -1 +a^b +``` + +```{julia} +#| hold: true +#| echo: false +yesnoq(false) +``` + +(This shows the special casing that is done when powers use literal numbers.) + + +###### Question + +In [NewScientist](https://www.newscientist.com/article/2112537-smallest-sliver-of-time-yet-measured-sees-electrons-fleeing-atom/) we learn "For the first time, physicists have measured changes in an atom to the level of zeptoseconds, or trillionths of a billionth of a second – the smallest division of time yet observed." + +That is + +```{julia} +1e-9 / 1e12 +``` + +Finding the value through division introduces a floating point deviation. Which of the following values will directly represent a zeptosecond? + +```{julia} +#| echo: false +as = ["1/10^21", "1e-21"] +explanation = "The scientific notation is correct. Due to integer overflow `10^21` is not the same number as `10.0^21`." +buttonq(as, 2; explanation) +``` diff --git a/quarto/basics/numbers_types-II.html b/quarto/basics/numbers_types-II.html new file mode 100644 index 0000000..86c4b3c --- /dev/null +++ b/quarto/basics/numbers_types-II.html @@ -0,0 +1,2804 @@ + + + + + + + + + +numbers_types-ii + + + + + + + + + + + + + + + + + + + + + + + + + + + +

+ +
+ +
+
+

Number systems

+
+ + + +
+ + + + +
+ + + +
+ + +

In mathematics there are many different number systems in common use. For example by the end of pre-calculus, all of the following have been introduced:

+
    +

  • The integers, \(\{\dots, -3, -2, -1, 0, 1, 2, 3, \dots\}\);

    • +

  • The rational numbers, \(\{p/q: p, q \text{ are integers}, q \neq 0\}\);

    • +

  • The real numbers, \(\{x: -\infty < x < \infty\}\);

    • +

  • The complex numbers, \(\{a + bi: a,b \text{ are real numbers and } i^2=-1\}\).

    • +
+

On top of these, we have special subsets, such as the natural numbers \(\{1, 2, \dots\}\) (sometimes including \(0\)), the even numbers, the odd numbers, the positive numbers, the non-negative numbers, etc.

+

Mathematically, these number systems are naturally nested within each other as integers are rational numbers which are real numbers, which can be viewed as part of the complex numbers.

+

Calculators typically have just one type of number—floating point values. These model the real numbers.

+

Julia, on the other hand, has a rich type system, and within that has several different number types. There are types that model each of the four main systems above, and within each type, specializations for how these values are stored.

+

For now, let’s consider the number \(1\). It can be viewed as either an integer, rational, real, or complex number. To construct “\(1\)” in each type within Julia we have these different styles:

+
+
1, 1.0, 1//1, 1 + 0im
+
+
(1, 1.0, 1//1, 1 + 0im)
+
+
+

The basic number types in Julia are Int, Float64, Rational and Complex, though in fact there are many more, and the last two aren’t even concrete types. This distinction is important, as the type of number dictates how it will be displayed, how it will be stored, and how precisely the stored value can be expected to be to the mathematical value it models.

+

Though there are explicit constructors for these types, these notes avoid them unless necessary, as Julia’s parser can distinguish these types through an easy to understand syntax:

+
    +

  • integers have no decimal point;

    • +

  • floating point numbers have a decimal point (or are written with scientific notation);

    • +

  • rationals are constructed from integers using the double division operator, //; and

    • +

  • complex numbers are formed by including a term with the imaginary unit, im.

    • +
+
+
+
+ +
+
+NoteWarning +
+
+
+

Heads up, the difference between 1 and 1.0 is subtle. Even more so, as 1. will parse as 1.0. This means some expressions, such as 2.*3, are ambiguous, as the . might be part of the 2 (as in 2. * 3) or the operation * (as in 2 .* 3).

+
+
+

The key distinction is between integers and floating points. While floating point values include integers, and so can be used exclusively on the calculator, the difference is that an integer is guaranteed to be an exact value, whereas a floating point value, while often an exact representation of a number is also often just an approximate value. This can be an advantage—floating point values can model a much wider range of numbers.

+

In nearly all cases the differences are not noticeable. To see why take, for instance, this simple calculation involving mixed types.

+
+
1 + 1.25 + 3//2
+
+
3.75
+
+
+

The sum of an integer, a floating point number and rational number returns a floating point number without a complaint.

+
+

Promotion

+

This is because behind the scenes, Julia will often “promote” the two numbers to a common type. In particular, before adding mixed-type numbers, the two are promoted to a common type by promote. In the example, first when computing 1 + 1.25 the integer 1 will be promoted to a floating point value, 1.0, and then the two values are added. Similarly, with 2.25 + 3//2, where the fraction is promoted to the floating point value 1.5 and afterwards addition is carried out.

+

We can see the promotion here:

+
+
promote(1, 1.25)
+
+
(1.0, 1.25)
+
+
+

and

+
+
promote(2.25, 3//2)
+
+
(2.25, 1.5)
+
+
+
+
+

Integers

+

Integers are often used casually, as they come about from parsing. As with a calculator, floating point numbers could be used for integers, but in Julia—and other languages—it proves useful to have numbers known to have exact values. Integers are needed for indexing and counting.

+

Except on older machines, the default integer is stored with 64 bits, though there are many available types for integers. With \(64\) bits, the range of integers that can be represented is \(-9223372036854775808=-(2^{63})\) to \(9223372036854775807 = 2^{63}-1\).

+
+
+

Floating point numbers

+

Floating point numbers are a model for the real numbers. With the same size storage, the integers provide exact numbers evenly spaced between the smallest and largest integer. Floating point values are exact for some values but as there are infinitely many real numbers are only approximations except in special cases. This leads to some differences between math done by hand and math done on the computer.

+
+

Float64

+

Float64 is the most common type of floating point number, as it is the most supported by the underlying hardware. Julia has other floating point types, notably Float32 and BigFloat for certain uses, but our focus here is on 64-bit floating point numbers.

+

The double-precision model for floating point numbers has three parts: a sign, an exponent (for a base of \(2\)), and a significand (in base \(2\)) representing numbers as \(\pm a \cdot 2^n\). The 64 bits are apportioned as follows: \(1\) is for the sign, \(11\) for the exponent, \(52\) for the significand.

+

The \(52\) bits of the significand are used to represent \(1.a_1a_2a_3\cdots a_{52}\) in base \(2\) or \(1 + a_12^{-1} + a_22^{-2} + a_32^{-3} + \cdots a_{52}2^{-52} = b/2^{52}\) for some integer \(b\). This means the significand represents a rational number.

+

The following shows the bits in the significand for a given number written in the form above:1

+
+
bitstring(1 + 1/2 + 1/4 + 0/8 + 1/16 + 1/32 + 0/64)[13:end] # 1101100…
+
+
"1101100000000000000000000000000000000000000000000000"
+
+
+

The 11 bits for the exponent covers a range from \(-1023\) to \(1024\) which in base \(10\) is around \(10^{-308}\) to \(10^{308}\).

+

Together these can represent exactly any rational number of the form \(\pm a \cdot 2^b\) where \(a\) is a sum of powers of \(1/2\) and \(b\) is an integer with \(1.0 \leq a \leq 1 + (1/2^1) + (1/2^2) + \cdots + (1/2^{52})\) and \(-1023 \leq b \leq 1024\).

+

Figure 1 shows the possible positive values were there only \(2\) bits for the exponent (for \(-1, 0, 1, 2\)) and \(2\) bits for the significand (\(1 + 0/4 + 0/2\), \(1 + 1/4 + 0/2\), \(1 + 0/4 + 1/2\), \(1 + 1/4 + 1/2\)). The main takeaway is that numbers get less concentrated the farther they get from \(0\).

+
+
+
+
+
+
+ +
+
+
+
+
+Figure 1: Figure showing concentration of floating point values. The vertical ticks represent representable floating point values (were there only 2 bits (not 52) for the mantissa) and 2 bits (not 11) for the exponent. This leaves a range from a range of \(-1/2\) to not quite \(8\) being representable without using subnormal numbers. When the intervals double in length (from \([2^{i},2^{i+1}]\) to \([2^{i+1}, 2^{i+2}]\)) there are the same number of representable floating point values, so the concentration of representable values halves. With more bits there is a higher concentration, but the discrete nature is always present and leads to necessary approximations for modeling most all real numbers. +
+
+
+

In addition, there are special bit patterns recognized as 0.0 and even -0.0, which is a distinct number. There are also patterns for \(+\infty\) (Inf) and \(-\infty\) (-Inf). There are also patterns for NaN, or “not a number”, a value that is the result of some mathematical operations, such as 0.0 / 0.0. Finally, there are subnormal numbers representing even smaller numbers near \(0\) than described above, which are as small as \(2^{-1023} \approx 1.11 \cdot 10^{-308}\).

+
+
+

Scientific notation

+

Floating point numbers smaller than \(10^{-4}\) or bigger or equal to \(10^6\) (in absolute value) are displayed in scientific notation. Internally, most floating point numbers are stored in base \(2\) scientific notation as \(a \cdot 2^b\) with \(a=1.xxx\dots\). But when displayed, numbers are represented in base \(10\) and when scientific notation is used the numbers are normalized in the from \(a \cdot 10^b\) where \(1.0 \leq a < 10\).

+

The significand and exponent are separated by the character e—which is not the same as the constant \(e\)—rather denotes a 64-bit number separated into a significand and an exponent by a formatting character. (Float32 uses an f as a separator.)

+

Consider these two numbers one close to \(0\) one far from \(0\):

+
+
0.0000000123456789, 123456789.0
+
+
(1.23456789e-8, 1.23456789e8)
+
+
+

Their display is subtly different, as only a minus sign after e distinguishes them.

+

The parser will read in numbers with an e in the proper format as though they are scientific notation:

+
+
1e8
+
+
1.0e8
+
+
+

The above creates the same value as 10.0^8, but not 1e^8 which will error unless a value for e has been assigned.

+
+
+

Inexactness and consequences

+

For numbers not representable in floating point, some rounding must go on to fit the number into a representable floating point value. As such, some computed values are not quite what they would be mathematically:

+
+
sqrt(2) * sqrt(2) - 2, sin(1pi)
+
+
(4.440892098500626e-16, 1.2246467991473532e-16)
+
+
+

These values are very small numbers, but not exactly \(0\), as they are mathematically.

+

More surprisingly, simple fractions may also lead to mathematically different results:

+
+
1/10 + 2/10 - 3/10
+
+
5.551115123125783e-17
+
+
+

This, of course, is due to none of these fractions being of the form \(a\cdot 2^b\) for integers \(a, b\).

+

Another surprise: floating point addition is not necessarily associative. That is the property \(a + (b+c) = (a+b) + c\) may not hold exactly. For example:

+
+
l2r = (1/10 + 2/10) + 3/10
+r2l = 1/10 + (2/10 + 3/10)
+l2r - r2l
+
+
1.1102230246251565e-16
+
+
+

One other surprise. Mathematically, for real numbers, subtraction of similar-sized numbers is not exceptional, for example \(1 - \cos(x)\) is positive if \(0 < x < \pi/2\), say. This will not be the case for floating point values. If \(x\) is close enough to \(0\), then \(\cos(x)\) and \(1\) will be so close, that they will be represented by the same floating point value, 1.0, so the difference will be zero:

+
+
1.0 - cos(1e-8)
+
+
0.0
+
+
+
+
+
+

Rational numbers

+

Rational numbers can be used when the exactness of the number is more important than the speed or wider range of values offered by floating point numbers. In Julia a rational number is comprised of a numerator and a denominator, each an integer of the same type, and reduced to lowest terms. The operations of addition, subtraction, multiplication, and division will keep their answers as rational numbers. As well, raising a rational number to an integer value will produce a rational number.

+

As mentioned, these are constructed using double slashes:

+
+
1//2, 2//1, 6//4
+
+
(1//2, 2//1, 3//2)
+
+
+

Rational numbers are exact, so the following are identical to their mathematical counterparts:

+
+
1//10 + 2//10 == 3//10
+
+
true
+
+
+

and associativity:

+
+
(1//10 + 2//10) + 3//10 == 1//10 + (2//10 + 3//10)
+
+
true
+
+
+

Here we see that the type is preserved under the basic operations:

+
+
(1//2 + 1//3 * 1//4 / 1//5) ^ 6
+
+
1771561//2985984
+
+
+

For powers, a non-integer exponent is converted to floating point, so this operation is defined, though will always return a floating point value:

+
+
(1//2)^(1//2)   # the first parentheses are necessary as `^` will be evaluated before `//`.
+
+
0.7071067811865476
+
+
+
+
+

Complex numbers

+

Complex numbers in Julia are stored as two numbers, a real and imaginary part, each some type of Real number. The special constant im is used to represent \(i=\sqrt{-1}\). This makes the construction of complex numbers fairly standard:

+
+
1 + 2im, 3 + 4.0im
+
+
(1 + 2im, 3.0 + 4.0im)
+
+
+

(These two aren’t exactly the same, the 3 is promoted from an integer to a float to match the 4.0. Each of the components must be of the same type of number.)

+

Mathematically, complex numbers are needed so that certain equations can be satisfied. For example \(x^2 = -2\) has solutions \(-\sqrt{2}i\) and \(\sqrt{2}i\) over the complex numbers. Finding this in Julia requires some attention, as we have both sqrt(-2) and sqrt(-2.0) throwing a DomainError, as the sqrt function expects non-negative real arguments. However first creating a complex number and then taking a square root does work:

+
+
sqrt(-2 + 0im)
+
+
0.0 + 1.4142135623730951im
+
+
+

For complex arguments, the sqrt function will return complex values (even if the answer is a real number).

+

This means, if you wanted to perform the quadratic equation for any real inputs, your computations might involve something like the following:

+
+
a,b,c = 1,2,3  ## x^2 + 2x + 3
+discr = b^2 - 4a*c
+(-b + sqrt(discr + 0im))/(2a), (-b - sqrt(discr + 0im))/(2a)
+
+
(-1.0 + 1.4142135623730951im, -1.0 - 1.4142135623730951im)
+
+
+

When learning calculus, the only common usage of complex numbers arises when solving polynomial equations for roots, or zeros, though they are very important for subsequent work using the concepts of calculus.

+
+
+

Irrational numbers

+

Julia has a a few mathematical constants that are stored with a special type Irrational. One such value is pi. There are others in the Base.MathConstants module, and an external package IrrationalConstants.jl.

+

Irrational values may have special methods defined for them which can lead to subtle differences, such as:

+
+
sin(pi), sin(2pi)
+
+
(0.0, -2.4492935982947064e-16)
+
+
+

In computing the product 2pi first the two values are promoted to Float64 and then multiplied, leaving a floating-point approximation of \(2\pi\) for sin to evaluate.

+
+
+

Other types of data: strings and symbols

+

For text, Julia has a String type. When double quotes are used to specify a string, the parser creates this type:

+
+
x = "The quick brown fox jumped over the lazy dog"
+typeof(x)
+
+
String
+
+
+

Values can be inserted into a string through interpolation using a dollar sign.

+
+
animal = "lion"
+x = "The quick brown $(animal) jumped over the lazy dog"
+
+
"The quick brown lion jumped over the lazy dog"
+
+
+

The use of parentheses allows more complicated expressions; it isn’t always necessary.

+

Longer strings can be produced using triple quotes:

+
+
lincoln = """
+Four score and seven years ago our fathers brought forth, upon this continent, a new nation, conceived in Liberty, and dedicated to the proposition that all men are created equal.
+"""
+
+
"Four score and seven years ago our fathers brought forth, upon this continent, a new nation, conceived in Liberty, and dedicated to the proposition that all men are created equal.\n"
+
+
+

Strings are comprised of characters which can be produced directly using single quotes:

+
+
'c'
+
+
'c': ASCII/Unicode U+0063 (category Ll: Letter, lowercase)
+
+
+

We won’t use characters in these notes.

+

Finally, Julia has symbols which are interned strings which are used as identifiers. Symbols are used for advanced programming techniques; we will only see them as shortcuts to specify plotting arguments.

+
+
+

Questions

+
+
Question
+

The number created by pi/2 is?

+
+
+ +
+
+
+
+ +
+
+Select an item +
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+
+
+ + + + +
+
+
+
+
Question
+

The number created by 2/2 is?

+
+
+ +
+
+
+
+ +
+
+Select an item +
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+
+
+ + + + +
+
+
+
+
Question
+

The number created by 2//2 is?

+
+
+ +
+
+
+
+ +
+
+Select an item +
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+
+
+ + + + +
+
+
+
+
Question
+

The number created by 1 + 1//2 + 1/3 is?

+
+
+ +
+
+
+
+ +
+
+Select an item +
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+
+
+ + + + +
+
+
+
+
Question
+

The number created by 2^3 is?

+
+
+ +
+
+
+
+ +
+
+Select an item +
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+
+
+ + + + +
+
+
+
+
Question
+

The number created by sqrt(im) is?

+
+
+ +
+
+
+
+ +
+
+Select an item +
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+
+
+ + + + +
+
+
+
+
Question
+

The number created by 2^(-1) is?

+
+
+ +
+
+
+
+ +
+
+Select an item +
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+
+
+ + + + +
+
+
+
+
Question
+

The “number” created by 1/0 is?

+
+
+ +
+
+
+
+ +
+
+Select an item +
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+
+
+ + + + +
+
+
+
+
Question
+

Is (2 + 6) + 7 equal to 2 + (6 + 7)?

+
+
+ +
+
+
+
+ +
+
+Select an item +
+ +
+
+ +
+
+ +
+
+
+
+ + + + +
+
+
+
+
Question
+

Is (2/10 + 6/10) + 7/10 equal to 2/10 + (6/10 + 7/10)?

+
+
+ +
+
+
+
+ +
+
+Select an item +
+ +
+
+ +
+
+ +
+
+
+
+ + + + +
+
+
+
+
Question
+

The following should compute 2^(-1), which if entered directly will return 0.5. Does it?

+
+
a, b = 2, -1
+a^b
+
+
+
+ +
+
+
+
+ +
+
+Select an item +
+ +
+
+ +
+
+ +
+
+
+
+ + + + +
+
+

(This shows the special casing that is done when powers use literal numbers.)

+
+
+
Question
+

In NewScientist we learn “For the first time, physicists have measured changes in an atom to the level of zeptoseconds, or trillionths of a billionth of a second—the smallest division of time yet observed.”

+

That is

+
+
1e-9 / 1e12
+
+
1.0000000000000001e-21
+
+
+

Finding the value through division introduces a floating point deviation. Which of the following values will directly represent a zeptosecond?

+
+
+ +
+
+
+
+ +
+
+ + +
+ + +
+
+
+ +
+ + + + +
+
+
+
+
Question
+

Signed integers are stored on a computer in a special manner. We will see with 8 bit integers, formed by Int8. Eight bit means only 8 0’s or 1’s are used to store a given number. This is a useful format for storing many small integers but for this example, useful as we can more easily track the values.

+

The first bit is a sign bit. Based on these two outputs, can you guess how that works:

+
+
bitstring(Int8(-1)), bitstring(Int8(1))
+
+
("11111111", "00000001")
+
+
+
+
+ +
+
+
+
+ +
+
+ + +
+ + +
+
+
+
+ + + + +
+
+

Positive numbers and negative numbers are stored a bit differently. Positive numbers just use binary: \(a_0 \cdot 2^0 + a_1 \cdot 2^1 + a_2 \cdot 2^2 + \cdots a_7 \cdot 2^7\). The number \(27\) is \(1 + 2 + 8 + 16\). so have \(a_0 = a_1 = a_3 = a_4 = 1\), the others are \(0\). The bitstring shows:

+
+
bitstring(Int8(27))
+
+
"00011011"
+
+
+

Which bit pattern is used?

+
+
+ +
+
+
+
+ +
+
+ + +
+ + +
+
+
+
+ + + + +
+
+

Negative numbers are stored using two’s complement format:

+
    +
  • represent the positive number
    • +
  • flip 0 to 1; 1 to 0
    • +
  • add 1 to the value (long addition with carrying)
    • +
+

For \(-27\) we have

+
    +
  • first \(27\): 00011011
    • +
  • then flip each: 11100100
    • +
  • finally add \(1\): 11100101
    • +
+
+
bitstring(Int8(-27))
+
+
"11100101"
+
+
+

The largest positive number is \(127\) for 8-bits and is represented by 01111111. What is the bit pattern of \(-127\)?

+
+
+ +
+
+
+
+ +
+
+ + +
+ + +
+
+
+ +
+ + + + +
+
+

The smallest negative number is \(-128\). Why?

+
+
+ +
+
+
+
+ +
+
+ + +
+ + +
+
+
+
+ + + + +
+
+

Why all this fuss? Couldn’t there be an easier way?

+

This storage has a big advantage when adding numbers. Let’s look at adding \(-5\) to \(6\). we have:

+
-5 => 11111011
+ 6 => 00000110
+      --------
+     100000001  => 00000001
+

The addition is done by carrying a 1 across and then dropping the 9th number when there is such a carry. This leaves the representation for what number?

+
+
+ +
+
+
+
+ +
+
+ + + +
+ + +
+
+
+
+ + + + +
+
+

The largest positive number that can be represented is \(2^0 + 2^1 + 2^2 + \cdots + 2^7\), where \(7\) is the number of bits minus \(1\). The representation is 01111111. What happens if we add 1 to this number?

+
+
+ +
+
+
+
+ +
+
+ + +
+ + +
+
+
+
+ + + + +
+
+

The largest possible number for a type is returned by typemax. For Int64 (just Int on most systems) what is the largest number?

+
+
+ +
+
+
+
+ +
+
+
+ +
+ +
+
+
+
+ + + + +
+
+
+
+
Question
+

The Float64 type uses \(11\) bits for an exponent (base \(2\)) between \(-1023\) and \(1024\). We can see how these are stored as follows:

+
+
bitstring(2.0^(-1023))[2:12], bitstring(2.0^(1024))[2:12]
+
+
("00000000000", "11111111111")
+
+
+

This is the full range of values. However the values are shifted with \(0\) representing \(-1023\) and \(x\) representing \(1024\). The value \(x\) is can be found from:

+
+
2^0 + 2^1 + 2^2 + 2^3 + 2^4 + 2^5 + 2^6 + 2^7 + 2^8 + 2^9 + 2^10
+
+

What is the value of \(x\)?

+
+
+ +
+
+
+
+ +
+
+
+ +
+ +
+
+
+
+ + + + +
+
+

The value 1023 is called a bias. The exponent is coded as the binary value as a positve integer minus \(1023\). A bias is used, and not the two’s complement format, as storage with a bias makes multiplying by powers of \(2\) as easy as shifting the bits.

+

To find the storage for, say, \(2^4 + 2^2 + 2^0\) or 00000010101 we would add 1023 or 01111111111 and see:

+
  00000010101
++ 01111111111
+  -----------
+  10000010100
+

Which we can see:

+
+
bitstring(2.0^(2^4 + 2^2 + 2^0))[2:12]
+
+
"10000010100"
+
+
+
+
+ + +

Footnotes

+ +
    +

  1. The output of bitstring is 64 characters. The first is the sign bit, the second through twelfth the exponent, the rest the significand. The notation [13:end] is used to return just those for the significand. A value of [2:12] would return the bits for the exponent.↩︎

    2. +
+
+ + +
+ + + + + \ No newline at end of file diff --git a/quarto/basics/numbers_types-II.qmd b/quarto/basics/numbers_types-II.qmd new file mode 100644 index 0000000..8e56a0b --- /dev/null +++ b/quarto/basics/numbers_types-II.qmd @@ -0,0 +1,675 @@ +# Number systems + + +{{< include ../_common_code.qmd >}} + +```{julia} +#| echo: false +#| results: "hidden" +using CalculusWithJulia + +nothing +``` + +In mathematics there are many different number systems in common use. For example by the end of pre-calculus, all of the following have been introduced: + + +* The integers, $\{\dots, -3, -2, -1, 0, 1, 2, 3, \dots\}$; + +* The rational numbers, $\{p/q: p, q \text{ are integers}, q \neq 0\}$; + +* The real numbers, $\{x: -\infty < x < \infty\}$; + +* The complex numbers, $\{a + bi: a,b \text{ are real numbers and } i^2=-1\}$. + + +On top of these, we have special subsets, such as the natural numbers $\{1, 2, \dots\}$ (sometimes including $0$), the even numbers, the odd numbers, the positive numbers, the non-negative numbers, etc. + + +Mathematically, these number systems are naturally nested within each other as integers are rational numbers which are real numbers, which can be viewed as part of the complex numbers. + + +Calculators typically have just one type of number---floating point values. These model the real numbers. + +`Julia`, on the other hand, has a rich type system, and within that has several different number types. There are types that model each of the four main systems above, and within each type, specializations for how these values are stored. + + +For now, let's consider the number $1$. It can be viewed as either an integer, rational, real, or complex number. To construct "$1$" in each type within `Julia` we have these different styles: + + +```{julia} +1, 1.0, 1//1, 1 + 0im +``` + +The basic number types in `Julia` are `Int`, `Float64`, `Rational` and `Complex`, though in fact there are many more, and the last two aren't even *concrete* types. This distinction is important, as the type of number dictates how it will be displayed, how it will be stored, and how precisely the stored value can be expected to be to the mathematical value it models. + + +Though there are explicit constructors for these types, these notes avoid them unless necessary, as `Julia`'s parser can distinguish these types through an easy to understand syntax: + + +* integers have no decimal point; + +* floating point numbers have a decimal point (or are written with scientific notation); + +* rationals are constructed from integers using the double division operator, `//`; and + +* complex numbers are formed by including a term with the imaginary unit, `im`. + + +:::{.callout-note} +## Warning +Heads up, the difference between `1` and `1.0` is subtle. Even more so, as `1.` will parse as `1.0`. This means some expressions, such as `2.*3`, are ambiguous, as the `.` might be part of the `2` (as in `2. * 3`) or the operation `*` (as in `2 .* 3`). + +::: + + +The key distinction is between integers and floating points. While floating point values include integers, and so can be used exclusively on the calculator, the difference is that an integer is guaranteed to be an exact value, whereas a floating point value, while often an exact representation of a number is also often just an *approximate* value. This can be an advantage---floating point values can model a much wider range of numbers. + +In nearly all cases the differences are not noticeable. To see why take, for instance, this simple calculation involving mixed types. + + +```{julia} +1 + 1.25 + 3//2 +``` + +The sum of an integer, a floating point number and rational number returns a floating point number without a complaint. + +### Promotion + +This is because behind the scenes, `Julia` will often "promote" the two numbers to a common type. In particular, before adding mixed-type numbers, the two are promoted to a common type by `promote`. In the example, first when computing `1 + 1.25` the integer `1` will be promoted to a floating point value, `1.0`, and then the two values are added. Similarly, with `2.25 + 3//2`, where the fraction is promoted to the floating point value `1.5` and afterwards addition is carried out. + +We can see the promotion here: + +```{julia} +promote(1, 1.25) +``` + +and + +```{julia} +promote(2.25, 3//2) +``` + + + +## Integers + +Integers are often used casually, as they come about from parsing. As with a calculator, floating point numbers *could* be used for integers, but in `Julia`---and other languages---it proves useful to have numbers known to have *exact* values. Integers are needed for indexing and counting. + +Except on older machines, the default integer is stored with 64 bits, though there are many available types for integers. With $64$ bits, the range of integers that can be represented is $-9223372036854775808=-(2^{63})$ to $9223372036854775807 = 2^{63}-1$. + +## Floating point numbers + +Floating point numbers are a model for the real numbers. With the same size storage, the integers provide exact numbers evenly spaced between the smallest and largest integer. Floating point values are exact for some values but as there are infinitely many real numbers are only approximations except in special cases. This leads to some differences between math done by hand and math done on the computer. + +### Float64 + +`Float64` is the most common type of floating point number, as it is the most supported by the underlying hardware. `Julia` has other floating point types, notably `Float32` and `BigFloat` for certain uses, but our focus here is on 64-bit floating point numbers. + +The double-precision model for floating point numbers has three parts: a sign, an exponent (for a base of $2$), and a significand (in base $2$) representing numbers as $\pm a \cdot 2^n$. The 64 bits are apportioned as follows: $1$ is for the sign, $11$ for the exponent, $52$ for the significand. + + +The $52$ bits of the significand are used to represent $1.a_1a_2a_3\cdots a_{52}$ in base $2$ or $1 + a_12^{-1} + a_22^{-2} + a_32^{-3} + \cdots a_{52}2^{-52} = b/2^{52}$ for some integer $b$. This means the significand represents a rational number. + +The following shows the bits in the significand for a given number written in the form above:^[The output of `bitstring` is 64 characters. The first is the sign bit, the second through twelfth the exponent, the rest the significand. The notation `[13:end]` is used to return just those for the significand. A value of `[2:12]` would return the bits for the exponent.] + +```{julia} +bitstring(1 + 1/2 + 1/4 + 0/8 + 1/16 + 1/32 + 0/64)[13:end] # 1101100… +``` + + +The 11 bits for the exponent covers a range from $-1023$ to $1024$ which in base $10$ is around $10^{-308}$ to $10^{308}$. + +Together these can represent exactly any *rational* number of the form $\pm a \cdot 2^b$ where $a$ is a sum of powers of $1/2$ and $b$ is an integer with $1.0 \leq a \leq 1 + (1/2^1) + (1/2^2) + \cdots + (1/2^{52})$ and $-1023 \leq b \leq 1024$. + +@fig-floating-point-concentration shows the possible positive values *were* there only $2$ bits for the exponent (for $-1, 0, 1, 2$) and $2$ bits for the significand ($1 + 0/4 + 0/2$, $1 + 1/4 + 0/2$, $1 + 0/4 + 1/2$, $1 + 1/4 + 1/2$). The main takeaway is that numbers get less concentrated the farther they get from $0$. + +::: {#fig-floating-point-concentration layout-ncol=1} + +```{julia} +#| echo: false +#| fig: floating_point_concentration +#| fig-label: floating point concentration +using Plots +gr() +using LaTeXStrings +xvals = [0, 1/2, 1, 2, 4, 8] +xlabels = [L"0", L"\frac{1}{2}", L"1", L"2", L"4", L"8"] +p = plot(; size=(600, 200), xlim = (0, 8.5), ylim=(-1, 1), + xaxis=([], false), + yaxis=([], false), + framestyle=:origin, + legend=false, + ) + +plot!(p, [(0,0), (8.5,0)]; arrow=true, line=(2, :black)) +xs = range(0, 1 - 1/2, 5)[1:end-1] +for (i,xi) in enumerate(xvals[2:end-1]) + for x in xs + x₀ = xi + 2^(i-0)*x/2 + Δ = 1/20 + plot!(p, [(x₀, -Δ), (x₀, 2Δ)]; linecolor=:black) + end +end + +plot!(p; annotations=tuple.(xvals, zero.(xvals), text.(xlabels, :top))) +p +``` + +Figure showing concentration of floating point values. The vertical ticks represent representable floating point values (were there only 2 bits (not 52) for the mantissa) and 2 bits (not 11) for the exponent. This leaves a range from a range of $-1/2$ to not quite $8$ being representable *without* using subnormal numbers. When the intervals double in length (from $[2^{i},2^{i+1}]$ to $[2^{i+1}, 2^{i+2}]$) there are the same number of representable floating point values, so the concentration of representable values halves. With more bits there is a higher concentration, but the discrete nature is always present and leads to necessary approximations for modeling most all real numbers. +::: + +In addition, there are special bit patterns recognized as `0.0` and even `-0.0`, which is a distinct number. There are also patterns for $+\infty$ (`Inf`) and $-\infty$ (`-Inf`). There are also patterns for `NaN`, or "not a number", a value that is the result of some mathematical operations, such as `0.0 / 0.0`. Finally, there are *subnormal* numbers representing even smaller numbers near $0$ than described above, which are as small as $2^{-1023} \approx 1.11 \cdot 10^{-308}$. + + +### Scientific notation + +Floating point numbers smaller than $10^{-4}$ or bigger or equal to $10^6$ (in absolute value) are displayed in scientific notation. Internally, most floating point numbers are stored in base $2$ scientific notation as $a \cdot 2^b$ with $a=1.xxx\dots$. But when displayed, numbers are represented in base $10$ and when scientific notation is used the numbers are normalized in the from $a \cdot 10^b$ where $1.0 \leq a < 10$. + +The significand and exponent are separated by the character `e`---which is not the same as the constant $e$---rather denotes a 64-bit number separated into a significand and an exponent by a formatting character. (`Float32` uses an `f` as a separator.) + +Consider these two numbers one close to $0$ one far from $0$: + +```{julia} +0.0000000123456789, 123456789.0 +``` + +Their display is subtly different, as only a minus sign after `e` distinguishes them. + +The parser will read in numbers with an `e` in the proper format as though they are scientific notation: + +```{julia} +1e8 +``` + +The above creates the same value as `10.0^8`, but not `1e^8` which will error unless a value for `e` has been assigned. + +### Inexactness and consequences + +For numbers not representable in floating point, some rounding must go on to fit the number into a representable floating point value. As such, some computed values are not quite what they would be mathematically: + +```{julia} +sqrt(2) * sqrt(2) - 2, sin(1pi) +``` + +These values are *very* small numbers, but not exactly $0$, as they are mathematically. + +More surprisingly, simple fractions may also lead to mathematically different results: + +```{julia} +1/10 + 2/10 - 3/10 +``` + +This, of course, is due to none of these fractions being of the form $a\cdot 2^b$ for integers $a, b$. + +Another surprise: floating point addition is not necessarily associative. That is the property $a + (b+c) = (a+b) + c$ may not hold exactly. For example: + + +```{julia} +l2r = (1/10 + 2/10) + 3/10 +r2l = 1/10 + (2/10 + 3/10) +l2r - r2l +``` + +One other surprise. Mathematically, for real numbers, subtraction of similar-sized numbers is not exceptional, for example $1 - \cos(x)$ is positive if $0 < x < \pi/2$, say. This will not be the case for floating point values. If $x$ is close enough to $0$, then $\cos(x)$ and $1$ will be so close, that they will be represented by the same floating point value, `1.0`, so the difference will be zero: + + +```{julia} +1.0 - cos(1e-8) +``` + +## Rational numbers + + +Rational numbers can be used when the exactness of the number is more important than the speed or wider range of values offered by floating point numbers. In `Julia` a rational number is comprised of a numerator and a denominator, each an integer of the same type, and reduced to lowest terms. The operations of addition, subtraction, multiplication, and division will keep their answers as rational numbers. As well, raising a rational number to an integer value will produce a rational number. + + +As mentioned, these are constructed using double slashes: + + +```{julia} +1//2, 2//1, 6//4 +``` + +Rational numbers are exact, so the following are identical to their mathematical counterparts: + + +```{julia} +1//10 + 2//10 == 3//10 +``` + +and associativity: + + +```{julia} +(1//10 + 2//10) + 3//10 == 1//10 + (2//10 + 3//10) +``` + +Here we see that the type is preserved under the basic operations: + + +```{julia} +(1//2 + 1//3 * 1//4 / 1//5) ^ 6 +``` + +For powers, a non-integer exponent is converted to floating point, so this operation is defined, though will always return a floating point value: + + +```{julia} +(1//2)^(1//2) # the first parentheses are necessary as `^` will be evaluated before `//`. +``` + +## Complex numbers + +Complex numbers in `Julia` are stored as two numbers, a real and imaginary part, each some type of `Real` number. The special constant `im` is used to represent $i=\sqrt{-1}$. This makes the construction of complex numbers fairly standard: + + +```{julia} +1 + 2im, 3 + 4.0im +``` + +(These two aren't exactly the same, the `3` is promoted from an integer to a float to match the `4.0`. Each of the components must be of the same type of number.) + + +Mathematically, complex numbers are needed so that certain equations can be satisfied. For example $x^2 = -2$ has solutions $-\sqrt{2}i$ and $\sqrt{2}i$ over the complex numbers. Finding this in `Julia` requires some attention, as we have both `sqrt(-2)` and `sqrt(-2.0)` throwing a `DomainError`, as the `sqrt` function expects non-negative real arguments. However first creating a complex number and then taking a square root does work: + + +```{julia} +sqrt(-2 + 0im) +``` + +For complex arguments, the `sqrt` function will return complex values (even if the answer is a real number). + + +This means, if you wanted to perform the quadratic equation for any real inputs, your computations might involve something like the following: + + +```{julia} +a,b,c = 1,2,3 ## x^2 + 2x + 3 +discr = b^2 - 4a*c +(-b + sqrt(discr + 0im))/(2a), (-b - sqrt(discr + 0im))/(2a) +``` + +When learning calculus, the only common usage of complex numbers arises when solving polynomial equations for roots, or zeros, though they are very important for subsequent work using the concepts of calculus. + +## Irrational numbers + +`Julia` has a a few mathematical constants that are stored with a special type `Irrational`. One such value is `pi`. There are others in the `Base.MathConstants` module, and an external package `IrrationalConstants.jl`. + +`Irrational` values may have special methods defined for them which can lead to subtle differences, such as: + +```{julia} +sin(pi), sin(2pi) +``` + +In computing the product `2pi` first the two values are promoted to `Float64` and then multiplied, leaving a floating-point approximation of $2\pi$ for `sin` to evaluate. + +## Other types of data: strings and symbols + + +For text, `Julia` has a `String` type. When double quotes are used to specify a string, the parser creates this type: + +```{julia} +x = "The quick brown fox jumped over the lazy dog" +typeof(x) +``` + +Values can be inserted into a string through *interpolation* using a dollar sign. + +```{julia} +animal = "lion" +x = "The quick brown $(animal) jumped over the lazy dog" +``` + +The use of parentheses allows more complicated expressions; it isn't always necessary. + +Longer strings can be produced using *triple* quotes: + +```{julia} +lincoln = """ +Four score and seven years ago our fathers brought forth, upon this continent, a new nation, conceived in Liberty, and dedicated to the proposition that all men are created equal. +""" +``` + +Strings are comprised of *characters* which can be produced directly using *single* quotes: + +```{julia} +'c' +``` + +We won't use characters in these notes. + +Finally, `Julia` has *symbols* which are *interned* strings which are used as identifiers. Symbols are used for advanced programming techniques; we will only see them as shortcuts to specify plotting arguments. + + + + +## Questions + + +```{julia} +#| echo: false +choices = ["Integer", "Rational", "Floating point", "Complex", "None, an error occurs"] +nothing +``` + +###### Question + + +The number created by `pi/2` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 3 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The number created by `2/2` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 3 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The number created by `2//2` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 2 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The number created by `1 + 1//2 + 1/3` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 3 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The number created by `2^3` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 1 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The number created by `sqrt(im)` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 4 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The number created by `2^(-1)` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 3 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +The "number" created by `1/0` is? + + +```{julia} +#| hold: true +#| echo: false +answ = 3 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +Is `(2 + 6) + 7` equal to `2 + (6 + 7)`? + + +```{julia} +#| hold: true +#| echo: false +yesnoq(true) +``` + +###### Question + + +Is `(2/10 + 6/10) + 7/10` equal to `2/10 + (6/10 + 7/10)`? + + +```{julia} +#| hold: true +#| echo: false +yesnoq(false) +``` + +###### Question + + +The following *should* compute `2^(-1)`, which if entered directly will return `0.5`. Does it? + + +```{julia} +#| eval: false +a, b = 2, -1 +a^b +``` + +```{julia} +#| hold: true +#| echo: false +yesnoq(false) +``` + +(This shows the special casing that is done when powers use literal numbers.) + + +###### Question + +In [NewScientist](https://www.newscientist.com/article/2112537-smallest-sliver-of-time-yet-measured-sees-electrons-fleeing-atom/) we learn "For the first time, physicists have measured changes in an atom to the level of zeptoseconds, or trillionths of a billionth of a second---the smallest division of time yet observed." + +That is + +```{julia} +1e-9 / 1e12 +``` + +Finding the value through division introduces a floating point deviation. Which of the following values will directly represent a zeptosecond? + +```{julia} +#| echo: false +as = ["1/10^21", "1e-21"] +explanation = "The scientific notation is correct. Due to integer overflow `10^21` is not the same number as `10.0^21`." +buttonq(as, 2; explanation) +``` + +###### Question + +Signed integers are stored on a computer in a special manner. We will see with 8 bit integers, formed by `Int8`. Eight bit means only 8 0's or 1's are used to store a given number. This is a useful format for storing many small integers but for this example, useful as we can more easily track the values. + +The first bit is a sign bit. Based on these two outputs, can you guess how that works: + +```{julia} +bitstring(Int8(-1)), bitstring(Int8(1)) +``` + +```{julia} +#| echo: false +choices = ["The first digit is `1` for positive numbers, `0` for negative numbers", + "The first digit is `0` for positive numbers, `1` for negative numbers"] +answer = 2 +buttonq(choices, answer) +``` + +Positive numbers and negative numbers are stored a bit differently. Positive numbers just use binary: $a_0 \cdot 2^0 + a_1 \cdot 2^1 + a_2 \cdot 2^2 + \cdots a_7 \cdot 2^7$. The number $27$ is $1 + 2 + 8 + 16$. so have $a_0 = a_1 = a_3 = a_4 = 1$, the others are $0$. The bitstring shows: + +```{julia} +bitstring(Int8(27)) +``` + +Which bit pattern is used? + +```{julia} +#| echo: false +choices = ["`sign bit | a₀ | a₁ | a₂ | a₃ | a₄ | a₅ | a₆ | a₇`", + "`sign bit | a₇ | a₆ | a₅ | a₄ | a₃ | a₂ | a₁ | a₀`"] +answer = 2 +buttonq(choices, answer) +``` + +Negative numbers are stored using *two's complement* format: + +* represent the positive number +* flip `0` to `1`; `1` to `0` +* add `1` to the value (long addition with carrying) + +For $-27$ we have + +* first $27$: `00011011` +* then flip each: `11100100` +* finally add $1$: `11100101` + +```{julia} +bitstring(Int8(-27)) +``` + +The largest positive number is $127$ for 8-bits and is represented by `01111111`. What is the bit pattern of $-127$? + +```{julia} +#| echo: false +choices = ["`01111111 => 10000000 => 10000001`", + "`01111111 => 10000000 => 100000000`"] +answer = 1 +explanation = "Add the `1` on the right side, not the left side" +buttonq(choices, answer; explanation) +``` + +The smallest negative number is $-128$. Why? + +```{julia} +#| echo: false +choices = ["Subtracting 1 from `10000001` leaves `10000000` which is the smallest negative number", + "Tricky, the smallest negative number is ``-127``" + ] +answer = 1 +buttonq(choices, answer) +``` + +Why all this fuss? Couldn't there be an easier way? + +This storage has a big advantage when adding numbers. Let's look at adding $-5$ to $6$. we have: + +``` +-5 => 11111011 + 6 => 00000110 + -------- + 100000001 => 00000001 +``` + +The addition is done by carrying a `1` across and then *dropping* the 9th number when there is such a carry. This leaves the representation for what number? + +```{julia} +#| echo: false +choices = ["`1`", "`-1`", "`0`"] +answer = 1 +buttonq(choices, answer) +``` + + +The largest *positive* number that can be represented is $2^0 + 2^1 + 2^2 + \cdots + 2^7$, where $7$ is the number of bits minus $1$. The representation is `01111111`. What happens if we add `1` to this number? + + +```{julia} +#| echo: false +choices = ["The value after carrying is `100000000`. After dropping the 9th number, we get `00000000` or zero", + "The value after carrying is `10000000` or ``-128``"] +answer = 2 +explanation = "Addition wraps around from the largest to the smallest and goes from there." +buttonq(choices, answer) +``` + +The largest possible number for a type is returned by `typemax`. For `Int64` (just `Int` on most systems) what is the largest number? + +```{julia} +#| echo: false +numericq(typemax(Int64)) +``` + + +###### Question + +The `Float64` type uses $11$ bits for an exponent (base $2$) between $-1023$ and $1024$. We can see how these are stored as follows: + +```{julia} +bitstring(2.0^(-1023))[2:12], bitstring(2.0^(1024))[2:12] +``` + +This is the full range of values. However the values are *shifted* with $0$ representing $-1023$ and $x$ representing $1024$. The value $x$ is can be found from: + +```{julia} +#| eval: false +2^0 + 2^1 + 2^2 + 2^3 + 2^4 + 2^5 + 2^6 + 2^7 + 2^8 + 2^9 + 2^10 +``` + +What is the value of $x$? + +```{julia} +#| echo: false +val = sum(2^i for i in 0:10) +numericq(val) +``` + +The value `1023` is called a bias. The exponent is coded as the binary value as a positve integer minus $1023$. A bias is used, and not the two's complement format, as storage with a bias makes multiplying by powers of $2$ as easy as shifting the bits. + +To find the storage for, say, $2^4 + 2^2 + 2^0$ or `00000010101` we would add `1023` or `01111111111` and see: + +``` + 00000010101 ++ 01111111111 + ----------- + 10000010100 +``` + +Which we can see: + +```{julia} +bitstring(2.0^(2^4 + 2^2 + 2^0))[2:12] +``` diff --git a/quarto/basics/numbers_types.qmd b/quarto/basics/numbers_types.qmd index 67223d9..da47b11 100644 --- a/quarto/basics/numbers_types.qmd +++ b/quarto/basics/numbers_types.qmd @@ -11,13 +11,16 @@ using CalculusWithJulia nothing ``` -In mathematics, there are many different number systems in common use. For example by the end of pre-calculus, all of the following have been introduced: +In mathematics there are many different number systems in common use. For example by the end of pre-calculus, all of the following have been introduced: - * The integers, $\{\dots, -3, -2, -1, 0, 1, 2, 3, \dots\}$; - * The rational numbers, $\{p/q: p, q \text{ are integers}, q \neq 0\}$; - * The real numbers, $\{x: -\infty < x < \infty\}$; - * The complex numbers, $\{a + bi: a,b \text{ are real numbers and } i^2=-1\}$. +* The integers, $\{\dots, -3, -2, -1, 0, 1, 2, 3, \dots\}$; + +* The rational numbers, $\{p/q: p, q \text{ are integers}, q \neq 0\}$; + +* The real numbers, $\{x: -\infty < x < \infty\}$; + +* The complex numbers, $\{a + bi: a,b \text{ are real numbers and } i^2=-1\}$. On top of these, we have special subsets, such as the natural numbers $\{1, 2, \dots\}$ (sometimes including $0$), the even numbers, the odd numbers, the positive numbers, the non-negative numbers, etc. @@ -26,10 +29,9 @@ On top of these, we have special subsets, such as the natural numbers $\{1, 2, \ Mathematically, these number systems are naturally nested within each other as integers are rational numbers which are real numbers, which can be viewed as part of the complex numbers. -Calculators typically have just one type of number - floating point values. These model the real numbers. `Julia`, on the other hand, has a rich type system, and within that has several different number types. There are types that model each of the four main systems above, and within each type, specializations for how these values are stored. +Calculators typically have just one type of number---floating point values. These model the real numbers. - -Most of the details will not be of interest to all, and will be described later. +`Julia`, on the other hand, has a rich type system, and within that has several different number types. There are types that model each of the four main systems above, and within each type, specializations for how these values are stored. For now, let's consider the number $1$. It can be viewed as either an integer, rational, real, or complex number. To construct "$1$" in each type within `Julia` we have these different styles: @@ -39,16 +41,19 @@ For now, let's consider the number $1$. It can be viewed as either an integer, r 1, 1.0, 1//1, 1 + 0im ``` -The basic number types in `Julia` are `Int`, `Float64`, `Rational` and `Complex`, though in fact there are many more, and the last two aren't even *concrete* types. This distinction is important, as the type of number dictates how it will be stored and how precisely the stored value can be expected to be to the mathematical value it models. +The basic number types in `Julia` are `Int`, `Float64`, `Rational` and `Complex`, though in fact there are many more, and the last two aren't even *concrete* types. This distinction is important, as the type of number dictates how it will be displayed, how it will be stored, and how precisely the stored value can be expected to be to the mathematical value it models. Though there are explicit constructors for these types, these notes avoid them unless necessary, as `Julia`'s parser can distinguish these types through an easy to understand syntax: - * integers have no decimal point; - * floating point numbers have a decimal point (or are in scientific notation); - * rationals are constructed from integers using the double division operator, `//`; and - * complex numbers are formed by including a term with the imaginary unit, `im`. +* integers have no decimal point; + +* floating point numbers have a decimal point (or are written with scientific notation); + +* rationals are constructed from integers using the double division operator, `//`; and + +* complex numbers are formed by including a term with the imaginary unit, `im`. :::{.callout-note} @@ -57,12 +62,10 @@ Heads up, the difference between `1` and `1.0` is subtle. Even more so, as `1.` ::: -Similarly, each type is printed slightly differently. +The key distinction is between integers and floating points. While floating point values include integers, and so can be used exclusively on the calculator, the difference is that an integer is guaranteed to be an exact value, whereas a floating point value, while often an exact representation of a number is also often just an *approximate* value. This can be an advantage---floating point values can model a much wider range of numbers. -The key distinction is between integers and floating points. While floating point values include integers, and so can be used exclusively on the calculator, the difference is that an integer is guaranteed to be an exact value, whereas a floating point value, while often an exact representation of a number is also often just an *approximate* value. This can be an advantage – floating point values can model a much wider range of numbers. - -In nearly all cases the differences are not noticeable. Take for instance this simple calculation involving mixed types. +In nearly all cases the differences are not noticeable. To see why take, for instance, this simple calculation involving mixed types. ```{julia} @@ -71,241 +74,151 @@ In nearly all cases the differences are not noticeable. Take for instance this s The sum of an integer, a floating point number and rational number returns a floating point number without a complaint. +### Promotion -This is because behind the scenes, `Julia` will often "promote" a type to match, so for example to compute `1 + 1.25` the integer `1` will be promoted to a floating point value and the two values are then added. Similarly, with `2.25 + 3//2`, where the fraction is promoted to the floating point value `1.5` and addition is carried out. - - -As floating point numbers may be approximations, some values are not quite what they would be mathematically: +This is because behind the scenes, `Julia` will often "promote" the two numbers to a common type. In particular, before adding mixed-type numbers, the two are promoted to a common type by `promote`. In the example, first when computing `1 + 1.25` the integer `1` will be promoted to a floating point value, `1.0`, and then the two values are added. Similarly, with `2.25 + 3//2`, where the fraction is promoted to the floating point value `1.5` and afterwards addition is carried out. +We can see the promotion here: ```{julia} -sqrt(2) * sqrt(2) - 2, sin(1pi), 1/10 + 1/5 - 3/10 +promote(1, 1.25) +``` + +and + +```{julia} +promote(2.25, 3//2) +``` + + + +## Integers + +Integers are often used casually, as they come about from parsing. As with a calculator, floating point numbers *could* be used for integers, but in `Julia`---and other languages---it proves useful to have numbers known to have *exact* values. Integers are needed for indexing and counting. + +Except on older machines, the default integer is stored with 64 bits, though there are many available types for integers. With $64$ bits, the range of integers that can be represented is $-9223372036854775808=-(2^{63})$ to $9223372036854775807 = 2^{63}-1$. + +## Floating point numbers + +Floating point numbers are a model for the real numbers. With the same size storage, the integers provide exact numbers evenly spaced between the smallest and largest integer. Floating point values are exact for some values but as there are infinitely many real numbers are only approximations except in special cases. This leads to some differences between math done by hand and math done on the computer. + +### Float64 + +`Float64` is the most common type of floating point number, as it is the most supported by the underlying hardware. `Julia` has other floating point types, notably `Float32` and `BigFloat` for certain uses, but our focus here is on 64-bit floating point numbers. + +The double-precision model for floating point numbers has three parts: a sign, an exponent (for a base of $2$), and a significand (in base $2$) representing numbers as $\pm a \cdot 2^n$. The 64 bits are apportioned as follows: $1$ is for the sign, $11$ for the exponent, $52$ for the significand. + + +The $52$ bits of the significand are used to represent $1.a_1a_2a_3\cdots a_{52}$ in base $2$ or $1 + a_12^{-1} + a_22^{-2} + a_32^{-3} + \cdots a_{52}2^{-52} = b/2^{52}$ for some integer $b$. This means the significand represents a rational number. + +The following shows the bits in the significand for a given number written in the form above:^[The output of `bitstring` is 64 characters. The first is the sign bit, the second through twelfth the exponent, the rest the significand. The notation `[13:end]` is used to return just those for the significand. A value of `[2:12]` would return the bits for the exponent.] + +```{julia} +bitstring(1 + 1/2 + 1/4 + 0/8 + 1/16 + 1/32 + 0/64)[13:end] # 1101100… +``` + + +The 11 bits for the exponent covers a range from $-1023$ to $1024$ which in base $10$ is around $10^{-308}$ to $10^{308}$. + +Together these can represent exactly any *rational* number of the form $\pm a \cdot 2^b$ where $a$ is a sum of powers of $1/2$ and $b$ is an integer with $1.0 \leq a \leq 1 + (1/2^1) + (1/2^2) + \cdots + (1/2^{52})$ and $-1023 \leq b \leq 1024$. + +@fig-floating-point-concentration shows the possible positive values *were* there only $2$ bits for the exponent (for $-1, 0, 1, 2$) and $2$ bits for the significand ($1 + 0/4 + 0/2$, $1 + 1/4 + 0/2$, $1 + 0/4 + 1/2$, $1 + 1/4 + 1/2$). The main takeaway is that numbers get less concentrated the farther they get from $0$. + +::: {#fig-floating-point-concentration layout-ncol=1} + +```{julia} +#| echo: false +#| fig: floating_point_concentration +#| fig-label: floating point concentration +using Plots +gr() +using LaTeXStrings +xvals = [0, 1/2, 1, 2, 4, 8] +xlabels = [L"0", L"\frac{1}{2}", L"1", L"2", L"4", L"8"] +p = plot(; size=(600, 200), xlim = (0, 8.5), ylim=(-1, 1), + xaxis=([], false), + yaxis=([], false), + framestyle=:origin, + legend=false, + ) + +plot!(p, [(0,0), (8.5,0)]; arrow=true, line=(2, :black)) +xs = range(0, 1 - 1/2, 5)[1:end-1] +for (i,xi) in enumerate(xvals[2:end-1]) + for x in xs + x₀ = xi + 2^(i-0)*x/2 + Δ = 1/20 + plot!(p, [(x₀, -Δ), (x₀, 2Δ)]; linecolor=:black) + end +end + +plot!(p; annotations=tuple.(xvals, zero.(xvals), text.(xlabels, :top))) +p +``` + +Toy illustration showing uneven concentration of floating point values. The vertical ticks represent representable floating point values (*were* there only 2 bits (not 52) for the mantissa) and 2 bits (not 11) for the exponent. This leaves a range from a range of $-1/2$ to not quite $8$ being representable *without* using subnormal numbers. When the intervals double in length (from $[2^{i},2^{i+1}]$ to $[2^{i+1}, 2^{i+2}]$) there are the same number of representable floating point values, so the concentration of representable values halves. With more bits there is a higher concentration, but the discrete nature is always present and leads to necessary approximations for modeling most all real numbers. +::: + +In addition, there are special bit patterns recognized as `0.0` and even `-0.0`, which is a distinct number. There are also patterns for $+\infty$ (`Inf`) and $-\infty$ (`-Inf`). There are also patterns for `NaN`, or "not a number", a value that is the result of some mathematical operations, such as `0.0 / 0.0`. Finally, there are *subnormal* numbers representing even smaller numbers near $0$ than described above, which are as small as $2^{-1023} \approx 1.11 \cdot 10^{-308}$. + + +### Scientific notation + +Floating point numbers smaller than $10^{-4}$ or bigger or equal to $10^6$ (in absolute value) are displayed in scientific notation. Internally, most floating point numbers are stored in base $2$ scientific notation as $a \cdot 2^b$ with $a=1.xxx\dots$. But when displayed, numbers are represented in base $10$ and when scientific notation is used the numbers are normalized in the form $a \cdot 10^b$ where $1.0 \leq a < 10$. + +The significand and exponent are separated by the character `e`---which is not the same as the constant $e$---rather denotes a 64-bit number separated into a significand and an exponent by a formatting character. (`Float32` uses an `f` as a separator.) + +Consider these two numbers one close to $0$ one far from $0$: + +```{julia} +0.0000000123456789, 123456789.0 +``` + +Their display is subtly different, as only a minus sign after `e` distinguishes them. + +The parser will read in numbers with an `e` in the proper format as though they are scientific notation: + +```{julia} +1e8 +``` + +The above creates the same value as `10.0^8`, but not `1e^8` which will error unless a value for `e` has been assigned. + +### Inexactness and consequences + +For numbers not representable in floating point, some rounding must go on to fit the number into a representable floating point value. As such, some computed values are not quite what they would be mathematically: + +```{julia} +sqrt(2) * sqrt(2) - 2, sin(1pi) ``` These values are *very* small numbers, but not exactly $0$, as they are mathematically. +More surprisingly, simple fractions may also lead to mathematically different results: ---- +```{julia} +1/10 + 2/10 - 3/10 +``` +This, of course, is due to none of these fractions being of the form $a\cdot 2^b$ for integers $a, b$. -The only common issue is with powers. We saw this previously when discussing a distinction between `2^64` and `2.0^64`. `Julia` tries to keep a predictable output from the input types (not their values). Here are the two main cases that arise where this can cause unexpected results: - - -* integer bases and integer exponents can *easily* overflow. Not only `m^n` is always an integer, it is always an integer with a fixed storage size computed from the sizes of `m` and `n`. So the powers can quickly get too big. This can be especially noticeable on older $32$-bit machines, where too big is $2^{32} = 4,294,967,296$. On $64$-bit machines, this limit is present but much bigger. - - -Rather than give an error though, `Julia` gives seemingly arbitrary answers, as can be seen in this example on a $64$-bit machine: +Another surprise: floating point addition is not necessarily associative. That is the property $a + (b+c) = (a+b) + c$ may not hold exactly. For example: ```{julia} -2^62, 2^63 +l2r = (1/10 + 2/10) + 3/10 +r2l = 1/10 + (2/10 + 3/10) +l2r - r2l ``` -(They aren't arbitrary, as explained previously.) - - -This could be worked around, as it is with some programming languages, but it isn't, as it would slow down this basic computation. So, it is up to the user to be aware of cases where their integer values can grow to big. The suggestion is to use floating point numbers in this domain, as they have more room, at the cost of sometimes being approximate values for fairly large values. - - -* the `sqrt` function will give a domain error for negative values: - - -```{julia} -#| error: true -sqrt(-1.0) -``` - -This is because for real-valued inputs `Julia` expects to return a real-valued output. Of course, this is true in mathematics until the complex numbers are introduced. Similarly in `Julia` - to take square roots of negative numbers, start with complex numbers: - - -```{julia} -sqrt(-1.0 + 0im) -``` - - * At one point, `Julia` had an issue with a third type of power: - - -integer bases and negative integer exponents. For example `2^(-1)`. This is now special cased, though only for numeric literals. If `z=-1`, `2^z` will throw a `DomainError`. Historically, the desire to keep a predictable type for the output (integer) led to defining this case as a domain error, but its usefulness led to special casing. - - -## Additional details. - - -What follows is only needed for those seeking more background. - - -Julia has *abstract* number types `Integer`, `Real`, and `Number`. All four types described above are of type `Number`, but `Complex` is not of type `Real`. - - -However, a specific value is an instance of a *concrete* type. A concrete type will also include information about how the value is stored. For example, the *integer* `1` could be stored using $64$ bits as a signed integers, or, should storage be a concern, as an $8$ bits signed or even unsigned integer, etc.. If storage isn't an issue, but exactness at all scales is, then it can be stored in a manner that allows for the storage to grow using "big" numbers. - - -These distinctions can be seen in how `Julia` parses these three values: - - - * `1234567890` will be a $64$-bit integer (on newer machines), `Int64` - * `12345678901234567890` will be a $128$ bit integer, `Int128` - * `1234567890123456789012345678901234567890` will be a big integer, `BigInt` - - -Having abstract types allows programmers to write functions that will work over a wide range of input values that are similar, but have different implementation details. - - -### Integers - - -Integers are often used casually, as they come about from parsing. As with a calculator, floating point numbers *could* be used for integers, but in `Julia` - and other languages - it proves useful to have numbers known to have *exact* values. In `Julia` there are built-in number types for integers stored in $8$, $16$, $32$, $64$, and $128$ bits and `BigInt`s if the previous aren't large enough. ($8$ bits can hold $8$ binary values representing $1$ of $256=2^8$ possibilities, whereas the larger $128$ bit can hold one of $2^{128}$ possibilities.) Smaller values can be more efficiently used, and this is leveraged at the system level, but not a necessary distinction with calculus where the default size along with an occasional usage of `BigInt` suffice. - - -### Floating point numbers - - -[Floating point](http://en.wikipedia.org/wiki/Floating_point) numbers are a computational model for the real numbers. For floating point numbers, $64$ bits are used by default for both $32$- and $64$-bit systems, though other storage sizes can be requested. This gives a large range - but still finite - set of real numbers that can be represented. However, there are infinitely many real numbers just between $0$ and $1$, so there is no chance that all can be represented exactly on the computer with a floating point value. Floating point then is *necessarily* an approximation for all but a subset of the real numbers. Floating point values can be viewed in normalized [scientific notation](http://en.wikipedia.org/wiki/Scientific_notation) as $a\cdot 2^b$ where $a$ is the *significand* and $b$ is the *exponent*. Save for special values, the significand $a$ is normalized to satisfy $1 \leq \lvert a\rvert < 2$, the exponent can be taken to be an integer, possibly negative. - - -As per IEEE Standard 754, the `Float64` type gives 52 bits to the precision (with an additional implied one), 11 bits to the exponent and the other bit is used to represent the sign. Positive, finite, floating point numbers have a range approximately between $10^{-308}$ and $10^{308}$, as 308 is about $\log_{10} 2^{1023}$. The numbers are not evenly spread out over this range, but, rather, are much more concentrated closer to $0$. - -The use of 32-bit floating point values is common, as some widely used computer chips expect this. These values have a narrower range of possible values. - -:::{.callout-warning} -## More on floating point numbers -You can discover more about the range of floating point values provided by calling a few different functions. - - * `typemax(0.0)` gives the largest value for the type (`Inf` in this case). - * `prevfloat(Inf)` gives the largest finite one, in general `prevfloat` is the next smallest floating point value. - - * `nextfloat(-Inf)`, similarly, gives the smallest finite floating point value, and in general returns the next largest floating point value. - * `nextfloat(0.0)` gives the closest positive value to 0. - * `eps()` gives the distance to the next floating point number bigger than `1.0`. This is sometimes referred to as machine precision. - -::: - - -#### Scientific notation - - -Floating point numbers may print in a familiar manner: - - -```{julia} -x = 1.23 -``` - -or may be represented in scientific notation: - - -```{julia} -6.022 * 10.0^23 -``` - -The special coding `aeb` (or if the exponent is negative `ae-b`) is used to represent the number $a \cdot 10^b$ ($1 \leq a < 10$). This notation can be used directly to specify a floating point value: - - -```{julia} -avogadro = 6.022e23 -``` - -::: {.callout-note} -## Not `e` -Here `e` is decidedly *not* the Euler number, rather **syntax** to separate the exponent from the mantissa. -::: - -The first way of representing this number required using `10.0` and not `10` as the integer power will return an integer and even for 64-bit systems is only valid up to `10^18`. Using scientific notation avoids having to concentrate on such limitations. - - -##### Example - - -Floating point values in scientific notation will always be normalized. This is easy for the computer to do, but tedious to do by hand. Here we see: - - -```{julia} -4e30 * 3e40 -``` - -```{julia} -3e40 / 4e30 -``` - -The power in the first is $71$, not $70 = 30+40$, as the product of $3$ and $4$ as $12$ or `1.2e^1`. (We also see the artifact of `1.2` not being exactly representable in floating point.) - - -##### Example: 32-bit floating point - - -In some uses, such as using a GPU, $32$-bit floating point (single precision) is also common. These values may be specified with an `f` in place of the `e` in scientific notation: - - -```{julia} -1.23f0 -``` - -As with the use of `e`, some exponent is needed after the `f`, even if it is `0`. - - -#### Special values: Inf, -Inf, NaN - - -The coding of floating point numbers also allows for the special values of `Inf`, `-Inf` to represent positive and negative infinity. As well, a special value `NaN` ("not a number") is used to represent a value that arises when an operation is not closed (e.g., $0.0/0.0$ yields `NaN`). (Technically `NaN` has several possible "values," a point ignored here.) Except for negative bases, the floating point numbers with the addition of `Inf` and `NaN` are closed under the operations `+`, `-`, `*`, `/`, and `^`. Here are some computations that produce `NaN`: - - -```{julia} -0/0, Inf/Inf, Inf - Inf, 0 * Inf -``` - -Whereas, these produce an infinity - - -```{julia} -1/0, Inf + Inf, 1 * Inf -``` - -Finally, these are mathematically undefined, but still yield a finite value with `Julia`: - - -```{julia} -0^0, Inf^0 -``` - -#### Floating point numbers and real numbers - - -Floating point numbers are an abstraction for the real numbers. For the most part this abstraction works in the background, though there are cases where one needs to have it in mind. Here are a few: - - - * For real and rational numbers, between any two numbers $a < b$, there is another real number in between. This is not so for floating point numbers which have a finite precision. (Julia has some functions for working with this distinction.) - * Floating point numbers are approximations for most values, even simple rational ones like $1/3$. This leads to oddities such as this value not being $0$: - - -```{julia} -sqrt(2)*sqrt(2) - 2 -``` - -It is no surprise that an irrational number, like $\sqrt{2}$, can't be represented **exactly** within floating point, but it is perhaps surprising that simple numbers can not be, so $1/3$, $1/5$, $\dots$ are approximated. Here is a surprising-at-first consequence: - - -```{julia} -1/10 + 2/10 == 3/10 -``` - -That is adding `1/10` and `2/10` is not exactly `3/10`, as expected mathematically. Such differences are usually very small and are generally attributed to rounding error. The user needs to be mindful when testing for equality, as is done above with the `==` operator. - - - * Floating point addition is not necessarily associative, that is the property $a + (b+c) = (a+b) + c$ may not hold exactly. For example: - - -```{julia} -1/10 + (2/10 + 3/10) == (1/10 + 2/10) + 3/10 -``` - - * Mathematically, for real numbers, subtraction of similar-sized numbers is not exceptional, for example $1 - \cos(x)$ is positive if $0 < x < \pi/2$, say. This will not be the case for floating point values. If $x$ is close enough to $0$, then $\cos(x)$ and $1$ will be so close, that they will be represented by the same floating point value, `1.0`, so the difference will be zero: +One other surprise. Mathematically, for real numbers, subtraction of similar-sized numbers is not exceptional, for example $1 - \cos(x)$ is positive if $0 < x < \pi/2$, say. This will not be the case for floating point values. If $x$ is close enough to $0$, then $\cos(x)$ and $1$ will be so close, that they will be represented by the same floating point value, `1.0`, so the difference will be zero: ```{julia} 1.0 - cos(1e-8) ``` -### Rational numbers +## Rational numbers Rational numbers can be used when the exactness of the number is more important than the speed or wider range of values offered by floating point numbers. In `Julia` a rational number is comprised of a numerator and a denominator, each an integer of the same type, and reduced to lowest terms. The operations of addition, subtraction, multiplication, and division will keep their answers as rational numbers. As well, raising a rational number to an integer value will produce a rational number. @@ -346,25 +259,27 @@ For powers, a non-integer exponent is converted to floating point, so this opera (1//2)^(1//2) # the first parentheses are necessary as `^` will be evaluated before `//`. ``` -##### Example: different types of real numbers +---- + +@tbl-real-number-types compares different number types for storing a real number. The "closed under" column indicates which operations will return the same type as the inputs. -This table shows what attributes are implemented for the different types. +::: {#tbl-real-number-types} + +| Attributes | Integer | Rational | FloatingPoint | +|:------------|:---------|:---------|:--------------| +| construction| 1 | 1//1 |1.0 | +| exact | true | true |not always | +| wide range | false | false |true | +| has infinity| false | false |true | +| has -0 | false | false |true | +| fast | true | false |true | +| closed under| `+`, `-`, `*`, `^` (non-negative exponent)| `+`, `-`, `*`, `/` (non zero denominator),`^` (integer power) | `+`, `-`, `*`, `/` (possibly `NaN`, `Inf`),`^` (non-negative base) | + +::: -```{julia} -#| echo: false -using DataFrames -attributes = ["construction", "exact", "wide range", "has infinity", "has `-0`", "fast", "closed under"] -integer = [q"1", "true", "false", "false", "false", "true", "`+`, `-`, `*`, `^` (non-negative exponent)"] -rational = ["`1//1`", "true", "false", "false", "false", "false", "`+`, `-`, `*`, `/` (non zero denominator),`^` (integer power)"] -float = [q"1.0", "not usually", "true", "true", "true", "true", "`+`, `-`, `*`, `/` (possibly `NaN`, `Inf`),`^` (non-negative base)"] -d = DataFrame(Attributes=attributes, Integer=integer, Rational=rational, FloatingPoint=float) -table(d) -``` - -### Complex numbers - +## Complex numbers Complex numbers in `Julia` are stored as two numbers, a real and imaginary part, each some type of `Real` number. The special constant `im` is used to represent $i=\sqrt{-1}$. This makes the construction of complex numbers fairly standard: @@ -376,7 +291,7 @@ Complex numbers in `Julia` are stored as two numbers, a real and imaginary part, (These two aren't exactly the same, the `3` is promoted from an integer to a float to match the `4.0`. Each of the components must be of the same type of number.) -Mathematically, complex numbers are needed so that certain equations can be satisfied. For example $x^2 = -2$ has solutions $-\sqrt{2}i$ and $\sqrt{2}i$ over the complex numbers. Finding this in `Julia` requires some attention, as we have both `sqrt(-2)` and `sqrt(-2.0)` throwing a `DomainError`, as the `sqrt` function expects non-negative real arguments. However first creating a complex number does work: +Mathematically, complex numbers are needed so that certain equations can be satisfied. For example $x^2 = -2$ has solutions $-\sqrt{2}i$ and $\sqrt{2}i$ over the complex numbers. Finding this in `Julia` requires some attention, as we have both `sqrt(-2)` and `sqrt(-2.0)` throwing a `DomainError`, as the `sqrt` function expects non-negative real arguments. However first creating a complex number and then taking a square root does work: ```{julia} @@ -397,14 +312,20 @@ discr = b^2 - 4a*c When learning calculus, the only common usage of complex numbers arises when solving polynomial equations for roots, or zeros, though they are very important for subsequent work using the concepts of calculus. +## Irrational numbers -:::{.callout-note} -## Note -Though complex numbers are stored as pairs of numbers, the imaginary unit, `im`, is of type `Complex{Bool}`, a type that can be promoted to more specific types when `im` is used with different number types. +`Julia` has a a few mathematical constants that are stored with a special type `Irrational`. One such value is `pi`. There are others in the `Base.MathConstants` module, and an external package `IrrationalConstants.jl`. -::: +`Irrational` values may have special methods defined for them which can lead to subtle differences, such as: + +```{julia} +sin(pi), sin(2pi) +``` + +In computing the product `2pi` first the two values are promoted to `Float64` and then multiplied, leaving a floating-point approximation of $2\pi$ for `sin` to evaluate. + +## Other types of data: strings and symbols -### Strings and symbols For text, `Julia` has a `String` type. When double quotes are used to specify a string, the parser creates this type: @@ -436,76 +357,12 @@ Strings are comprised of *characters* which can be produced directly using *sing 'c' ``` -We won't use these. +We won't use characters in these notes. Finally, `Julia` has *symbols* which are *interned* strings which are used as identifiers. Symbols are used for advanced programming techniques; we will only see them as shortcuts to specify plotting arguments. -## Type stability -One design priority of `Julia` is that it should be fast. How can `Julia` do this? In a simple model, `Julia` is an interface between the user and the computer's processor(s). Processors consume a set of instructions, the user issues a set of commands. `Julia` is in charge of the translation between the two. Ultimately `Julia` calls a compiler to create the instructions. A basic premise is the shorter the instructions, the faster they are to process. Shorter instructions can come about by being more explicit about what types of values the instructions concern. Explicitness means, there is no need to reason about what a value can be. When `Julia` can reason about the type of value involved without having to reason about the values themselves, it can work with the compiler to produce shorter lists of instructions. - - -So knowing the type of the output of a function based only on the type of the inputs can be a big advantage. In `Julia` this is known as *type stability*. In the standard `Julia` library, this is a primary design consideration. - - -##### Example: closure - - -To motivate this a bit, we discuss how mathematics can be shaped by a desire to stick to simple ideas. A desirable algebraic property of a set of numbers and an operation is *closure*. That is, if one takes an operation like `+` and then uses it to add two numbers in a set, will that result also be in the set? If this is so for any pair of numbers, then the set is closed with respect to the operation addition. - - -Lets suppose we start with the *natural numbers*: $1,2, \dots$. Natural, in that we can easily represent small values in terms of fingers. This set is closed under addition - as a child learns when counting using their fingers. However, if we started with the odd natural numbers, this set would *not* be closed under addition - $3+3=6$. - - -The natural numbers are not all the numbers we need, as once a desire for subtraction is included, we find the set isn't closed. There isn't a $0$, needed as $n-n=0$ and there aren't negative numbers. The set of integers are needed for closure under addition and subtraction. - - -The integers are also closed under multiplication, which for integer values can be seen as just regrouping into longer additions. - - -However, the integers are not closed under division - even if you put aside the pesky issue of dividing by $0$. For that, the rational numbers must be introduced. So aside from division by $0$, the rationals are closed under addition, subtraction, multiplication, and division. There is one more fundamental operation though, powers. - - -Powers are defined for positive integers in a simple enough manner - - -$$ -a^n=a \cdot a \cdot a \cdots a \text{ (n times); } a, n \text{ are integers } n \text{ is positive}. -$$ - -We can define $a^0$ to be $1$, except for the special case of $0^0$, which is left undefined mathematically (though it is also defined as `1` within `Julia`). We can extend the above to include negative values of $a$, but what about negative values of $n$? We can't say the integers are closed under powers, as the definition consistent with the rules that $a^{(-n)} = 1/a^n$ requires rational numbers to be defined. - - -Well, in the above `a` could be a rational number, is `a^n` closed for rational numbers? No again. Though it is fine for $n$ as an integer (save the odd case of $0$, simple definitions like $2^{1/2}$ are not answered within the rationals. For this, we need to introduce the *real* numbers. It is mentioned that [Aristotle](http://tinyurl.com/bpqbkap) hinted at the irrationality of the square root of $2$. To define terms like $a^{1/n}$ for integer values $a,n > 0$ a reference to a solution to an equation $x^n-a$ is used. Such solutions require the irrational numbers to have solutions in general. Hence the need for the real numbers (well, algebraic numbers at least, though once the exponent is no longer a rational number, the full set of real numbers are needed.) - - -So, save the pesky cases, the real numbers will be closed under addition, subtraction, multiplication, division, and powers - provided the base is non-negative. - - -Finally for that last case, the complex numbers are introduced to give an answer to $\sqrt{-1}$. - - ---- - - -How does this apply with `Julia`? - - -The point is, if we restrict our set of inputs, we can get more precise values for the output of basic operations, but to get more general inputs we need to have bigger output sets. - - -A similar thing happens in `Julia`. For addition say, the addition of two integers of the same type will be an integer of that type. This speed consideration is not solely for type stability, but also to avoid checking for overflow. - - -Another example, the division of two integers will always be a number of the same type - floating point, as that is the only type that ensures the answer will always fit within. (The explicit use of rationals notwithstanding.) So even if two integers are the input and their answer *could* be an integer, in `Julia` it will be a floating point number, (cf. `2/1`). - - -Hopefully this helps explain the subtle issues around powers: in `Julia` an integer raised to an integer should be an integer, for speed, though certain cases are special cased, like `2^(-1)`. However since a real number raised to a real number makes sense always when the base is non-negative, as long as real numbers are used as outputs, the expressions `2.0^(-1)` and `2^(-1.0)` are computed and real numbers (floating points) are returned. For type stability, even though $2.0^1$ could be an integer, a floating point answer is returned. - - -As for negative bases, `Julia` could always return complex numbers, but in addition to this being slower, it would be irksome to users. So user's must opt in. Hence `sqrt(-1.0)` will be an error, but the more explicit - but mathematically equivalent - `sqrt(-1.0 + 0im)` will not be a domain error, but rather a complex value will be returned. - ## Questions @@ -667,7 +524,7 @@ yesnoq(false) ###### Question -In [NewScientist](https://www.newscientist.com/article/2112537-smallest-sliver-of-time-yet-measured-sees-electrons-fleeing-atom/) we learn "For the first time, physicists have measured changes in an atom to the level of zeptoseconds, or trillionths of a billionth of a second – the smallest division of time yet observed." +In [NewScientist](https://www.newscientist.com/article/2112537-smallest-sliver-of-time-yet-measured-sees-electrons-fleeing-atom/) we learn "For the first time, physicists have measured changes in an atom to the level of zeptoseconds, or trillionths of a billionth of a second---the smallest division of time yet observed." That is @@ -683,3 +540,156 @@ as = ["1/10^21", "1e-21"] explanation = "The scientific notation is correct. Due to integer overflow `10^21` is not the same number as `10.0^21`." buttonq(as, 2; explanation) ``` + +###### Question + +Signed integers are stored on a computer in a special manner. We will see with 8 bit integers, formed by `Int8`. Eight bit means only 8 0's or 1's are used to store a given number. This is a useful format for storing many small integers but for this example, useful as we can more easily track the values. + +The first bit is a sign bit. Based on these two outputs, can you guess how that works: + +```{julia} +bitstring(Int8(-1)), bitstring(Int8(1)) +``` + +```{julia} +#| echo: false +choices = ["The first digit is `1` for positive numbers, `0` for negative numbers", + "The first digit is `0` for positive numbers, `1` for negative numbers"] +answer = 2 +buttonq(choices, answer) +``` + +Positive numbers and negative numbers are stored a bit differently. Positive numbers just use binary: $a_0 \cdot 2^0 + a_1 \cdot 2^1 + a_2 \cdot 2^2 + \cdots a_7 \cdot 2^7$. The number $27$ is $1 + 2 + 8 + 16$. so have $a_0 = a_1 = a_3 = a_4 = 1$, the others are $0$. The bitstring shows: + +```{julia} +bitstring(Int8(27)) +``` + +Which bit pattern is used? + +```{julia} +#| echo: false +choices = ["`sign bit | a₀ | a₁ | a₂ | a₃ | a₄ | a₅ | a₆ | a₇`", + "`sign bit | a₇ | a₆ | a₅ | a₄ | a₃ | a₂ | a₁ | a₀`"] +answer = 2 +buttonq(choices, answer) +``` + +Negative numbers are stored using *two's complement* format: + +* represent the positive number +* flip `0` to `1`; `1` to `0` +* add `1` to the value (long addition with carrying) + +For $-27$ we have + +* first $27$: `00011011` +* then flip each: `11100100` +* finally add $1$: `11100101` + +```{julia} +bitstring(Int8(-27)) +``` + +The largest positive number is $127$ for 8-bits and is represented by `01111111`. What is the bit pattern of $-127$? + +```{julia} +#| echo: false +choices = ["`01111111 => 10000000 => 10000001`", + "`01111111 => 10000000 => 100000000`"] +answer = 1 +explanation = "Add the `1` on the right side, not the left side" +buttonq(choices, answer; explanation) +``` + +The smallest negative number is $-128$. Why? + +```{julia} +#| echo: false +choices = ["Subtracting 1 from `10000001` leaves `10000000` which is the smallest negative number", + "Tricky, the smallest negative number is ``-127``" + ] +answer = 1 +buttonq(choices, answer) +``` + +Why all this fuss? Couldn't there be an easier way? + +This storage has a big advantage when adding numbers. Let's look at adding $-5$ to $6$. we have: + +``` +-5 => 11111011 + 6 => 00000110 + -------- + 100000001 => 00000001 +``` + +The addition is done by carrying a `1` across and then *dropping* the 9th number when there is such a carry. This leaves the representation for what number? + +```{julia} +#| echo: false +choices = ["`1`", "`-1`", "`0`"] +answer = 1 +buttonq(choices, answer) +``` + + +The largest *positive* number that can be represented is $2^0 + 2^1 + 2^2 + \cdots + 2^7$, where $7$ is the number of bits minus $1$. The representation is `01111111`. What happens if we add `1` to this number? + + +```{julia} +#| echo: false +choices = ["The value after carrying is `100000000`. After dropping the 9th number, we get `00000000` or zero", + "The value after carrying is `10000000` or ``-128``"] +answer = 2 +explanation = "Addition wraps around from the largest to the smallest and goes from there." +buttonq(choices, answer) +``` + +The largest possible number for a type is returned by `typemax`. For `Int64` (just `Int` on most systems) what is the largest number? + +```{julia} +#| echo: false +numericq(typemax(Int64)) +``` + + +###### Question + +The `Float64` type uses $11$ bits for an exponent (base $2$) between $-1023$ and $1024$. We can see how these are stored as follows: + +```{julia} +bitstring(2.0^(-1023))[2:12], bitstring(2.0^(1024))[2:12] +``` + +This is the full range of values. However the values are *shifted* with $0$ representing $-1023$ and $x$ representing $1024$. The value $x$ is can be found from: + +```{julia} +#| eval: false +2^0 + 2^1 + 2^2 + 2^3 + 2^4 + 2^5 + 2^6 + 2^7 + 2^8 + 2^9 + 2^10 +``` + +What is the value of $x$? + +```{julia} +#| echo: false +val = sum(2^i for i in 0:10) +numericq(val) +``` + +The value `1023` is called a bias. The exponent is coded as the binary value as a positve integer minus $1023$. A bias is used, and not the two's complement format, as storage with a bias makes multiplying by powers of $2$ as easy as shifting the bits. + +To find the storage for, say, $2^4 + 2^2 + 2^0$ or `00000010101` we would add `1023` or `01111111111` and see: + +``` + 00000010101 ++ 01111111111 + ----------- + 10000010100 +``` + +Which we can see: + +```{julia} +bitstring(2.0^(2^4 + 2^2 + 2^0))[2:12] +``` diff --git a/quarto/basics/ranges.qmd b/quarto/basics/ranges.qmd index a50cf77..65f85d4 100644 --- a/quarto/basics/ranges.qmd +++ b/quarto/basics/ranges.qmd @@ -63,68 +63,53 @@ Rather than express sequences by the $a_0$, $h$, and $n$, `Julia` uses the start ```{julia} -1:10 +1:5 ``` -But wait, nothing different printed? This is because `1:10` is efficiently stored. Basically, a recipe to generate the next number from the previous number is created and `1:10` just stores the start and end points (the step size is implicit in how this is stored) and that recipe is used to generate the set of all values. To expand the values, you have to ask for them to be `collect`ed (though this typically isn't needed in practice, as values are usually *iterated* over): +But wait, nothing different printed? This is because `1:5` is efficiently stored. Basically, a recipe to generate the next number from the previous number is created and `1:5` just stores the start and end points (the step size is implicit in how this is stored) and that recipe is used to generate the set of all values. To expand the values, you have to ask for them to be `collect`ed (though this typically isn't needed in practice, as values are usually *iterated* over): ```{julia} -collect(1:10) +collect(1:5) ``` -When a non-default step size is needed, it goes in the middle, as in `a:h:b`. For example, counting by sevens from $1$ to $50$ is achieved by: +When a non-default step size is needed, it goes in the middle, as in `a:h:b`. For example, counting by sevens from $1$ to 29 is achieved by: ```{julia} -collect(1:7:50) +collect(1:7:29) ``` Or counting down from 100: ```{julia} -collect(100:-7:1) +collect(100:-7:70) ``` -In this last example, we said end with $1$, but it ended with $2$. The ending value in the range is a suggestion to go up to, but not exceed. Negative values for `h` are used to make decreasing sequences. +In this last example, we said end with $70$, but it ended with $72$. The ending value in the range is a suggestion to go up to, but not exceed. Negative values for `h` are used to make decreasing sequences. ### The range function -For generating points to make graphs, a natural set of points to specify is $n$ evenly spaced points between $a$ and $b$. We can mimic creating this set with the range operation by solving for the correct step size. We have $a_0=a$ and $a_0 + (n-1) \cdot h = b$. (Why $n-1$ and not $n$?) Solving yields $h = (b-a)/(n-1)$. To be concrete we might ask for $9$ points between $-1$ and $1$: +For generating points to make graphs, a natural set of points to specify is $n$ *evenly* spaced points between $a$ and $b$. We can mimic creating this set with the range operation by solving for the correct step size. We have $a_0=a$ and $a_0 + (n-1) \cdot h = b$. (Why $n-1$ and not $n$?) Solving yields $h = (b-a)/(n-1)$. To be concrete we might ask for $5$ points between $-1$ and $1$: ```{julia} #| hold: true -a, b, n = -1, 1, 9 +a, b, n = -1, 1, 5 h = (b-a)/(n-1) collect(a:h:b) ``` -Pretty neat. If we were doing this many times - such as once per plot - we'd want to encapsulate this into a function, for example using a comprehension: +Now, our recipe is straightforward, but only because it avoids somethings. Look at something simple: ```{julia} -function evenly_spaced(a, b, n) - h = (b-a)/(n-1) - [a + i*h for i in 0:n-1] -end -``` - -Great, let's try it out: - - -```{julia} -evenly_spaced(0, 2pi, 5) -``` - -Now, our implementation was straightforward, but only because it avoids somethings. Look at something simple: - - -```{julia} -evenly_spaced(1/5, 3/5, 3) +a, b, n = 1/5, 3/5, 3 +h = (b-a)/(n-1) +collect(a:h:b) ``` It seems to work as expected. But looking just at the algorithm it isn't quite so clear: @@ -137,7 +122,7 @@ It seems to work as expected. But looking just at the algorithm it isn't quite s Floating point roundoff leads to the last value *exceeding* `0.6`, so should it be included? Well, here it is pretty clear it *should* be, but better to have something programmed that hits both `a` and `b` and adjusts `h` accordingly. Something which isn't subject to the vagaries of `(3/5 - 1/5)/2` not being `0.2`. -Enter the base function `range` which solves this seemingly simple - but not really - task. It can use `a`, `b`, and `n`. Like the range operation, this function returns a generator which can be collected to realize the values. +Enter the base function `range` which solves this seemingly simple---but not really---task. It can use `a`, `b`, and `n`. Like the range operation, this function returns a generator which can be collected to realize the values. The number of points is specified as a third argument (though keyword arguments can be given): @@ -153,11 +138,25 @@ There is also the `LinRange(a, b, n)` function which can be more performant than ::: + ## Modifying sequences Now we concentrate on some more general styles to modify a sequence to produce a new sequence. +### The repeat function + +The `repeat` function is used to repeat entries of an array a certain number of times. For a vector, there are two different patterns of interest: repeating the first element $k$ times, then the second, then the third etc. Or repeating the entire vector a number of times. The first one repeats *inner* values, the second *outer*. We illustrate with two examples: + +```{julia} +repeat(1:3, inner=2) # repeat 1 twice, then 2 twice, then 3 twice +``` + +and + +```{julia} +repeat(1:3, outer=2) # repeat 1:3 twice +``` ### Filtering @@ -167,28 +166,28 @@ The act of throwing out elements of a collection based on some condition is call For example, another way to get the values between $0$ and $100$ that are multiples of $7$ is to start with all $101$ values and throw out those that don't match. To check if a number is divisible by $7$, we could use the `rem` function. It gives the remainder upon division. Multiples of `7` match `rem(m, 7) == 0`. Checking for divisibility by seven is unusual enough there is nothing built in for that, but checking for division by $2$ is common, and for that, there is a built-in function `iseven`. -The `filter` function does this in `Julia`; the basic syntax being `filter(predicate_function, collection)`. The "`predicate_function`" is one that returns either `true` or `false`, such as `iseven`. The output of `filter` consists of the new collection of values - those where the predicate returns `true`. +The `filter` function does this in `Julia`; the basic syntax being `filter(predicate_function, collection)`. The "`predicate_function`" is one that returns either `true` or `false`, such as `iseven`. The output of `filter` consists of the new collection of values---those in the collection for which the predicate returns `true`. -To see it used, lets start with the numbers between `0` and `25` (inclusive) and filter out those that are even: +To see it used, lets start with the numbers between `0` and `9` (inclusive) and filter out those that are even: ```{julia} -filter(iseven, 0:25) +filter(iseven, 0:9) ``` To get the numbers between $1$ and $100$ that are divisible by $7$ requires us to write a function akin to `iseven`, which isn't hard (e.g., `is_seven(x) = x%7 == 0` or if being fancy `Base.Fix2(iszero∘rem, 7)`), but isn't something we continue with just yet. -For another example, here is an inefficient way to list the prime numbers between $100$ and $200$. This uses the `isprime` function from the `Primes` package +For another example, here is an inefficient way to list the prime numbers between $100$ and $130$. This uses the `isprime` function from the `Primes` package ```{julia} -using Primes +using Primes: isprime ``` ```{julia} -filter(isprime, 100:200) +filter(isprime, 100:130) ``` Illustrating `filter` at this point is mainly a motivation to illustrate that we can start with a regular set of numbers and then modify or filter them. The function takes on more value once we discuss how to write predicate functions. @@ -202,7 +201,7 @@ Let's return to the case of the set of even numbers between $0$ and $100$. We ha * The collection of numbers $0, 2, 4, 6 \dots, 100$, or the arithmetic sequence with step size $2$, which is returned by `0:2:100`. * The numbers between $0$ and $100$ that are even, that is `filter(iseven, 0:100)`. - * The set of numbers $\{2k: k=0, \dots, 50\}$. + * The set of numbers $\{2k: k=0, \dots, 5\}$. While `Julia` has a special type for dealing with sets, we will use a vector for such a set. (Unlike a set, vectors can have repeated values, but, as vectors are more widely used, we demonstrate them.) Vectors are described more fully in a previous section, but as a reminder, vectors are constructed using square brackets: `[]` (a special syntax for [concatenation](http://docs.julialang.org/en/latest/manual/arrays/#concatenation)). Square brackets are used in different contexts within `Julia`, in this case we use them to create a *collection*. If we separate single values in our collection by commas (or semicolons), we will create a vector: @@ -215,48 +214,45 @@ x = [0, 2, 4, 6, 8, 10] That is of course only part of the set of even numbers we want. Creating more might be tedious were we to type them all out, as above. In such cases, it is best to *generate* the values. -For this simple case, a range can be used, but more generally a [comprehension](https://docs.julialang.org/en/v1/manual/arrays/#man-comprehensions) provides this ability using a construct that closely mirrors a set definition, such as $\{2k: k=0, \dots, 50\}$. The simplest use of a comprehension takes this form (as we described in the section on vectors): +For this simple case, a range can be used, but more generally a [comprehension](https://docs.julialang.org/en/v1/manual/arrays/#man-comprehensions) provides this ability using a construct that closely mirrors a set definition, such as $\{2k: k=0, \dots, 5\}$. The simplest use of a comprehension takes this form (as we described in the section on vectors): `[expr for variable in collection]` -The expression typically involves the variable specified after the keyword `for`. The collection can be a range, a vector, or many other items that are *iterable*. Here is how the mathematical set $\{2k: k=0, \dots, 50\}$ may be generated by a comprehension: +The expression typically involves the variable specified after the keyword `for`. The collection can be a range, a vector, or many other items that are *iterable*. (We previously discussed the `map` and broadcasting alternatives to a comprehension, but those are most useful once we have discussed user-defined functions.) + + + +Here is how the mathematical set $\{2k: k=0, \dots, 5\}$ may be generated by a comprehension: ```{julia} -[2k for k in 0:50] +[2k for k in 0:5] ``` -The expression is `2k`, the variable `k`, and the collection is the range of values, `0:50`. The syntax is basically identical to how the math expression is typically read aloud. +The expression is `2k`, the variable `k`, and the collection is the range of values, `0:5`. The syntax is basically identical to how the math expression is typically read aloud. -For some other examples, here is how we can create the first $10$ numbers divisible by $7$: +For some other examples, here is how we can create the first 5 numbers divisible by $7$: ```{julia} -[7k for k in 1:10] +[7k for k in 1:5] ``` -Here is how we can square the numbers between $1$ and $10$: +Here is how we can square the numbers between $1$ and 5: ```{julia} -[x^2 for x in 1:10] +[x^2 for x in 1:5] ``` -To generate other progressions, such as powers of $2$, we could do: +To generate other progressions, such as decreasing powers of $2$, we could do: ```{julia} -[2^i for i in 1:10] -``` - -Here are decreasing powers of $2$: - - -```{julia} -[1/2^i for i in 1:10] +[1/2^i for i in 1:5] ``` Sometimes, the comprehension does not produce the type of output that may be expected. This is related to `Julia`'s more limited abilities to infer types at the command line. If the output type is important, the extra prefix of `T[]` can be used, where `T` is the desired type. @@ -269,14 +265,14 @@ A typical pattern would be to generate a collection of numbers and then apply a ```{julia} -sum([2^i for i in 1:10]) +sum([2^i for i in 1:5]) ``` Conceptually this is easy to understand: one step generates the numbers, the other adds them up. Computationally it is a bit inefficient. The generator syntax allows this type of task to be done more efficiently. To use this syntax, we just need to drop the `[]`: ```{julia} -sum(2^i for i in 1:10) +sum(2^i for i in 1:5) ``` The difference being no intermediate object is created to store the collection of all values specified by the generator. Not all functions allow generators as arguments, but most common reductions do. @@ -314,7 +310,7 @@ The same `if` can be used in a comprehension. For example, this is an alternativ ```{julia} -[k for k in 1:100 if rem(k,7) == 0] +[k for k in 71:100 if rem(k,7) == 0] ``` #### Example: Making change @@ -402,11 +398,11 @@ If the command is run again, it is almost certain that a different value will be rand() ``` -This call will return a vector of $10$ such random numbers: +This call will return a vector of 4 such random numbers: ```{julia} -rand(10) +rand(4) ``` The `rand` function is easy to use. The only common source of confusion is the subtle distinction between `rand()` and `rand(1)`, as the latter is a vector of $1$ random number and the former just $1$ random number. @@ -544,23 +540,6 @@ answ = 1 radioq(choices, answ) ``` -###### Question - -An arithmetic sequence ($a_0$, $a_1 = a_0 + h$, $a_2=a_0 + 2h, \dots,$ $a_n=a_0 + n\cdot h$) is specified with a starting point ($a_0$), a step size ($h$), and a number of points $(n+1)$. This is not the case with the colon constructor which take a starting point, a step size, and a suggested last value. This is not the case a with the default for the `range` function, with signature `range(start, stop, length)`. However, the documentation for `range` shows that indeed the three values ($a_0$, $h$, and $n$) can be passed in. Which signature (from the docs) would allow this: - -```{julia} -#| echo: false -choices = [ - "`range(start, stop, length)`", - "`range(start, stop; length, step)`", - "`range(start; length, stop, step)`", - "`range(;start, length, stop, step)`"] -answer = 3 -explanation = """ -This is a somewhat vague question, but the use of `range(a0; length=n+1, step=h)` will produce the arithmetic sequence with this parameterization. -""" -buttonq(choices, answer; explanation) -``` ###### Question @@ -708,28 +687,22 @@ Credit card numbers have a check digit to ensure data entry of a 16-digit number Let's see if `4137 8947 1175 5804` is a valid credit card number? -First, we enter it as a value and immediately break the number into its digits: +First, we enter it as a value, the immediately break the number into its digits and call `reverse` to flip their order: ```{julia} x = 4137_8947_1175_5804 # _ in a number is ignored by parser -xs = digits(x) +xs = reverse(digits(x)) ``` -We reverse the order, so the first number in digits is the largest place value in `xs` + +Now, the 1st, 3rd, 5th, ... digit is doubled. We do this by creating a vector with the pattern 2,1,2, ... and then multiplying by this component by component:^[There are many different ways this alternate multiplying by `2` can be achieved, but most benefit from knowing an if-then-else command.] ```{julia} -reverse!(xs) +two_ones = repeat([2,1]; outer=length(xs) ÷ 2) +xs = two_ones .* xs ``` -Now, the 1st, 3rd, 5th, ... digit is doubled. We do this through indexing: - -```{julia} -for i in 1:2:length(xs) - xs[i] = 2 * xs[i] -end -``` - -Numbers greater than 9, have their digits added, then all the resulting numbers are added. This can be done with a generator: +Numbers greater than 9, have their digits added, then all the resulting numbers are added. This can be done different ways. Here we use a generator: ```{julia} @@ -747,5 +720,14 @@ Darn. A typo. is `4137 8047 1175 5804` a possible credit card number? ```{julia} #| hold: true #| echo: false -booleanq(true) +let + x = 4137_8047_1175_5804 + xs = digits(x) + reverse!(xs) + two_ones = repeat([2,1]; outer=length(xs) ÷ 2) + xs = two_ones .* xs + z = sum(sum(digits(xi)) for xi in xs) + val = iszero(rem(z,10)) + booleanq(val) +end ``` diff --git a/quarto/basics/variables.qmd b/quarto/basics/variables.qmd index ad1a8b8..dba6085 100644 --- a/quarto/basics/variables.qmd +++ b/quarto/basics/variables.qmd @@ -14,22 +14,18 @@ using CalculusWithJulia nothing ``` -```{julia} -#| echo: false -imgfile = "figures/calculator.png" -caption = "Screenshot of a calculator provided by the Google search engine." -# ImageFile(:precalc, imgfile, caption) -nothing -``` - -![Screenshot of a calculator provided by the Google search engine.](figures/calculator.png) +::: {#fig-google-calculator-screenshot} +![](figures/calculator.png) +Screenshot of a calculator provided by the Google search engine +::: The Google calculator has a button `Ans` to refer to the answer to the previous evaluation. This is a form of memory. The last answer is stored in a specific place in memory for retrieval when `Ans` is used. In some calculators, more advanced memory features are possible. For some, it is possible to push values onto a stack of values for them to be referred to at a later time. This proves useful for complicated expressions, say, as the expression can be broken into smaller intermediate steps to be computed. These values can then be appropriately combined. This strategy is a good one, though the memory buttons can make its implementation a bit cumbersome. -With `Julia`, as with other programming languages, it is very easy to refer to past evaluations. This is done by *assignment* whereby a computed value stored in memory is associated with a name (sometimes thought of as symbol or label). The name can be used to look up the value later. Assignment does not change the value of the object being assigned, it only introduces a reference to it. +`Julia`, as with other programming languages, makes it is very easy to refer to many past evaluations. +This is done by *assignment* whereby the storage of a value in memory is associated with a name. This pairing results in a *variable* and the value can be referenced through the variable name. The variable name may be called an identifier and internally kept as a symbol. Julia is a dynamic language which means the pairing can be updated and the referenced values may have different storage types. An example of a built-in variable is the special variable `ans` which refers to the last computed value in an interactive session. Assignment in `Julia` is handled by the equals sign and takes the general form `variable_name = value`. For example, here we assign values to the variables `x` and `y` @@ -49,7 +45,7 @@ x Just typing a variable name (without a trailing semicolon) causes the assigned value to be displayed. -Variable names can be reused (or reassigned), as here, where we redefine `x`: +Variable names can be reused (or reassigned)^[The `Pluto` interface for `Julia` is idiosyncratic, as variables are *reactive*. This interface allows changes to a variable `x` to propagate to all other cells referring to `x`. Consequently, the variable name can only be assigned *once* per notebook **unless** the name is in some other namespace, which can be arranged by including the assignment inside a function or a `let` block.], as here, where we redefine `x`: ```{julia} @@ -57,13 +53,8 @@ Variable names can be reused (or reassigned), as here, where we redefine `x`: x = 2 ``` -:::{.callout-note} -## Note -The `Pluto` interface for `Julia` is idiosyncratic, as variables are *reactive*. This interface allows changes to a variable `x` to propagate to all other cells referring to `x`. Consequently, the variable name can only be assigned *once* per notebook **unless** the name is in some other namespace, which can be arranged by including the assignment inside a function or a `let` block. -::: - -`Julia` is referred to as a "dynamic language" which means (in most cases) that a variable can be reassigned with a value of a different type, as we did with `x` where first it was assigned to a floating point value then to an integer value. (Though we meet some cases - generic functions - where `Julia` balks at reassigning a variable if the type is different.) +`Julia` is referred to as a "dynamic language" which means (in most cases) that a variable can be reassigned with a value of a different type, as we did with `x` where first it was assigned to a floating point value then to an integer value. (Though we meet some cases---generic functions---where `Julia` balks at reassigning a variable if the type is different.) More importantly than displaying a value, is the use of variables to build up more complicated expressions. For example, to compute @@ -181,12 +172,18 @@ Q = R^(2/3) * S^(1/2) / n * A ## Where math and computer notations diverge -It is important to recognize that `=` to `Julia` is not in analogy to how $=$ is used in mathematical notation. The following `Julia` code is not an equation: +It is important to recognize that `=` to `Julia` is not in analogy to how $=$ is used in mathematical notation. + +First, set `x` to be $3$: + +```{julia} +x = 3 +``` + +The following `Julia` code is not an equation: ```{julia} -#| hold: true -x = 3 x = x^2 ``` @@ -219,12 +216,12 @@ x = x - (x^2 - 2) / (2x) x = x - (x^2 - 2) / (2x) ``` -Repeating this last line will generate new values of `x` based on the previous one - no need for subscripts. This is exactly what the mathematical notation indicates is to be done. +Repeating this last line will generate new values of `x` based on the previous one---no need for subscripts. This is exactly what the mathematical notation indicates is to be done. ::: {.callout-note} ## Use of = -The distinction between ``=`` versus `=` is important and one area where common math notation and common computer notation diverge. The mathematical ``=`` indicates *equality*, and is often used with equations and also for assignment. Later, when symbolic math is introduced, the `~` symbol will be used to indicate an equation, though this is by convention and not part of base `Julia`. The computer syntax use of `=` is for *assignment* and *re-assignment*. Equality is tested with `==` and `===`. +The distinction between ``=`` versus `=` is important and one area where common math notation and common computer notation diverge. The mathematical ``=`` indicates *equality*, and is often used with equations and also for assignment. Later, when symbolic math is introduced, the `~` symbol will be used to indicate an equation, though this is by convention and not part of base `Julia`. The computer syntax use of `=` is for *assignment* and *re-assignment*. Equality is tested with `==` and identicalness (or egal) with `===`. ::: @@ -283,20 +280,14 @@ For example, we could have defined `theta` (`\theta[tab]`) and `v0` (`v\_0[tab]` θ = 45; v₀ = 200 ``` +These notes often use Unicode alternatives for some variable. + :::{.callout-note} ## Emojis There is even support for tab-completion of [emojis](https://github.com/JuliaLang/julia/blob/master/stdlib/REPL/src/emoji_symbols.jl) such as `\:snowman:[tab]` or `\:koala:[tab]` ::: - -:::{.callout-note} -## Unicode -These notes often use Unicode alternatives for some variable. Originally this was to avoid a requirement of `Pluto` of a single use of assigning to a variable name in a notebook without placing the assignment in a `let` block or a function body. Now, they are just for clarity through distinction. - -::: - - ##### Example diff --git a/quarto/basics/vectors.qmd b/quarto/basics/vectors.qmd index 8a5c6d4..1247b5e 100644 --- a/quarto/basics/vectors.qmd +++ b/quarto/basics/vectors.qmd @@ -1,4 +1,4 @@ -# Vectors and containers +# Points, vectors, and containers {{< include ../_common_code.qmd >}} @@ -8,16 +8,155 @@ #| results: "hidden" using CalculusWithJulia using Plots -plotly() +gr() # static images throughout using Measures using LaTeXStrings - +using LinearAlgebra nothing ``` -One of the first models learned in physics are the equations governing the laws of motion with constant acceleration: $x(t) = x_0 + v_0 t + 1/2 \cdot a t^2$. This is a consequence of Newton's second [law](http://tinyurl.com/8ylk29t) of motion applied to the constant acceleration case. A related formula for the velocity is $v(t) = v_0 + at$. The following figure is produced using these formulas applied to both the vertical position and the horizontal position: +This section discusses the mathematical objects points and vectors and then their representation using different container types from `Julia`. It ends with a discussion of the basics of some fundamental container types. + +## Points + +A point in two-dimensional space is typically represented in the Cartesian plane through a pair of coordinates $(x,y)$. The origin is the point $(0,0)$. Common mathematical notation is to use parentheses to group the two coordinates with the order on the axes to be understood ($x$ is first, $y$ is second). If a point is labeled, it is typical to use a capital letter. For a point in 3-dimensions, the notation just adds a coordinate to get $(x,y,z)$ representing a generic point and $(0,0,0)$ the origin. A generalization to $n$-dimensions might use the notation $(x_1, x_2, \dots, x_n)$ to represent a point. Some books might refer to this as an $n$-tuple. + +The coordinates represent a position along an axis. For example, the point $(4,5)$ being $4$ units along the $x$ axis and $5$ units up the $y$ axis. + +For the $1$ dimensional case, the point is just a number on the number line. Typically a point is written as a number; there isn't a common convention to write as a $1$-tuple, in the sense above. + +A typical question about points is how far apart are two different points. For $2$-dimensional points $P = (x_1, y_1)$ and $Q = (x_2, y_2)$ we can use the distance formula: + +$$ +d = \overline{PQ} = \sqrt{(x_1 - x_2)^2 + (y_1 - y_2)^2}. +$$ + +This formula agrees with Pythagorean's theorem for right triangles. + +For $n$-dimensional points, the same formula may be used with adjustements to the notation. Suppose $P = (x_1, x_2, \dots, x_n)$ and $Q = (y_1, y_2, \cdots, y_n)$. then + +$$ +d = \overline{PQ} = \sqrt{(x_1 - y_1)^2 + (x_2 - y_2)^2 + \cdots + (x_n - y_n)^2}. +$$ + +(Note the difference in how the point is described---in two dimensions we use variations of $(x,y)$, in $n$-dimensions just a single subscripted name for the components of a point. + +It is well known that two distinct points determine a line in the Cartesian plane, a fact that extends to $n$-dimensional space. Two points also define a line segment, that part of the line bounded by the two endpoints. + +## Vectors + +We now discuss a related but not identical mathematical object, a vector. + +As mentioned, a line segment is the portion of a line between two points. A line segment can be given a direction by assigning an initial point and a terminal point. A directed line segment has both a direction and a magnitude. A vector is an abstraction where just these two properties---a **direction** and a **magnitude**---are intrinsic. + +While a directed line segment can be represented by a vector, a single vector describes all such line segments found by translation. That is, how the vector is located when visualized is for convenience, it is not a characteristic of the vector. +A two-dimensional vector has two components, as does a point in the Cartesian plane. To distinguish a vector from a point a different mathematical notation is used. The vector components are combined with angle brackets and any name traditionally has an arrow mark above it: $\vec{v} = \langle x,~ y \rangle$^[The diacritical arrow mark indicates a vector. Alternatively, vector names may be typset in boldface. Both are a means to distinguish from a scalar or number]. + +Suppose a vector is defined as being between two points $P = (x_1, y_1)$ and $Q = (x_2, y_2)$ with $P$ the endpoint, then the vector connecting $P$ to $Q$ would be $\vec{v} = \langle x_2 - x_1, ~ y_2 - y_1 \rangle$. + +When $P = (0,0)$, the the point $Q = (x_2, y_2)$ has the same components as the vector $\vec{v} = \langle x_2, ~ y_2 \rangle$ leading to a natural indentification between a point and vector, though they represent different things. + + + + +### Basic properties + +The magnitude of a vector is the length of the line segment between the tip and tail and is given by the distance formula. + +::: {.definition title="The norm of a vector"} +$$ +\lVert \vec{v} \rVert = \lVert \langle x, ~ y\rangle \rVert = \sqrt{x^2 + y^2} +$$ +::: +The notation $\lVert \vec{v} \rVert$ is called the *norm* of the vector. A vector with magnitude $1$ is called a *unit* vector. + + +There are two fundamental vector operations. + +::: {.definition title="Scalar multiplication"} + +Vectors can be multiplied by a scalar: $c\vec{v} = \langle cx,~ cy \rangle$. A scalar is a number and scalar multiplication multiplies each component of a vector by that scalar to produce a scaled vector. For $c > 0$ scalar multiplication does not change the direction. If $c < 0$ then the direction is flipped opposite. + +::: + +::: {.definition title="Vector addition"} + +Vectors can be added component by component: $\vec{v} + \vec{w} = \langle v_x + w_x,~ v_y + w_y \rangle$. That is, each corresponding component adds to form a new vector. The $\vec{0}$ vector then would be just $\langle 0,~ 0 \rangle$ and would satisfy $\vec{0} + \vec{v} = \vec{v}$ for any vector $\vec{v}$. +::: + +Vector subtraction $\vec{v} - \vec{w}$ can be defined by $\vec{v} + (-\vec{w})$ and can be computed component by component. Subtraction can be visualized by drawing a vector of the same magnitude, but opposite direction of $\vec{w}$, anchored at the tip of $\vec{v}$. + +Scalar division is just scalar multiplication written through $\vec{v}/c$ (with the scalar last). There is no definition of a scalar divided by a vector. Scalar multiplication can also be written first or last for commutative number types. + +A general expression of the type $a\vec{v} + b\vec{w}$ for scalars $a$ and $b$ is called a *linear combination* of the vectors $\vec{v}$ and $\vec{w}$. + +For any non-zero vector, $\vec{v}$, the norm is the length of the vector and the scaled vector $\hat{v} = \vec{v} / \lVert\vec{v}\rVert$ is a unit vector in the direction of $\vec{v}$. Writing $\vec{v} = \lVert \vec{v} \rVert \hat{v}$ decomposes $\vec{v}$ into a length and a direction. + +#### Visualization + +In the Cartesian plane, we can visualize a vector by fixing a point and then drawing the corresponding arrow with the point as the end point. However vectors can be anchored anywhere in the plane so this fixed point should be chosen to illustrate some feature. + +We illustrate scalar multiplication and scalar addition in @fig-scalar-multiplication-vector-addition. + +::: {#fig-scalar-multiplication-vector-addition} + +```{julia} +#| echo: false +let + plt = plot(; + xaxis=([], false), + yaxis=([], false), + framestyle=:origin, + legend=false) + P = (0,0) + v = (1, 2) + d = norm(v) + v̂ = v ./ d + scatter!(plt, [P]; marker=(4,:black)) + plot!(plt, [P, v]; line=(3, :black), arrow=true) + plot!(plt, [P, (2).*v]; line=(1, :blue), arrow=true) + plot!(plt, [P, (-1).*v]; line=(1, :red), arrow=true) + annotate!(plt, + [(P..., text(L"P", :top)), + (((1/2).*v)..., text(L"\vec{v}", :top)), + (((-1/2).*v)..., text(L"-\vec{v}", :top)), + (((1 + 1/2).* v)..., text(L"2\vec{v}", :top))]) + + P = (0,0) + v = (1, 2) + w = (2, 1) + vw = v .+ w + plt2 = plot(; + xaxis=([], false), + yaxis=([], false), + framestyle=:origin, + legend=false) + scatter!(plt2, [P]; marker=(4, :black)) + plot!(plt2, [P, v]; arrow=true, line=(2, :red)) + plot!(plt2, [v, vw]; arrow=true, line=(2, :blue)) + plot!(plt2, [P, vw]; arrow=true, line=(2, :black)) + annotate!(plt2, + [(P..., text(L"P", :top)), + (((1/2).*v)..., text(L"\vec{v}", :top)), + ((v .+ (1/2).*w)..., text(L"\vec{w}", :top)), + ((1/2) .* vw..., text(L"\vec{v} + \vec{w}", :top, :left)) + ]) + + plot(plt, plt2) +end +``` + + +The left figure shows vector $\vec{v}$, $-\vec{v}$, and the vector $2\vec{v}$ all anchored at $P$. The vector $-\vec{v}$ has the same magnitude but opposite direction as $\vec{v}$; the vector $2\vec{v}$ has the same direction but double the magnitude as $\vec{v}$. The right figure visualizes $\vec{v} + \vec{w}$ by drawing $\vec{w}$ anchored at the top of $\vec{v}$ and viewing the new vector from $P$ to the tip of $\vec{w}$. +::: + +### Example: motion +One of the first models learned in physics are the equations governing the laws of motion with constant acceleration: $x(t) = x_0 + v_0 t + 1/2 \cdot a \cdot t^2$. This is a consequence of Newton's second [law](http://tinyurl.com/8ylk29t) of motion applied to the constant acceleration case. A related formula for the velocity is $v(t) = v_0 + a\cdot t$. The following figure is produced using these formulas applied to both the vertical position and the horizontal position of some projectile. + + +::: {#fig-projectile-motion layout-ncol=1} ```{julia} #| hold: true #| echo: false @@ -42,14 +181,24 @@ function make_plot(t) t = 1/10 + t*2/10 - ts = range(0, stop=2, length=100) + ts = range(0, stop=t, length=100) xys = map(xn, ts) xs, ys = [p[1] for p in xys], [p[2] for p in xys] - plt = plot(xs, ys, legend=false, size=fig_size, xlims=(0,45), ylims=(0,70)) + + plt = plot(;legend=false, size=fig_size, xlims=(0,45), ylims=(0,75)) + plot!(plt, xs, ys; line=(1, :black)) + + ts = range(t, 2, length=100) + xys = map(xn, ts) + xs, ys = [p[1] for p in xys], [p[2] for p in xys] + plot!(plt, xs, ys; line=(:dot, 1, :gray)) + plot!(plt, zero, extrema(xs)...) - arrow!(xn(t), 10*unit(xn(t)), color="black") + #arrow!(xn(t), 10*unit(xn(t)), color="black") + scatter!([Tuple(xn(t))]; marker=(5, :black)) + arrow!(0*xn(t), xn(t); color="black") arrow!(xn(t), 10*unit(vn(t)), color="red") arrow!(xn(t), 10*unit(an(t)), color="green") @@ -59,31 +208,29 @@ function make_plot(t) end imgfile = tempname() * ".gif" -caption = """ + +n = 8 +anim = @animate for i=1:1/2:n + make_plot(i) +end + +gif(anim, imgfile, fps = 2) + +plotly() +ImageFile(imgfile) +``` Position, velocity, and acceleration vectors (scaled) for projectile motion. Vectors are drawn with tail on the projectile. The position vector (black) points from the origin to the projectile, the velocity vector (red) is in the direction of the trajectory, and the acceleration vector (green) is a constant pointing downward. +::: -""" - -n = 8 -anim = @animate for i=1:n - make_plot(i) -end - -gif(anim, imgfile, fps = 1) - -plotly() -ImageFile(imgfile, caption) -``` - -For the motion in the above figure, the object's $x$ and $y$ values change according to the same rule, but, as the acceleration is different in each direction, we get different formula, namely: $x(t) = x_0 + v_{0x} t$ and $y(t) = y_0 + v_{0y}t - 1/2 \cdot gt^2$. +For the motion in @fig-projectile-motion, the object's $x$ and $y$ values change according to the same rule, but, as the acceleration is different in each direction, we get different formula, namely: $x(t) = x_0 + v_{0x} t$ and $y(t) = y_0 + v_{0y}t - 1/2 \cdot gt^2$. The acceleration is $0$ in the $x$ direction and $-g$ in the $y$ direction (all due to gravity). -It is common to work with *both* formulas at once. Mathematically, when graphing, we naturally pair off two values using Cartesian coordinates (e.g., $(x,y)$). Another means of combining related values is to use a *vector*. The notation for a vector varies, but to distinguish them from a point we will use $\langle x,~ y\rangle$. With this notation, we can use it to represent the position, the velocity, and the acceleration at time $t$ through: +It is common to work with *both* formulas at once. The following uses vectors to define the position, velocity, and acceleration dependent on $t$: $$ @@ -96,292 +243,120 @@ $$ -Don't spend time thinking about the formulas if they are unfamiliar. The point emphasized here is that we have used the notation $\langle x,~ y \rangle$ to collect the two values into a single object, which we indicate through a label on the variable name. These are vectors, and we shall see they find use far beyond this application. +Don't spend time thinking about the formulas if they are unfamiliar. The point emphasized here is that we have used vector notation to collect the two values into a single object. + +@fig-projectile-motion shows the position vector, $\vec{x}$, anchored at the origin and the velocity and acceleration vectors, $\vec{v}$ and $\vec{a}$, anchored at the position. The acceleration is a constant, pointing downwards; the velocity tracking the direction of the motion. -Initially, our primary use of vectors will be as containers, but it is worthwhile to spend some time to discuss properties of vectors and their visualization. - - -A line segment in the plane connects two points $(x_0, y_0)$ and $(x_1, y_1)$. The length of a line segment (its magnitude) is given by the distance formula $\sqrt{(x_1 - x_0)^2 + (y_1 - y_0)^2}$. A line segment can be given a direction by assigning an initial point and a terminal point. A directed line segment has both a direction and a magnitude. A vector is an abstraction where just these two properties $-$ a **direction** and a **magnitude** $-$ are intrinsic. While a directed line segment can be represented by a vector, a single vector describes all such line segments found by translation. That is, how the vector is located when visualized is for convenience, it is not a characteristic of the vector. In the figure above, all vectors are drawn with their tails at the position of the projectile over time. - - -We can visualize a (two-dimensional) vector as an arrow in space. This arrow has two components. We represent a vector mathematically as $\langle x,~ y \rangle$. For example, the vector connecting the point $(x_0, y_0)$ to $(x_1, y_1)$ is $\langle x_1 - x_0,~ y_1 - y_0 \rangle$. - - -The magnitude of a vector comes from the distance formula applied to a line segment, and is $\| \vec{v} \| = \sqrt{x^2 + y^2}$. - - -```{julia} -#| hold: true -#| echo: false -## generic vector -gr() - -p0 = [0,0] -a1 = [4,1] -b1 = [-2,2] -unit(v::Vector) = v / norm(v) - -plt = plot(legend=false, size=fig_size) -arrow!(p0, a1, color="blue") -arrow!([1,1], unit(a1), color="red") -annotate!([(2, .4, L"v"), (1.6, 1.05, L"\hat{v}")]) - -imgfile = tempname() * ".png" -png(plt, imgfile) - -caption = "A vector and its unit vector. They share the same direction, but the unit vector has a standardized magnitude." - -plotly() -ImageFile(imgfile, caption) -``` - -We call the values $x$ and $y$ of the vector $\vec{v} = \langle x,~ y \rangle$ the components of the $v$. - - -Two operations on vectors are fundamental. - - -* *Scalar multiplication*: Vectors can be multiplied by a scalar (a real number): $c\vec{v} = \langle cx,~ cy \rangle$. Geometrically this scales the vector by a factor of $\lvert c \rvert$ and switches the direction of the vector by $180$ degrees (in the $2$-dimensional case) when $c < 0$. A *unit vector* is one with magnitude $1$, and, except for the $\vec{0}$ vector, can be formed from $\vec{v}$ by dividing $\vec{v}$ by its magnitude. A vector's two parts are summarized by its direction given by a unit vector **and** its magnitude given by the norm. - -* *Vector addition*: Vectors can be added: $\vec{v} + \vec{w} = \langle v_x + w_x,~ v_y + w_y \rangle$. That is, each corresponding component adds to form a new vector. Similarly for subtraction. The $\vec{0}$ vector then would be just $\langle 0,~ 0 \rangle$ and would satisfy $\vec{0} + \vec{v} = \vec{v}$ for any vector $\vec{v}$. Vector addition, $\vec{v} + \vec{w}$, is visualized by placing the tail of $\vec{w}$ at the tip of $\vec{v}$ and then considering the new vector with tail coming from $\vec{v}$ and tip coming from the position of the tip of $\vec{w}$. Subtraction is different, place both the tails of $\vec{v}$ and $\vec{w}$ at the same place and the new vector has tail at the tip of $\vec{w}$ and tip at the tip of $\vec{v}$. - - -```{julia} -#| hold: true -#| echo: false -## vector_addition_image -gr() -p0 = [0,0] -a1 = [4,1] -b1 = [-2,2] - - -plt = plot(legend=false, size=fig_size) -arrow!(p0, a1, color="blue") -arrow!(p0+a1, b1, color="red") -arrow!(p0, a1+b1, color="black") -annotate!([(2, .25, L"a"), (3, 2.25, L"b"), (1.35, 1.5, L"a+b")]) - -imgfile = tempname() * ".png" -png(plt, imgfile) - -caption = "The sum of two vectors can be visualized by placing the tail of one at the tip of the other" - -plotly() -ImageFile(imgfile, caption) -``` - -```{julia} -#| hold: true -#| echo: false -## vector_subtraction_image -gr() -p0 = [0,0] -a1 = [-2,2] -b1 = [4,1] - -plt = plot(legend=false, size=fig_size) -arrow!(p0, a1, color="blue") -arrow!(p0, b1, color="red") -arrow!(b1, a1-b1, color="black") -annotate!(plt, [(-1, .5, L"a"), (2.45, .5, L"b"), (1, 1.75, L"a-b")]) - - -imgfile = tempname() * ".png" -png(plt, imgfile) - -caption = "The difference of two vectors can be visualized by placing the tail of one at the tip of the other" -plotly() -ImageFile(imgfile, caption) -``` - -The concept of scalar multiplication and addition, allow the decomposition of vectors into standard vectors. The standard unit vectors in two dimensions are $e_x = \langle 1,~ 0 \rangle$ and $e_y = \langle 0,~ 1 \rangle$. Any two dimensional vector can be written uniquely as $a e_x + b e_y$ for some pair of scalars $a$ and $b$ (or as, $\langle a, b \rangle$). This is true more generally where the two vectors are not the standard unit vectors - they can be *any* two non-parallel vectors. - - -```{julia} -#| hold: true -#| echo: false -### {{{vector_decomp}}} -gr() -p0 = [0,0] -aa = [1,2] -bb = [2,1] -cc = [4,3] -alpha = 2/3 -beta = 5/3 - -plt = plot(legend=false, size=fig_size) -arrow!(p0, cc, color="black", width=1) -arrow!(p0, aa, color="black", width=1) -arrow!(alpha*aa, bb, color="black", width=1) -arrow!(p0, alpha*aa, color="orange", width=4, opacity=0.5) -arrow!(alpha*aa, beta*bb, color="orange", width=4, opacity=0.5) -#annotate!(collect(zip([2, .5, 1.75], [1.25,1.0,2.25], [L"c",L"2/3 \cdot a", L"5/3 \cdot b"]))) - - -imgfile = tempname() * ".png" -png(plt, imgfile) - -caption = raw""" - -The vector ``\langle 4,3 \rangle`` is written as -``2/3 \cdot\langle 1,2 \rangle + 5/3 \cdot\langle 2,1 \rangle``. -Any vector ``\vec{c}`` -can be written uniquely as -``\alpha\cdot\vec{a} + \beta \cdot \vec{b}`` -provided ``\vec{a}`` and ``\vec{b}`` are not parallel. - -""" -plotly() -ImageFile(imgfile, caption) -``` - -The two operations of scalar multiplication and vector addition are defined in a component-by-component basis. We will see that there are many other circumstances where performing the same action on each component in a vector is desirable. - - ---- - - -When a vector is placed with its tail at the origin, it can be described in terms of the angle it makes with the $x$ axis, $\theta$, and its length, $r$. The following formulas apply: - +Now, let's breakdown $\vec{x}$. It comes from two components each of the form +$x_0 + v_0\cdot t -(1/2) a \cdot t^2$. If we define the initial position, initial velocity, and constant acceleration into vectors we have: $$ -r = \sqrt{x^2 + y^2}, \quad \tan(\theta) = y/x. +\begin{align*} +\vec{x}_0 &= \langle x_0 ,~ y_0 \rangle,\\ +\vec{v}_0 &= \langle v_{0x},~ v_{0y} \rangle, \text{ and }\\ +\vec{a} &= \langle 0,~ -g \rangle. +\end{align*} $$ -If we are given $r$ and $\theta$, then the vector is $v = \langle r \cdot \cos(\theta),~ r \cdot \sin(\theta) \rangle$. +With this, the vector $\vec{x}$ can be written in a way using scalar multiplication and vector addition that exactly mirrors where each component comes from: +$$ +\vec{x} = \vec{x}_0 + \vec{v}_0 \cdot t + (1/2) \vec{a} \cdot t^2, +$$ -```{julia} -#| hold: true -#| echo: false -## vector_rtheta -gr() -p0 = [0,0] +This shows one reason why vectors (and generalizations) are so useful, they can unify multiple components into a single equation. -plt = plot(legend=false, size=fig_size) -arrow!(p0, [2,3], color="black") -arrow!(p0, [2,0], color="orange") -arrow!(p0+[2,0], [0,3], color="orange") -annotate!(plt, collect(zip([.25, 1,1,1.75], [.15, 1.85,.25,1], [L"\theta",L"r", L"r \cdot \cos(\theta)", L"r \cdot \sin(\theta)"]))) #["θ","r", "r ⋅ cos(θ)", "r ⋅ sin(θ)"] - -imgfile = tempname() * ".png" -png(plt, imgfile) - -caption = raw""" - -A vector ``\langle x, y \rangle`` can be written as ``\langle r\cdot -\cos(\theta), r\cdot\sin(\theta) \rangle`` for values ``r`` and -``\theta``. The value ``r`` is a magnitude, the direction parameterized by -``\theta``.""" - -plotly() -ImageFile(imgfile, caption) -``` - -## Vectors in Julia - - -A vector in `Julia` can be represented by its individual components, but it is more convenient to combine them into a collection using the `[,]` notation: - - -```{julia} -x, y = 1, 2 -v = [x, y] # square brackets, not angles -``` - -The basic vector operations are implemented for vector objects. For example, the vector `v` has scalar multiplication defined for it: - - -```{julia} -10 * v -``` - -The `norm` function returns the magnitude of the vector (by default): - - -```{julia} -import LinearAlgebra: norm -``` - -```{julia} -norm(v) -``` - -A unit vector is then found by scaling by the reciprocal of the magnitude: - - -```{julia} -v / norm(v) -``` - -In addition, if `w` is another vector, we can add and subtract: - - -```{julia} -w = [3, 2] -v + w, v - 2w -``` - -We see above that scalar multiplication, addition, and subtraction can be done without new notation. This is because the usual operators have methods defined for vectors. - - -Finally, to find an angle $\theta$ from a vector $\langle x,~ y\rangle$, we can employ the `atan` function using two arguments: - - -```{julia} -norm(v), atan(y, x) # v = [x, y] -``` ### Higher dimensional vectors +Vectors, like points, are not limited to $2$- or $3$-dimensional space. A vector in an $n$-dimensional space has $n$ components, say $\vec{x} = \langle x_1, x_2, \dots, x_n \rangle$. -Mathematically, vectors can be generalized to more than $2$ dimensions. For example, using $3$-dimensional vectors are common when modeling events happening in space, and $4$-dimensional vectors are common when modeling space and time. +For such vectors, scalar multiplication and vector addition are defined in the same component-by-component manner. +The norm is similar, but to simplify notation we introduce the *dot product* between two $n$-dimensional vectors. + +Let $\vec{x} = \langle x_1, x_2, \dots, x_n \rangle$ and $\vec{y} = \langle y_1, y_2, \dots, y_n \rangle$ then define the dot product of $\vec{x}$ and $\vec{y}$ by + +$$ +\vec{x} \cdot \vec{y} = x_1y_1 + x_2 y_2 + \cdots x_n y_n +$$ + +It is clear from commutivity of multiplication that $\vec{x} \cdot \vec{y} = \vec{y} \cdot \vec{x}$. + +With this, we can define the magnitude of an $n$-dimensional vector through: + +$$ +\lVert \vec{x} \rVert = \sqrt{\vec{x} \cdot \vec{x}} +$$ + + +## Containers + +`Julia` has multiple data structures available. In this discussion we focus on a few foundational containers used to hold different collections. + +### Vector + +A vector in `Julia` is a container in `Julia` for holding an arbitrary number of objects of the same type. These are used to represent a vector with $n$ components. Vectors can be constructed with square brackets, `[]`, as follows: + +```{julia} +v = [1, 2, 3] # 𝐯 = ⟨1, 2, 3⟩ +``` + +As can be read, this is a three element `Vector` of integer values. The standard display of a vector includes the length and the type. The square brackets in this case are used for array concatenation; they also have other meanings in the language, as we will soon see. + +Vectors have scalar multiplication and vector addition supported: + +```{julia} +w = [3, 2, 1] +2*v, v + w +``` + +The dot product and the norm functionality are found in the standard library `LinearAlgebra`. This must be loaded in order to be used: + +```{julia} +using LinearAlgebra +``` + +Once loaded, we can access these features as follows: + +```{julia} +dot(v, w), norm(v), sqrt(dot(v,v)) +``` In `Julia` there are many uses for vectors outside of physics applications. A vector in `Julia` is just a one-dimensional collection of similarly typed values and a special case of an array. Such objects find widespread usage. For example: * In plotting graphs with `Julia`, vectors are used to hold the $x$ and $y$ coordinates of a collection of points to plot and connect with straight lines. There can be hundreds of such points in a plot. - * Vectors are a natural container to hold the roots of a polynomial or zeros of a function. + * Vectors are a natural container to hold the coefficients of a polynomial. + * Vectors are a natural container to hold the roots of a polynomial, or the zeros of a function. * Vectors may be used to record the state of an iterative process. * Vectors are naturally used to represent a data set, such as arise when collecting survey data. -Creating higher-dimensional vectors is similar to creating a two-dimensional vector, we just include more components: - - -```{julia} -fibs = [1, 1, 2, 3, 5, 8, 13] -``` - -Later we will discuss different ways to modify the values of a vector to create new ones, similar to how scalar multiplication does. - - -As mentioned, vectors in `Julia` are comprised of elements of a similar type, but the type is not limited to numeric values. Some examples: +As mentioned, vectors in `Julia` are comprised of elements of a similar type, but the type is not limited to just numeric types. Some examples: * a vector of strings might be useful for text processing, For example, the `WordTokenizers.jl` package takes text and produces tokens from the words. * a vector of Boolean values can naturally arise and is widely used within Julia's `DataFrames.jl` package. -* some applications are even naturally represented in terms of vectors of vectors (such as happens when plotting a collection points). +* some data is naturally represented in terms of vectors of vectors. For example, GPS tracks might use a vector to record sampled positions, each of which is a vector holding latitude and longitude readings. Look at the output of these two vectors, in particular how the underlying type of the components is described on printing. ```{julia} -["one", "two", "three"] # Array{T, 1} is shorthand for Vector{T}. Here T - the type - is String +["one", "two", "three"] # T is String ``` ```{julia} -[true, false, true] # vector of Bool values +[true, false, true] # T is Bool for Boolean values ``` -Finally, we mention that if `Julia` has values of different types it will promote them to a common type, as possible. Here we combine three types of numbers, and see that each is promoted to `Float64`: +Finally, we mention that if `Julia` has values of different types when put into a vector, they will be promoted to a common type, as possible. Here we combine three types of numbers, and see that each is promoted to `Float64`: ```{julia} @@ -395,319 +370,221 @@ Whereas, in this example where there is no common type to promote the values to, ["one", 2, 3.0, 4//1] ``` -## Other container types - -We end this section with some general comments that are for those interested in a bit more, but in general aren't needed to understand most all of what follows later. - -Vectors in `Julia` are a container for values. Vectors are -one of many different types of containers. The `Julia` manual uses the word "collection" to refer to a container of values that has properties like a vector. -Here we briefly review some alternate container types that are common in Julia and find use in these notes. - -First, here are some of the properties of a *vector*: - -* Vectors are *homogeneous*. That is, the container holding the vectors all have a common type. This type might be an abstract type, but for high performance, concrete types (like 64-bit floating point or 64-bit integers) are more typical. - -* Vectors are $1$-dimensional. - -* Vectors are ordered and *indexable* by their order. In Julia, the default indexing for vectors is $1$-based (starting) with one, with numeric access to the first, second, third, ..., last entries. - -* Vectors are *mutable*. That is, their elements may be changed; the container may be grown or shrunk - -* Vectors are *iterable*. That is, their values can be accessed one-by-one in various manners. - -These properties may not all be desirable for one reason or the other and `Julia` has developed a large number of alternative container types of which we describe a few here. - -### Arrays - -Vectors are $1$-dimensional, but there are desires for other dimensions. Vectors are implemented as a special case of a more general array type. Arrays are of dimension $N$ for various non-negative values of $N$. A common, and somewhat familiar, mathematical use of a $2$-dimensional array is a matrix. - -Arrays can have their entries accessed by dimension and within that dimension their components. By default these are $1$-based, but other offsets are possible through the `OffsetArrays.jl` package. A matrix can refer to its values either by row and column indices or, as a matrix has linear indexing by a single index. - -For large collections of data with many entries being $0$ a sparse array is beneficial for less memory intensive storage. These are implemented in the `SparseArrays.jl` package. - -There are numerous array types available. `Julia` has a number of *generic* methods for working with different arrays. An example would be `eachindex`, which provides an iterator interface to the underlying array access by index in an efficient manner. - -### Tuples - -Tuples are fixed-length containers where there is no expectation or enforcement of their having a common type. Tuples just combine values together in an *immutable* container. Like vectors they can be accessed by index (also $1$-based). Unlike vectors, the containers are *immutable* - elements can not be changed and the length of the container may not change. This has benefits for performance purposes. (For fixed length, mutable containers that have the benefits of tuples and vectors, the `StaticArrays.jl` package is available). - -While a vector is formed by placing comma-separated values within a `[]` pair (e.g., `[1,2,3]`), a tuple is formed by placing comma-separated values within a `()` pair. A tuple of length $1$ uses a convention of a trailing comma to distinguish it from a parenthesized expression (e.g. `(1,)` is a tuple, `(1)` is just the value `1`). -Vectors and tuples can appear at the same time: a vector of tuples---each of length $n$---can be used in plotting to specify points. +### AbstractArray -:::{.callout-note} -## Well, actually... -Technically, the tuple is formed just by the use of commas, which separate different expressions. The parentheses are typically used, as they clarify the intent and disambiguate some usage. In a notebook interface, it is useful to just use commas to separate values to output, as typically the only the last command is displayed. This usage just forms a tuple of the values and displays that. - -::: - -#### Named tuples - -There are *named tuples* where each component has an associated name. Like a tuple these can be indexed by number and unlike regular tuples also by name. +In `Julia` a `Vector` is of a more general type `AbstractVector` which itself is a specialization of the `AbstractArray` type. Abstract arrays are containers with multiple dimensions (`N`) filled with values of a certain type (`T`). Vectors are $1$-dimensional arrays; we will later see matrices which are $2$ dimensional arrays. The language can be confusing. We have a vector with $n$ components sits in an $n$ dimensional space (a vector space) but it is represented with a $1$ dimensional array with $n$ components. The array dimension allows additional access patterns than are described in the following. -For example, here a named tuple is constructed, and then its elements referenced: +### Tuple +A *tuple* is another container for holding a collection of $n$ elements. Tuples are formed by separating values by commas; typically enclosed with parentheses. For example ```{julia} -nt = (one=1, two="two", three=:three) # heterogeneous values (Int, String, Symbol) -nt.one, nt[2], nt[end] # named tuples have name or index access +P = (1, 2, 3) ``` - -::: {.callout-note} -## Named tuple and destructuring - -A *named* tuple is a container that allows access by index *or* by name. They are easily constructed. For example: +Tuples can hold values of different types^[A `NTuple` is a tuple with a fixed length where each element has the same type.]: ```{julia} -nt = (x0 = 1, x1 = 4, y0 = 2, y1 = 6) +Q = (true, 1, pi) ``` -The values in a named tuple can be accessed using the "dot" notation: +A $1$ element tuple is constructed with a comma as follows: ```{julia} -nt.x1 +R = (1, ) ``` -Alternatively, the index notation---using a *symbol* for the name---can be used: +A $0$-element tuple is constructed with `()$. + + +Tuples as a container do not support scalar multiplication or vector addition without additional effort. + +#### Vectors versus tuples + +Tuples are a fundamantal part of the language as a fixed-length container that can hold values with different types. Here are some differences between tuples and vectors with some concepts yet to be illustrated. + +* *homogeneous*: Vectors hold homogeneous elements of some type, `T`; tuples are heterogeneous and can hold values of any type. + +* *mathematical*: Vectors have scalar multiplication and vector addition defined; tuples do not have mathematical operations defined for them. + +* *iterable*: Both vectors and tuples have standard ways to iterate over their components. + +* *indexable*: Both vectors and tuples can have their $i$th component accessed through indexing. + +* *mutability*: Vectors can have the components modified or the length of the underlying object modified. Tuples are *immutable*. We won't discuss this here. + +* *storage*: Vectors must allocate memory when created, small tuples can be stored on the "stack" and not allocate. + +#### Named tuple + +A variant of a tuple is the named tuple where a name can be assigned to each component. For example ```{julia} -nt[:x1] +nt = (x = 1, y = 2, z = 4) ``` -(Indexing is described a bit later, but it is a way to pull elements out of a collection.) - -Named tuples are employed to pass parameters to functions. To find the slope, we could do: +The individual components can be accessed by name through "dot" syntax^[The dot syntax `nt.a` resolves to `getproperty(nt, :a)`, `:a` being a symbol.] ```{julia} -(nt.y1 - nt.y0) / (nt.x1 - nt.x0) +nt.y ``` +### Dictionary -However, more commonly used is destructuring, where named variables are extracted by name when the left hand side matches the right hand side: +A named tuple is an immutable, associative array mapping names (symbols) to values. A dictionary is a general container type for associating a key (of arbitrary type) with a value. + +*Pair notation* in `Julia` associates a left hand side to a right hand side and is parsed through `=>`. For example, ```{julia} -(;x0, x1) = nt # only extract what is desired -x1 - x0 +pr = 3 => 4 ``` -(This works for named tuples and other iterable containers in `Julia`. It also works the other way, if `x0` and `x1` are defined then `(;x0, x1)` creates a named tuple with those values.) +A dictionary is container to hold pairs: -::: - - -### Pairs, associative arrays - -Named tuples associate a name (in this case a symbol) to a value. More generally an associative array associates to each key a value, where the keys and values may be of different types. - -The `pair` notation, `key => value`, is used to make one association between the first and second value. - -A *dictionary* is used to have a container of associations. - -This example constructs a simple dictionary associating a spelled out name with a numeric value: ```{julia} d = Dict("one" => 1, "two" => 2, "three" => 3) ``` -The print out shows the keys are of type `String`, the values of type `Int64`, in this case. There are a number of different means to construct dictionaries. - - -The values in a dictionary can be accessed by name: +There are other constructors for dictionaries that can prove more convenient. +The basic dictionary is not guaranteed to keep the order of its components, rather it is designed to efficiently lookup a value from a given key. `Julia` uses square-bracket notation for this lookup.^[Square brackets used for access resolve to `getindex`, which is defined for many different container types.] For example, the keys of `d` are strings here we retrieve the value associated to one of them: ```{julia} d["two"] ``` -Named tuples are associative arrays where the keys are restricted to symbols. There are other types of associative arrays, specialized cases of the `AbstractDict` type with performance benefits for specific use cases. In these notes, dictionaries appear as output in some function calls. +Access will throw a `KeyError` if the key is not defined. -Unlike vectors and tuples, dictionaries are not currently supported by broadcasting. (To be described in the next section.) This causes no loss in usefulness, as the values can easily be iterated over, but the convenience of the dot notation is lost. - - -## The container interface in Julia - -There are numerous generic functions for working across the many different types of containers. Some are specific to containers which can be modified, some to associative arrays. But it is expected for different container types to implement as many as possible. We list a few here for completeness. Only a few will be used in these notes. - -### Indexing - -Vectors have an implied order: first element, second, last, etc. Tuples do as well. Matrices have two orders: by a row-column pair or by linear order where the first column precedes the second etc. Arrays are similar in that they have a linear order and can be accessed by their individual dimensions. - -To access an element in a vector, say the second, the underlying `getindex` function is used. This is rarely typed, as the `[` notation is used. This notation is used in a style similar to a function call, the indexes go between matching pairs. - -For example, we create a vector, tuple, and matrix: +The collection of keys or the collection of the values can be returned with `keys` or `values`: ```{julia} -v = [1,2,3,4] -t = (1,2,3,4) -m = [1 2; 3 4] +keys(d) ``` -The second element of each is accessed similarly: +These are *not* vectors, however they can be collected into vectors:^[The `collect` method takes an iterator and collects the values into an array, in this example a vector.] ```{julia} -v[2], t[2], m[2] -``` - -(All of `v`, `t`, and `m` have $1$-based indexing.) - -There is special syntax to reference the last index when used within the square braces: - -```{julia} -v[end], t[end], m[end] -``` - -The last element is also returned by `last`: - -```{julia} -last(v), last(t), last(m) -``` - -These use `lastindex` behind the scenes. There is also a `firstindex` which is associated with the `first` method: - - -```{julia} -first(v), first(t), first(m) -``` - -For indexing by a numeric index, a container of numbers may be used. Containers can be generated different ways, here we just use a vector to get the second and third elements: - -```{julia} -I = [2,3] -v[I], t[I], m[I] -``` - -When indexing by a vector, the value will not be a scalar, even if there is only one element indicated. - -Indexing can also be done by a mask of Boolean values with a matching length. This following mask should do the same as indexing by `I` above: - -```{julia} -J = [false, true, true, false] -v[J], t[J], m[J] -``` - -For the matrix, values can be referenced by row/column values. The following will extract the second row, first column: - -```{julia} -m[2, 1] -``` - -*If* a container has *only* one entry, then the `only` method will return that element (not within the container). Here we use a tuple to illustrate to emphasize the trailing comma in construction: - -```{julia} -s = ("one", ) -``` - -```{julia} -only(s) -``` - -There will be an error with `only` should the container not have just one element. - -### Mutating values - -Vectors and matrices can have their elements changed or mutated; tuples can not. The process is similar to assignment---using an equals sign---but the left hand side has indexing notation to reference which values within the container are to be updated. - -To change the last element of `v` to `0` we have: - -```{julia} -v[end] = 0 -v -``` - -We might read this as assignment, but what happens is the underlying container has an element indicated by the index mutated. The `setindex!` function is called behind the scenes. - - -The `setindex!` function will try to promote the value (`0` above) to the element type of the container. This can throw an error if the promotion isn't possible. For example, to specify an element as `missing` with `v[end] = missing` will error, as missing can't be promoted to an integer. - -If more than one value is referenced in the assignment, then more than one value can be specified on the right-hand side. - -Mutation is different from reassignment. A command like `v=[1,2,3,0]` would have had the same effect as `v[end] = 0`, but would be quite different. The first *replaces* the binding to `v` with a new container, the latter reaches into the container and replaces just a value it holds. - -### Size and type - -The `length` of a container is the number of elements in linear order: - -```{julia} -length(v), length(t), length(m) -``` - -The `isempty` method will indicate if the length is 0, perhaps in a performant way: - -```{julia} -isempty(v), isempty([]), isempty(t), isempty(()) +collect(keys(d)) ``` -The `size` of a container, when defined, takes into account its shape or the dimensions: +## Working with containers + +Vectors, tuples, and dictionaries are all collections used to group multiple elements into a single unit. Collections in `Julia` have some commonalities. + +First, we define some different collections: ```{julia} -size(v), size(m) # no size defined for tuples +v = [1, 2, 3, 4, 5] +t = (1, 2, 3, 4) +nt = (one=1, two=2, three=3) +d = Dict("one"=>1, "two"=>2) ``` -Arrays, and hence vectors and matrices have an element type given by `eltype` (the `typeof` method returns the container type: +### Number of elements + +The number of elements is returned by `length`: ```{julia} -eltype(v), eltype(t), eltype(m) +length(v), length(t), length(nt), length(d) ``` -(The element type of the tuple is `Int64`, but this is only because of this particular tuple. Tuples are typically heterogeneous containers---not homogeneous like vectors---and do not expect to have a common type. The `NTuple` type is for tuples with elements of the same type.) - -### Modifying the length of a container - -Vectors and some other containers allow elements to be added on or elements to be taken off. In computer science a queue is a collection that is ordered and has addition at one or the other end. Vectors can be used as a queue, though for just that task, there are more performant structures available. - -Two key methods for queues are `push!` and `pop!`. We `push!` elements onto the end of the queue: +A check if there are no elements in the container is done by `isempty`: ```{julia} -push!(v, 5) +isempty(v), isempty(t), isempty(nt), isempty(d) ``` -The output is expected---`5` was added to the end of `v`. What might not be expected is the underlying `v` is changed without assignment. (Actually `mutated`, the underlying container assigned to the symbol `v` is extended, not replaced.) +### Element access -:::{.callout-note} -## Trailing exclamation point convention -The function `push!` has a trailing exclamation point which is a `Julia` convention to indicate one of the underlying arguments (traditionally the first) will be *mutated* by the function call. -::: +The dictionary and the tuple can have their values accessed by name, as shown. For all but the dictionary, the values have an order (first, second, ...). -The `pop!` function is somewhat of a reverse: it takes the last element and "pops" it off the queue, leaving the queue one element shorter and returning the last element. +This order can be reversed, using `reverse`: ```{julia} -pop!(v) +reverse(v), reverse(t), reverse(nt) ``` + +Ordered containers can be accessed by a linear index, starting at `1` for most collections.^[`Julia` is a 1-based language, so most counting starts at `1`, but this can be modified for a given type by defining a `firstindex` method for the type.] + +To find the third element, say, we have + ```{julia} -v +v[3], t[3], nt[3] ``` -There are also `pushfirst!`, `popfirst!`, `insert!` and `deleteat!` methods. +There are also ways to index most collections by specifying more than one element or using a mask of Boolean values to select elements. (A mask is a boolean vector of the same length.) +The collection of indices will be returned as an iterable by `keys` or `eachindex`. + +Special keywords `begin` and `end` can be used to reference the first and last elements of an ordered collection. + +```{julia} +v[end], t[end], nt[end] +``` + +Basic arithmetic can be used with these keywords:^[When parsed, these keywords lower to a call of `firstindex` or `lastindex` and the value returned is used in the arithmetic.] + +```{julia} +v[end - 1] +``` + +To access the first and last elements of an ordered collection, there are also `first` and `last` methods: + +```{julia} +first(v), last(v) +``` + +If a container has exactly one element, it can be retrieved generically by `only`: + +```{julia} +only(Dict("a" => 1)) +``` + +Calling `only` will error if the container is empty or has more than one element. ### Iteration -A very fundamental operation is to iterate over the elements of a collection one by one. +Each of these collection types can be accessed element by element through iteration. In computer programming a `for`-loop is the standard way to loop over the values in a collection. We illustrate alternatives. -In computer science the `for` loop is the basic construct to iterate over values. This example will iterate over `v` and add each value to `tot` which is initialized to be `0`: + +#### Destructuring + +A common means to name each element of a tuple or vector is to use destructuring. The following sets `a` to the first value, `b` to the second, etc.: ```{julia} -tot = 0 -for e in v - tot = tot + e -end -tot +a,b,c,d = t ``` -The `for` loop construct is central in many programming languages; in `Julia` for loops are very performant and very flexible, however, they are more verbose than needed. (In the above example we had to initialize an accumulator and then write three lines for the loop, whereas `sum(v)` would do the same---and in this case more flexibly, with just a single call.) Alternatives are usually leveraged---we mention a few. - -Iterating over a vector can be done by *value*, as above, or by *index*. For the latter the `eachindex` method creates an iterable for the indices of the container. For rectangular objects, like matrices, there are also many uses for `eachrow` and `eachcol`, though not in these notes. +Pairs can be destructured and also `first` and `last` are available: -There are a few basic patterns where alternatives to a `for` loop exist. We discuss two: +```{julia} +pr = 3 => 4 +a, b = pr +a, b, first(pr), last(pr) +``` -* mapping a function or expression over each element in the collection -* a reduction where a larger dimensional object is summarized by a lower dimensional one. In the example above, the $1$-dimensional vector is reduced to a $0$-dimensional scalar by summing the elements. + +Named tuples offer a way to destructure by name: + +```{julia} +(; two, three) = nt +three +``` + +This is useful more generally with other data structures. + +Named tuples can be constructed directly from variables in a reverse of the above. Notice the leading semicolon: + +```{julia} +nt1 = (; two, three) +``` #### Comprehensions @@ -740,20 +617,26 @@ as = [1, 2, 3] # evaluate a₀⋅x⁰, a₁⋅x¹, a₂⋅x² [a*x^(i-1) for (i, a) in enumerate(as)] ``` -(These values can then be easily summed to evaluate the polynomial.) +These values can then be easily summed to evaluate the polynomial: -When iterating over `enumerate` a tuple is returned. The use of `(i, a)` to iterate over these tuples destructures the tuple into parts to be used in the expression. +```{julia} +sum([a*x^(i-1) for (i, a) in enumerate(as)]) +``` -The `zip` function also is useful to *pair* off iterators. Redoing the above to have the powers iterated over: +The `sum` function is a *reduction* which takes a collection (in this case a comprehension, but more efficiently just the internal generator is needed) and adds the values together. There is a related `prod` function for finding the product of a collection of numbers. + + +When iterating over `enumerate` a tuple is returned at each step. The use of `(i, a)` to iterate over these tuples destructures the tuple into parts to be used in the expression. + +The `zip` function also is useful to *pair* off iterators. Redoing the above with powers that start at `0` and using just the generator part of the comprehension we have: ```{julia} as = [1, 2, 3] -inds = [0, 1, 2] -[a*x^i for (i, a) in zip(inds, as)] +powers = [0, 1, 2] +sum(a*x^i for (i, a) in zip(powers, as)) ``` -Like `enumerate`, the `zip` iterator has elements which are tuples. - +Like `enumerate`, the `zip` iterator has elements which are tuples. The `zip` operator is not limited to just two iterables. :::{.callout-note} ## Note @@ -761,11 +644,29 @@ The style generally employed herein is to use plural variable names for a collec ::: +:::{.callout-note} +## Numbers are iterable + +One design choice in `Julia` is to make numbers iterable. For example + +```{julia} +length(3) +``` + +Or even a comprehension: + +```{julia} +[x^2 for x in 3] +``` + +(The expression is also an `Array` but has `0` dimensions, unlike a vector which has `1`.) + +::: #### Broadcasting a function call -If we have a vector, `xs`, and a function, `f`, to apply to each value, there is a simple means to achieve this task that is shorter than a `for` loop or the comprehension `[f(x) for x in s]`. By adding a "dot" between the function name and the parenthesis that enclose the arguments, instructs `Julia` to "broadcast" the function call. The details allow for more much flexibility, but, for this purpose, broadcasting will take each value in `xs` and apply `f` to it, returning a vector of the same size as `xs`. When more than one argument is involved, broadcasting will try to pad out different sized objects to the same shape. Broadcasting can also *fuse* combined function calls. +If we have a vector, `xs`, and a function, `f`, to apply to each value, there is a simple means to achieve this task that is shorter than a `for` loop or the comprehension `[f(x) for x in xs]`. Adding a "dot" between the function name and the parenthesis that enclose the arguments, instructs `Julia` to "broadcast" the function call. The details allow for more much flexibility, but, for this purpose, broadcasting will take each element in `xs` and apply `f` to it, returning a vector of the same size as `xs`. When more than one argument is involved, broadcasting will try to pad out different sized objects to the same shape. Broadcasting can also efficiently *fuse* combined function calls. For example, the following will find, using `sqrt`, the square root of each value in a vector: @@ -785,14 +686,6 @@ sin.(xs) For each function call, the `.(` (and not `(`) after the name is the surface syntax for broadcasting. - -The `^` operator is an *infix* operator. Infix operators can be broadcast, as well, by using the form `.` prior to the operator, as in: - - -```{julia} -xs .^ 2 -``` - Here is an example involving the logarithm of a set of numbers. In astronomy, a logarithm with base $100^{1/5}$ is used for star [brightness](http://tinyurl.com/ycp7k8ay). We can use broadcasting to find this value for several values at once through: @@ -804,6 +697,49 @@ log.(base, ys) Broadcasting with multiple arguments allows for mixing of vectors and scalar values, as above, making it convenient when parameters are used. In broadcasting, there are times where it is desirable to treat a container as a scalar-like argument, a common idiom is to wrap that container in a 1-element tuple. +##### Broadcasting infix operators + +The `^` operator is an *infix* operator. Infix operators can be broadcast, as well, by placing the `.` prior to the operator, as in: + + +```{julia} +xs .^ 2 +``` + + +Tuples don't have built-in arithmetical operations like vectors do, but broadcasting can be used to give tuples scalar multiplication and vector addition. First we define two tuples of the same length then use broadcasting to add component-by-component: + +```{julia} +a, b = (1,2,3), (1,4,9) +a .+ b +``` + +For scalar multiplication, we have: + +```{julia} +2 .* a # space needed between "2" and the "." +``` + + +Broadcasting can be used over various shaped objects. Consider a vector of vector elements: + +```{julia} +xs = [1, 2, 3] +vv = [xs, xs.^2, xs.^3] +``` + +The last element of each element can be retrieved by broadcasting `last`: + +```{julia} +last.(vv) +``` + +To get the first, broadcasting `first` can be done. What about the second? The underlying `getindex` function can be used. In this example the scalar value `2` is padded out to match the length of `vv`: + +```{julia} +getindex.(vv, 2) +``` + As a final example, the task from statistics of centering and then squaring can be done with broadcasting. We go a bit further, showing how to compute the [sample variance](http://tinyurl.com/p6wa4r8) of a data set. This has the formula @@ -817,30 +753,32 @@ This can be computed, with broadcasting, through: ```{julia} #| hold: true -import Statistics: mean xs = [1, 1, 2, 3, 5, 8, 13] n = length(xs) -(1/(n-1)) * sum(abs2.(xs .- mean(xs))) +xbar = sum(xs) / n +ds = xs .- xbar +(1/(n-1)) * sum(ds.^2) ``` -This shows many of the manipulations that can be made with vectors. Rather than write `.^2`, we follow the definition of `var` and chose the possibly more performant `abs2` function which, in general, efficiently finds $|x|^2$ for various number types. The `.-` uses broadcasting to subtract a scalar (`mean(xs)`) from a vector (`xs`). Without the `.`, this would error. +This shows many of the manipulations that can be made with vectors. We subtracted the scalar mean from the vector of data using broadcasting. Without the `.`, this subtraction of mixed lengths would error. We then added these values after squaring. Of course this could be done with just one command, like `(xs .- xbar).^2`. There are more efficient ways of computing this, as there are intermediate arrays created, but this faithfully follows the formula. Broadcasting is a widely used and powerful surface syntax which we will employ occasionally in the sequel. #### Mapping a function over a collection -The `map` function is very much related to broadcasting. Similarly named functions are found in many different programming languages. (The "dot" broadcast is mostly limited to `Julia` and mirrors a similar usage of a dot in `MATLAB`.) For those familiar with other programming languages, using `map` may seem more natural. Its syntax is `map(f, xs)`. There may be one or more iterable passed to `map`. +The `map` function is very much related to broadcasting, in that it applies a function to each element of an iterable. When more than one iterable is specifed, `map` applies the function to the `zip`ped iterables. Unlike broadcasting, `map` does not reshape the underlying iterables. + +Similarly named functions are found in many different programming languages, as `map` is one of the foundational higher-order, functional programming operations. (The "dot" broadcast is mostly limited to `Julia` and mirrors a similar usage of a dot in `MATLAB`.) For those familiar with other programming languages, using `map` may seem more natural. Its syntax is `map(f, xs)`. Additional iterables are passed after `xs`. For example, this will map `sin` over each value in `xs`, computing the same things as `sin.(xs)`: ```{julia} +xs = [0, pi/6, pi/4, pi/3, pi/2] map(sin, xs) ``` -The `map` function can be used with one or more iterators. - The `map` function can also be used in combination with `reduce`, a reduction. Reductions take a container with one or more dimensions and reduces the number of dimensions. An example might be: ```{julia} @@ -849,126 +787,18 @@ sum(map(sin, xs)) This has a performance drawback---there are two passes through the container, one to apply `sin` another to add. -The `mapreduce` function combines the map and reduce operations in one pass. It takes a third argument to reduce by in the second position. This is a *binary* operator. So this combination will map `sin` over `xs` and then add the results up: - -```{julia} -mapreduce(sin, +, xs) -``` - -There are other specialized reduction functions that reverse the order of the mapper and the reducer. For example, we have `sum` (used above) and `prod` for adding and multiplying values in a collection: - -```{julia} -sum(xs), prod(xs) -``` - -These are reductions, which fall back to a `mapreduce` call. They require a starting value (`init`) of `0` and `1` (which in this case can be determined from `xs`). The `sum` and `prod` function also allow as a first argument an initial function to map over the collection: +For this task, the `sum` reduction, as others, allows a function to be specified that is applied to each value in the container while the sum is being computed. This argument comes first. A recommened alternative to the previous would be: ```{julia} sum(sin, xs) ``` - -#### Other reductions - -There are other reductions, which summarize a container. We mention those related to the maximum or minimum of a collection. For these examples, we have +The `mapreduce` method more generally combines the map and reduce operations in one pass and is called by the above form. The `mapreduce` method takes a third argument to reduce by in the second position. This is a *binary* operator. So this combination will map `sin` over `xs` and then add the results up sequentially: ```{julia} -v = [1, 2, 3, 4] +mapreduce(sin, +, xs) ``` -The largest value in a numeric collection is returned by `maximum`: - -```{julia} -maximum(v) -``` - -Where this maximum occurred is returned by `argmax`: - -```{julia} -argmax(v) -``` - -For `v` these are the same. But if we were to apply `sin` to `v` say, then the result may not be in order. This can be done with, say, a call to `map` and then `maximum`, but the functions allow an initial function to be specified: - -```{julia} -maximum(sin, v), argmax(sin, v) -``` - -This combination is also the duty of `findmax`: - -```{julia} -findmax(sin, v) -``` - -There are also `minimum`, `argmin`, and `findmin`. - -The `extrema` function returns the maximum and minimum of the collection: - -```{julia} -extrema(v) -``` - -:::{.callout-note} -## `maximum` and `max` -In `Julia` there are two related functions: `maximum` and `max`. The `maximum` function generically returns the largest element in a collection. The `max` function returns the maximum of its *arguments*. - -That is, these return identical values: - -```{julia} -xs = [1, 3, 2] -maximum(xs), max(1, 3, 2), max(xs...) -``` - -The latter using *splatting* to iterate over each value in `xs` and pass it to `max` as an argument. -::: - -### Predicate functions - -A few reductions work with *predicate* functions---those that return `true` or `false`. Let's use `iseven` as an example, which tests if a number is even. - -We can check if *all* the elements of a container are even or if *any* of the elements of a container are even with `all` and `even`: - -```{julia} -xs = [1, 1, 2, 3, 5] -all(iseven, xs), any(iseven, xs) -``` - - -Related, we can count the number of `true` responses of the predicate function: - -```{julia} -count(iseven, xs) -``` - - -#### methods for associative arrays - -For dictionaries, the collection is unordered (by default), but iteration can still be done over "key-value" pairs. - -In `Julia` a `Pair` matches a key and a value into one entity. Pairs are made with the `=>` notation with the `key` on the left and the value on the right. - - -Dictionaries are a collection of pairs. The `Dict` constructor can be passed pairs directly: - -```{julia} -ascii = Dict("a"=>97, "b"=>98, "c"=>99) # etc. -``` - -To iterate over these, the `pairs` iterator is useful: - -```{julia} -collect(pairs(ascii)) -``` - -(We used `collect` to iterate over values and return them as a vector.) - -The keys are returned by `keys`, the values by `values`: - -```{julia} -keys(ascii) -``` - - ## Questions @@ -1043,7 +873,7 @@ What vector is in the same direction as $\vec{v} = \langle 3,~ 4 \rangle$ but is ```{julia} #| hold: true #| echo: false -choices = [q"[3, 4]", q"[30, 40]", q"[9.48683, 12.6491 ]", q"[10, 10]"] +choices = [q"[3, 4]", q"[30, 40]", q"[9.48683, 12.6491]", q"[10, 10]"] answ = 2 radioq(choices, answ) ``` @@ -1112,7 +942,7 @@ numericq(val) From [transum.org](http://www.transum.org/Maths/Exam/Online_Exercise.asp?Topic=Vectors). - +::: {#fig-identify-superpositions-of-vectors-transum} ```{julia} #| hold: true #| echo: false @@ -1135,11 +965,11 @@ quiver!(p, [(4 , sqrt(3))], quiver=[(3/2,-sqrt(3)/2)], color=:black, linewidth= quiver!(p, [(6+1/2 , sqrt(3)/2)], quiver=[(1/2, sqrt(3)/2)], color=:black, linewidth=5) # e delta = 1/4 -annotate!(p, [(2, 3/2*sqrt(3) -delta, L"a"), - (2+1/4, sqrt(3), L"b"), - (3+3/2+3/2, 3/2*sqrt(3)-delta, L"c"), - (4+3/4, sqrt(3) - sqrt(3)/4-delta, L"d"), - (6+3/4+delta, sqrt(3)/2 + sqrt(3)/4-delta, L"e") +annotate!(p, [(2, 3/2*sqrt(3) -delta, L"\vec{a}"), + (2+1/4, sqrt(3), L"\vec{b}"), + (3+3/2+3/2, 3/2*sqrt(3)-delta, L"\vec{c}"), + (4+3/4, sqrt(3) - sqrt(3)/4-delta, L"\vec{d}"), + (6+3/4+delta, sqrt(3)/2 + sqrt(3)/4-delta, L"\vec{e}") ]) @@ -1147,38 +977,41 @@ annotate!(p, [(2, 3/2*sqrt(3) -delta, L"a"), end ``` -The figure shows $5$ vectors. +Vectors on a non-Cartesian grid +::: + +@fig-identify-superpositions-of-vectors-transum shows $5$ vectors. -Express vector **c** in terms of **a** and **b**: +Express vector $\vec{c}$ in terms of $\vec{a}$ and $\vec{b}$: ```{julia} #| hold: true #| echo: false -choices = ["3a", "3b", "a + b", "a - b", "b-a"] +choices = [L"3\vec{a}", L"3\vec{b}", L"\vec{a} + \vec{b}", L"\vec{a} - \vec{b}", L"\vec{b}-\vec{a}"] answ = 1 radioq(choices, answ) ``` -Express vector **d** in terms of **a** and **b**: +Express vector $\vec{d}$ in terms of $\vec{a}$ and $\vec{b}$: ```{julia} #| hold: true #| echo: false -choices = ["3a", "3b", "a + b", "a - b", "b-a"] +choices = [L"3\vec{a}", L"3\vec{b}", L"\vec{a} + \vec{b}", L"\vec{a} - \vec{b}", L"\vec{b}-\vec{a}"] answ = 3 radioq(choices, answ) ``` -Express vector **e** in terms of **a** and **b**: +Express vector $\vec{e}$ in terms of $\vec{a}$ and $\vec{b}$: ```{julia} #| hold: true #| echo: false -choices = ["3a", "3b", "a + b", "a - b", "b-a"] +choices = [L"3\vec{a}", L"3\vec{b}", L"\vec{a} + \vec{b}", L"\vec{a} - \vec{b}", L"\vec{b}-\vec{a}"] answ = 4 radioq(choices, answ) ``` @@ -1242,3 +1075,55 @@ q"zs^(1./2)" answ = 2 radioq(choices, answ, keep_order=true) ``` + + +###### Question + +Comprehensions mirror set notation and return a vector (as illustrated). There are some methods for working with collections a sets that may be of interest. + +Consider this vector: + +```{julia} +#| eval: false +ps = [2, 3, 5, 7, 11, 13, 17] +``` + +What does this expression test? + +```{julia} +#| eval: false +5 in ps +``` + +```{julia} +#| echo: false +choices = ["It tests inclusion---is `5` in the collection", + "It errors"] +answer = 1 +buttonq(choices, answer) +``` + +Now consider this additional vector + +```{julia} +#| eval: false +odds = [1, 3, 5, 7, 9, 11, 13, 15, 17, 19] +``` + +What does this command compute? + +```{julia} +#| eval: false +intersect(odds, ps) +``` + +```{julia} +#| echo: false +choices = ["It finds the *intersection* of the two vectors as sets", + "It finds the *union* of the the two vectors as sets", + "It finds the *set difference* of the the two vectors as sets"] +answer = 1 +buttonq(choices, answer) +``` + +(There are also `union` and `setdiff` methods along with `intersect`.) diff --git a/quarto/curves/early-curvature.qmd b/quarto/curves/early-curvature.qmd new file mode 100644 index 0000000..637c169 --- /dev/null +++ b/quarto/curves/early-curvature.qmd @@ -0,0 +1,161 @@ +XXX Was with linearization, but now not +XXX This is failing with current sympy not taking limits the same way + + +## Curvature + +The curvature of a function will be a topic in a later section on differentiable vector calculus, but the concept of linearization can be used to give an earlier introduction. + + +The tangent line linearizes the function, it being the best linear approximation to the graph of the function at the point. The slope of the tangent line is the limit of the slopes of different secant lines. Consider now, the orthogonal concept, the *normal line* at a point. This is a line perpendicular to the tangent line that goes through the point on the curve. + +At a point $(c,f(c))$ the slope of the normal line is $-1/f'(c)$. + +Following [Kirby C. Smith](https://doi.org/10.2307/2687102), consider two nearby points on the curve of $f$ and suppose we take the two normal lines at $x=c$ and $x=c+h$. These two curves will intersect if the lines are not parallel. To ensure this, assueme that in some neighborhood of $c$, $f'(c)$ is increasing. + +The two normal lines are: + +$$ +\begin{align*} +y &= f(c) - \frac{1}{f'(c)}(x-c)\\ +y &= f(c+h) - \frac{1}{f'(c+h)}(x-(c+h))\\ +\end{align*} +$$ + +Rearranging, we have + +$$ +\begin{align*} +-f'(c)(y-f(c)) &= x-c\\ +-f'(c+h)(y-f(c+h)) &= x-(c+h) +\end{align*} +$$ + + +Call $R$ the intersection point of the two normal lines, as illustrated in @fig-intersection-of-two-normal-lines-x-4. + +::: {#fig-intersection-of-two-normal-lines-x-4} +```{julia} +#| echo: false +using Roots +let + gr() + f(x) = x^4 + fp(x) = 4x^3 + c = 1/4 + h = 1/4 + nlc(x) = f(c) - 1/fp(c) * (x - c) + nlch(x) = f(c+h) - 1/fp(c+h) * (x-(c+h)) + canvas() = plot(axis=([],false), legend=false, aspect_ratio=:equal) + plt = canvas() + plot!(plt, f, 0, 3/4; line=(3,)) + plot!(plt, nlc; ylim=(-1/4, 1)) + plot!(plt, nlch; ylim=(-1/4, 1)) + Rx = find_zero(x -> nlc(x) - nlch(x), (-10, 10)) + scatter!(plt, [c,c+h], f.([c, c+h])) + scatter!(plt, [Rx], [nlc(Rx)]) + annotate!(plt, [(c, f(c), L"(c,f(c))",:top), + (c+h, f(c+h), L"(c+h, f(c+h))",:bottom), + (Rx, nlc(Rx), L"R",:left)]) + plotly() + plt +end +``` + +$R$ is intersection point of two normal lines. +::: + + +What happens to $R$ as $h \rightarrow 0$? + +We can symbolically solve to see: + +```{julia} +@syms 𝑓() 𝑓ₚ() 𝑓ₚₚ() x y c ℎ +n1 = -𝑓ₚ(c)*(y-𝑓(c)) ~ x - c +n2 = -𝑓ₚ(c+ℎ)*(y-𝑓(c+ℎ)) ~ x - (c+ℎ) +R = solve((n1, n2), (x, y)) +``` + + +Taking limits of each term as $h$ goes to zero we have after some notation-simplifying substitution: + +```{julia} +R = Dict(k => limit(v, ℎ=>0) for (k,v) in R) +Rx = R[x](limit((𝑓(c+ℎ)-𝑓(c))/ℎ, ℎ=>0) => 𝑓ₚ(c), + limit((𝑓ₚ(c+ℎ)-𝑓ₚ(c))/ℎ, ℎ=>0) => 𝑓ₚₚ(c)) +``` + + +and + +```{julia} +Ry = R[y](limit((𝑓(c+ℎ)-𝑓(c))/ℎ, ℎ=>0) => 𝑓ₚ(c), + limit((𝑓ₚ(c+ℎ)-𝑓ₚ(c))/ℎ, ℎ=>0) => 𝑓ₚₚ(c)) +``` + +The squared distance, $r^2$, of $R$ to $(c,f(c))$ is then: + +```{julia} +simplify((Rx-c)^2 + (Ry-𝑓(c))^2) +``` + +Or + +$$ +r^2 = \frac{(f'(c)^2 + 1)^3}{f''(c)^2}. +$$ + + +This formula for $r$ is known as the radius of curvature of $f$ -- the radius of the *circle* that best approximates the function at the point. That is, this value reflects the curvature of $f$ supplementing the tangent line or best *linear* approximation to the graph of $f$ at the point. + +::: {#fig-radius-of-curvature-is-R} +```{julia} +#| echo: false +let + gr() + f(x) = x^4 + fp(x) = 4x^3 + fpp(x) = 12x^2 + c = 1/4 + h = 1/4 + nlc(x) = f(c) - 1/fp(c) * (x - c) + nlch(x) = f(c+h) - 1/fp(c+h) * (x-(c+h)) + canvas() = plot(axis=([],false), legend=false, aspect_ratio=:equal) + plt = canvas() + plot!(plt, f, -1/4, 3/4; line=(3,)) + tl(x) = f(c) + f'(c)*(x-c) + plot!(plt, tl, ylim=(-1/4, 3/2); line=(2, :dot)) + + + Rx, Ry = c - fp(c)^3 / fpp(c) - fp(c)/fpp(c), f(c) + (fp(c)^2+1)/fpp(c) + r = (fp(c)^2 + 1)^(3/2) / abs(fpp(c)) + + scatter!(plt, [c], f.([c])) + scatter!(plt, [Rx], [nlc(Rx)]) + annotate!(plt, [(c, f(c), L"(c,f(c))",:top), + (Rx, nlc(Rx), L"R",:left)]) + + + Delta = pi/10 + theta = range(3pi/2 - Delta, 2pi - 3Delta, length=100) + xs, ys = cos.(theta), sin.(theta) + + + plot!(Rx .+ r.*xs, Ry .+ r.*ys) + + x0s, y0s = [Rx,Rx .+ r * first(xs)],[Ry,Ry .+ r * first(ys)] + xns, yns = [Rx,Rx .+ r * last(xs)],[Ry,Ry .+ r * last(ys)] + xcs, ycs = [Rx,c],[Ry,f(c)] + sty = (2, :0.25, :dash) + plot!(plt, x0s, y0s; line=sty); + plot!(plt, xcs, ycs; line=sty); + plot!(plt, xns, yns; line=sty) + + plotly() + plt +end +``` + +Illustration of radius of curvature +::: diff --git a/quarto/derivatives/curve_sketching.qmd b/quarto/derivatives/curve_sketching.qmd index 34a93a2..e9aa229 100644 --- a/quarto/derivatives/curve_sketching.qmd +++ b/quarto/derivatives/curve_sketching.qmd @@ -19,21 +19,26 @@ import Polynomials: variable, Polynomial, RationalFunction # avoid name clash --- -The figure illustrates a means to *sketch* a sine curve - identify as many of the following values as you can: +Drawing a sketch of a function can be enhanced by identifying as many of the following as possible before filling in a curve representing the function: + +* asymptotic behaviour (as $x \rightarrow \pm \infty$), + +* periodic behaviour, + +* vertical asymptotes, + +* the $y$ intercept, + +* any $x$ intercept(s), + +* local peaks and valleys (relative extrema). + +* concavity - * asymptotic behaviour (as $x \rightarrow \pm \infty$), - * periodic behaviour, - * vertical asymptotes, - * the $y$ intercept, - * any $x$ intercept(s), - * local peaks and valleys (relative extrema). - * concavity - - -With these, a sketch fills in between the points/lines associated with these values. - +@fig-animation-sketch-sin-plot and @fig-animation-sketch-rational-function-plot illustrate this approach for two different types of functions. +::: {#fig-animation-sketch-sin-plot} ```{julia} #| hold: true #| echo: false @@ -69,15 +74,7 @@ function sketch_sin_plot_graph(i) end -caption = L""" - -After identifying asymptotic behaviours, -a curve sketch involves identifying the $y$ intercept, if applicable; the $x$ intercepts, if possible; the local extrema; and changes in concavity. From there a sketch fills in between the points. In this example, the periodic function $f(x) = 10\cdot\sin(\pi/2\cdot x)$ is sketched over $[0,4]$. - -""" - - - +caption = "" n = 8 anim = @animate for i=1:n sketch_sin_plot_graph(i) @@ -89,6 +86,101 @@ plotly() ImageFile(imgfile, caption) ``` +After identifying asymptotic behaviors, a curve sketch involves identifying the $y$ intercept, if applicable; the $x$ intercepts, if possible; the local extrema; and changes in concavity. From there a sketch fills in between the points. In this example, the periodic function $f(x) = 10\cdot\sin(\pi/2\cdot x)$ is sketched over $[0,4]$. +::: + + + +::: {#fig-animation-sketch-rational-function-plot} +```{julia} +#| echo: false +#| cache: true +### {{{ sketch_sin_plot }}} +let + gr() + + function sketch_rational_plot_graph(i) + f(x) = 1/2 + (x-1)^2/((x-2)^2 * (x+1)) + + plt = plot(; xlim=(-4, 5), ylim=(-5, 5), xticks = -4:5, legend=false, framestyle=:origin) + + title = "Sketch a rational function" + # (1) asymptotic behaviour (as $x \rightarrow \pm \infty$), + i == 1 && (title = L"A horizontal asymptote $y=1/2$") + if i >= 1 + hline!(plt, [1/2]; line=(:dash, :gray50)) + end + + # (2) periodic behaviour, + i == 2 && (title = "No periodic behavious") + if i >= 2 + + end + + # (3) vertical asymptotes, + i == 3 && (title = L"Vertical asyptotes at $x=-1$ and $x=2$") + if i >= 3 + vline!(plt, [-1, 2]; line=(:dash, :gray50)) + end + + # (4) the $y$ intercept, + i == 4 && (title = L"f(0) = 3/4") + if i >= 4 + scatter!(plt, [(0, f(0))]; marker=(5, :green)) + end + + # (5) any $x$ intercept(s) + i == 5 && (title = L"Zero at $x=-2.1527576\cdots$") + if i >= 5 + z = -2.1527576 + scatter!(plt, [(z, 0)]; marker=(5, :red)) + end + + # (6) local peaks and valleys (relative extrema). + i == 6 && (title = L"Relative minimum at $(1, f(1))$") + if i >= 6 + scatter!(plt, [(1,f(1))]; marker = (5, :blue)) + end + + # (7) concavity + i == 7 && (title = L"Concave up on $(-1, 2)$, $(2, 3)$" ) + if i == 7 + xs = range(-0.4, 0.4, 100) + ys = 1 .+ sec.(xs.* pi) + plot!(xs .- 2.5, -1 .* ys; line=(3, 0.25, :blue), arrow=true, side=:right) + plot!(xs .+ 1.0, 1 .* ys; line=(3, 0.25, :blue), arrow=true, side=:right) + plot!(xs .+ 3.5, 1 .* ys; line=(3, 0.25, :blue), arrow=true, side=:right) + + end + + # connect the dots + if i >= 8 + title = "Sketch" + plot!(plt, rangeclamp(f, 5); line = (1, :black)) + end + title!(plt, title) + plt + + end + + caption = "" + n = 10 + anim = @animate for i=1:n + sketch_rational_plot_graph(i) + end + + imgfile = tempname() * ".gif" + gif(anim, imgfile, fps = 1) + plotly() + + ImageFile(imgfile, caption) +end +``` + +After identifying asymptotic behaviors, a curve sketch involves identifying the $y$ intercept, if applicable; the $x$ intercepts, if possible; the local extrema; and changes in concavity. From there a sketch fills in between the points. In this example, the rational function $f(x) = 1/2 + (x-1)^2/((x-2)^2 \cdot (x+1))$ is sketched. +::: + + Though this approach is most useful for hand-sketches, the underlying concepts are important for properly framing graphs made with the computer. @@ -148,42 +240,52 @@ We finally check that if we were to just use $[0,7]$ as a domain to plot over th f.([0, cps..., 7]) ``` -The values at $0$ and at $7$ are a bit large, as compared to the relative extrema, and since we know the graph is eventually $U$-shaped, this offers no insight. So we narrow the range a bit for the graph: - +The values at $0$ and at $7$ are a bit large, as compared to the relative extrema, and since we know the graph is eventually $U$-shaped, this offers no insight. So we narrow the range a bit for the graph of @fig-plot-poly-x-4-minus-13-x-3-plus-56-x-2-minus92-x-plus-48. +::: {#fig-plot-poly-x-4-minus-13-x-3-plus-56-x-2-minus92-x-plus-48} ```{julia} plot(f, 0.5, 6.5) ``` +Plot of $f(x) = x^4 - 13x^3 + 56x^2 -92x + 48$ over $[0.5, 6.5]$ +::: + --- This sort of analysis can be automated. The plot "recipe" for polynomials from the `Polynomials` package does similar considerations to choose a viewing window: - +::: {#fig-plot-polynomial-f-of-x-not-f} ```{julia} 𝐱 = variable(Polynomial) plot(f(𝐱)) # f(𝐱) of Polynomial type ``` +Plot of polynomial `f(x)` produced by a plot recipe +::: + ##### Example -Graph the function +Graph the rational function $$ f(x) = \frac{(x-1)\cdot(x-3)^2}{x \cdot (x-2)}. $$ -Not much to do here if you are satisfied with a graph that only gives insight into the asymptotes of this rational function: - +Not much to do here if you are satisfied with a graph that only gives insight into the asymptotes of this rational function (@fig-plot-x-minus-1-times-x-minus-3-squared-over-x-over-x-minus-2-bad-one). +::: {#fig-plot-x-minus-1-times-x-minus-3-squared-over-x-over-x-minus-2-bad-one} ```{julia} f(x) = ( (x-1)*(x-3)^2 ) / (x * (x-2) ) plot(f, -50, 50) ``` +Plot of rational function over $[-50, 50] shows some features, but not all that are commonly of interest +::: + + We can see the slant asymptote and hints of vertical asymptotes, but, we'd like to see more of the basic features of the graph. @@ -204,12 +306,16 @@ So a range over $[-5,5]$ should display the key features including the slant asy Previously we used the `rangeclamp` function defined in `CalculusWithJulia` to avoid the distortion that vertical asymptotes can have: - +::: {#fig-plot-x-minus-1-times-x-minus-3-squared-over-x-over-x-minus-2-after-settling-on-viewing-window} ```{julia} -plot(rangeclamp(f), -5, 5) +#| echo: false +plot(rangeclamp(f), -5, 5; legend=false) ``` -With this graphic, we can now clearly see in the graph the two zeros at $x=1$ and $x=3$, the vertical asymptotes at $x=0$ and $x=2$, and the slant asymptote. +Plot of rational function over identified domain $[-5, 5]$ and passing through `rangeclamp` to avoid spurious lines +::: + +With the graph in @fig-plot-x-minus-1-times-x-minus-3-squared-over-x-over-x-minus-2-after-settling-on-viewing-window, we can now clearly see in the graph the two zeros at $x=1$ and $x=3$, the vertical asymptotes at $x=0$ and $x=2$, and the slant asymptote. --- @@ -217,33 +323,42 @@ With this graphic, we can now clearly see in the graph the two zeros at $x=1$ an Again, this sort of analysis can be systematized. The rational function type in the `Polynomials` package takes a stab at that, but isn't quite so good at capturing the slant asymptote: - +::: {#fig-plot-of-rational-function-using-plot-recipe} ```{julia} -𝐱 = variable(RationalFunction) -plot(f(𝐱)) # f(x) of RationalFunction type +x = variable(RationalFunction) +plot(f(x)) # f(x) of RationalFunction type ``` +Plot of rational function using a plot recipe +::: + ##### Example Consider the function $V(t) = 170 \sin(2\pi\cdot 60 \cdot t)$, a model for the alternating current waveform for an outlet in the United States. Create a graph. -Blindly trying to graph this, we will see immediate issues: - +Blindly trying to graph this (@fig-plot-of-170-sin-2-pi-60-t-oscillates), we will see immediate issues. +::: {#fig-plot-of-170-sin-2-pi-60-t-oscillates} ```{julia} V(t) = 170 * sin(2*pi*60*t) plot(V, -2pi, 2pi) ``` +Plot of current function, $V(t)$ showing it is highly oscillatory +::: + Ahh, this periodic function is *too* rapidly oscillating to be plotted without care. We recognize this as being of the form $V(t) = a\cdot\sin(c\cdot t)$, so where the sine function has a period of $2\pi$, this will have a period of $2\pi/c$, or $1/60$. So instead of using $(-2\pi, 2\pi)$ as the interval to plot over, we need something much smaller: - +::: {#fig-plot-of-170-sin-2-pi-60-t-two-periods} ```{julia} plot(V, -1/60, 1/60) ``` +Plot of two periods of highly oscillatory function, $V(t)$ +::: + ##### Example @@ -262,10 +377,13 @@ limit(ex, x=>0, dir="+"), limit(ex, x=>oo) The $\ln(x/100)$ part of $f$ goes $-\infty$ as $x \rightarrow 0+$; yet $f(x)$ is eventually positive as $x \rightarrow \infty$. So a graph should - * not show too much of the vertical asymptote - * capture the point where $f(x)$ must cross $0$ - * capture the point where $f(x)$ has a relative maximum - * show enough past this maximum to indicate to the reader the eventual horizontal asymptote. +* not show too much of the vertical asymptote + +* capture the point where $f(x)$ must cross $0$ + +* capture the point where $f(x)$ has a relative maximum + +* show enough past this maximum to indicate to the reader the eventual horizontal asymptote. For that, we need to get the $x$ intercepts and the critical points. The $x/100$ means this graph has some scaling to it, so we first look between $0$ and $200$: @@ -282,36 +400,44 @@ Trying the same for the critical points comes up empty. We know there is one, bu find_zeros(diff(ex,x), 0, 500) ``` -So maybe graphing over $[50, 300]$ will be a good start: - +So maybe graphing over $[50, 300]$ will be a good start. A graph is shown in @fig-plot-log-x-over-100-over-x-take-1. +::: {#fig-plot-log-x-over-100-over-x-take-1} ```{julia} +#| echo: false plot(ex, 50, 300) ``` -But it isn't! The function takes its time getting back towards $0$. We know that there must be a change of concavity as $x \rightarrow \infty$, as there is a horizontal asymptote. We looks for the anticipated inflection point to ensure our graph includes that: +Plot of $f(x) = \ln(x/100)/x$ over $[50, 200]$ +::: + +But it isn't! The function takes its time getting back towards $0$. We know that there must be a change of concavity as $x \rightarrow \infty$, as there is a horizontal asymptote. We look for the anticipated inflection point to ensure our graph includes that: ```{julia} find_zeros(diff(ex, x, x), 1, 5000) ``` -So a better plot is found by going well beyond that inflection point: - +So a better plot is found by going well beyond that inflection point. In @fig-plot-log-x-over-100-over-x-take-1 we plot over $[75, 1500]$. +::: {#fig-plot-log-x-over-100-over-x-take-2} ```{julia} +#| echo: false plot(ex, 75, 1500) ``` +Plot of $f(x) = \ln(x/100)/x$ over $[75, 1500]$ +::: + ## Questions ###### Question -Consider this graph - +Consider the graph of a rational function in @fig-rational-function-x-minus-2-times-x-2point5-times-x-minus-3-over-x-minus-1-x-plus-1. +::: {#fig-rational-function-x-minus-2-times-x-2point5-times-x-minus-3-over-x-minus-1-x-plus-1} ```{julia} #| hold: true #| echo: false @@ -321,6 +447,9 @@ plot!(p, f, -1 + .2, 1 - .02, color=:blue) plot!(p, f, 1 + .05, 20, color=:blue) ``` +Graph of rational function +::: + What kind of *asymptotes* does it appear to have? @@ -333,8 +462,8 @@ L"Just vertical asymptotes at $x=-1$ and $x=1$", L"Vertical asymptotes at $x=-1$ and $x=1$ and a horizontal asymptote $y=1$", L"Vertical asymptotes at $x=-1$ and $x=1$ and a slant asymptote" ] -answ = 4 -radioq(choices, answ) +answer = 4 +radioq(choices, answer) ``` ###### Question @@ -448,15 +577,16 @@ radioq(qchoices, 7, keep_order=true) A function $f$ has +* zeros of $\{-0.7548\dots, 2.0\}$, - * zeros of $\{-0.7548\dots, 2.0\}$, - * critical points at $\{-0.17539\dots, 1.0, 1.42539\dots\}$, - * inflection points at $\{0.2712\dots,1.2287\}$. +* critical points at $\{-0.17539\dots, 1.0, 1.42539\dots\}$, + +* inflection points at $\{0.2712\dots,1.2287\}$. -Is this a possible graph of $f$? - +Does @fig-function-x-4-minus-3-x-3-plus-2-x-2-plus-x-minus-2 show a possible graph of $f$? +::: {#fig-function-x-4-minus-3-x-3-plus-2-x-2-plus-x-minus-2} ```{julia} #| hold: true #| echo: false @@ -464,6 +594,9 @@ f(x) = x^4 - 3x^3 + 2x^2 + x - 2 plot(f, -1, 2.5, legend=false) ``` +Plot of $f(x)$ over $[-1, 1.25]$ +::: + ```{julia} #| hold: true #| echo: false @@ -476,19 +609,22 @@ yesnoq("yes") Two models for population growth are *exponential* growth: $P(t) = P_0 a^t$ and [logistic growth](https://en.wikipedia.org/wiki/Logistic_function#In_ecology:_modeling_population_growth): $P(t) = K P_0 a^t / (K + P_0(a^t - 1))$. The exponential growth model has growth rate proportional to the current population. The logistic model has growth rate depending on the current population *and* the available resources (which can limit growth). -Letting $K=50$, $P_0=5$, and $a= e^{1/4}$. A plot over $[0,5]$ shows somewhat similar behaviour: - +Letting $K=50$, $P_0=5$, and $a= e^{1/4}$, the plots in @fig-plot-exponential-and-logistic-growth-over-0-5 $ shows somewhat similar behaviour over $[0,5]. +::: {#fig-plot-exponential-and-logistic-growth-over-0-5} ```{julia} K, P0, a = 50, 5, exp(1/4) exponential_growth(t) = P0 * a^t logistic_growth(t) = K * P0 * a^t / (K + P0*(a^t-1)) -plot(exponential_growth, 0, 5) -plot!(logistic_growth) +plot(exponential_growth, 0, 5; label="exponential growth") +plot!(logistic_growth; label="logistic growth") ``` -Does a plot over $[0,50]$ show qualitatively similar behaviour? +Plot of exponential growth model and logisitc growth model over $[0,5]$ +::: + +Does a plot over $[0,50]$ show qualitatively similar behaviour? ```{julia} @@ -526,8 +662,8 @@ choices = [ "The exponential growth model", "The limit does not exist", "The limit is ``P_0``"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -547,8 +683,8 @@ choices = [ "The function will have more curvature when the second derivative is large, so there needs to be more points to capture the shape", "The function will be much larger (in absolute value) when the second derivative is large, so there needs to be more points to capture the shape", ] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question @@ -567,8 +703,8 @@ choices = [ "An informative graph only needs to show one or two periods, as others can be inferred.", "An informative graph need only show a part of the period, as the rest can be inferred.", L"An informative graph needs to show several periods, as that will allow proper computation for the $y$ axis range."] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` Why should asymptotics matter? @@ -582,8 +718,8 @@ L"A vertical asymptote can distort the $y$ range, so it is important to avoid to L"A horizontal asymptote must be plotted from $-\infty$ to $\infty$", "A slant asymptote must be plotted over a very wide domain so that it can be identified." ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` Monotonicity means increasing or decreasing. This is important for what reason? @@ -597,6 +733,6 @@ choices = [ "For monotonic regions, a function is basically a straight line", "For monotonic regions, the function will have a vertical asymptote, so the region should not be plotted" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` diff --git a/quarto/derivatives/derivatives.qmd b/quarto/derivatives/derivatives.qmd index d64b1f0..58c1f15 100644 --- a/quarto/derivatives/derivatives.qmd +++ b/quarto/derivatives/derivatives.qmd @@ -22,7 +22,11 @@ nothing --- -![Device to measure units of distance by units of time](figures/galileo-ramp.png){width=60%} +::: {#fig-picture-of-galileo-ramp} +![](figures/galileo-ramp.png){width=60%} + +Device to measure units of distance by units of time +::: Before defining the derivative of a function, let's begin with two motivating examples. @@ -47,14 +51,17 @@ If the rate is a constant $60$ miles/hour, then in one hour the distance travele Of course, the odometer isn't just incrementing once per hour, it is incrementing once every $1/10$th of a mile. How much time does that take? Well, we would need to solve $1/10=60 \cdot t$ which means $t=1/600$ hours, better known as once every $6$ seconds. -Using some mathematical notation, would give $x(t) = v\cdot t$, where $x$ is position at time $t$, $v$ is the *constant* velocity and $t$ the time traveled in hours. A simple graph of the first three hours of travel would show: +Using some mathematical notation, would give $x(t) = v\cdot t$, where $x$ is position at time $t$, $v$ is the *constant* velocity and $t$ the time traveled in hours. The simple graph of @fig-plot-position-time-over-0-3 shows the first three hours of travel. +::: {#fig-plot-position-time-over-0-3} ```{julia} #| hold: true position(t) = 60 * t plot(position, 0, 3) ``` +Plot of position versus time for a constant speed +::: Oh no, we hit traffic. In the next $30$ minutes we only traveled $15$ miles. We were so busy looking out for traffic, the speedometer was not checked. What would the average speed have been? Though in the $30$ minutes of stop-and-go traffic, the displayed speed may have varied, the *average speed* would simply be the change in distance over the change in time, or $\Delta x / \Delta t$. That is @@ -63,9 +70,9 @@ Oh no, we hit traffic. In the next $30$ minutes we only traveled $15$ miles. We 15/(1/2) ``` -Now suppose that after $6$ hours of travel the GPS in the car gives us a readout of distance traveled as a function of time. The graph looks like this: - +Now suppose that after $6$ hours of travel the GPS in the car gives us a readout of distance traveled as a function of time. @fig-plot-complicated-position-versus-time-from-0-to-6 shows the graph. +::: {#fig-plot-complicated-position-versus-time-from-0-to-6} ```{julia} #| hold: true #| echo: false @@ -78,6 +85,9 @@ end plot(position, 0, 6) ``` +Plot of position versus time for a certain non constant speed +::: + We can see with some effort that the slope is steady for the first three hours, is slightly less between $3$ and $3.5$ hours, then is a bit steeper for the next half hour. After that, it is flat for the about half an hour, then the slope continues on with same value as in the first $3$ hours. What does that say about our speed during our trip? @@ -108,9 +118,9 @@ Okay, so there was some speeding involved. The next half hour the car did not move. What was the average speed? Well the change in position was $0$, but the time was $1/2$ hour, so the average was $0$. -Perhaps a graph of the speed is a bit more clear. We can do this based on the above: - +Perhaps a graph of the speed is a bit more clear. We can do this based on the above in @fig-plot-non-constant-speed-over-0-6. +::: {#fig-plot-non-constant-speed-over-0-6} ```{julia} function speed(t) 0 < t <= 3 ? 60 : @@ -121,6 +131,10 @@ end plot(speed, 0, 6) ``` +Plot of non-constant speed over $[0,6]$ +::: + + The jumps, as discussed before, are artifacts of the graphing algorithm. What is interesting, is we could have derived the graph of `speed` from that of `x` by just finding the slopes of the line segments, and we could have derived the graph of `x` from that of `speed`, just using the simple formula relating distance, rate, and time. @@ -133,42 +147,46 @@ We were pretty loose with some key terms. There is a distinction between "speed" ##### Example: Galileo's ball and ramp experiment -One of history's most famous experiments was performed by [Galileo](http://en.wikipedia.org/wiki/History_of_experiments) where he rolled balls down inclined ramps, making note of distance traveled with respect to time. As Galileo had no ultra-accurate measuring device, he needed to slow movement down by controlling the angle of the ramp. With this, he could measure units of distance per units of time. (Click through to *Galileo and Perspective* [Dauben](http://www.mcm.edu/academic/galileo/ars/arshtml/mathofmotion1.html).) +One of history's most famous experiments was performed by [Galileo](http://en.wikipedia.org/wiki/History_of_experiments) where he rolled balls down inclined ramps, like that in @fig-picture-of-galileo-ramp, making note of distance traveled with respect to time. As Galileo had no ultra-accurate measuring device, he needed to slow movement down by controlling the angle of the ramp. With this, he could measure units of distance per units of time. +Suppose that no matter what the incline was, Galileo observed that in units of the distance traveled in the first second that the distance traveled between subsequent seconds was $3$ times, then $5$ times, then $7$ times, ... @tbl-galileo-distance-time-delta summarizes. -Suppose that no matter what the incline was, Galileo observed that in units of the distance traveled in the first second that the distance traveled between subsequent seconds was $3$ times, then $5$ times, then $7$ times, ... This table summarizes. +::: {#tbl-galileo-distance-time-delta .striped .hover} +| t | distance | delta | +|:------|:--------|:---------| +| 0 | 0 | . | +| 1 | 1 | 1 | +| 2 | 4 | 3 | +| 3 | 9 | 5 | +| 4 | 16 | 7 | +| 5 | 25 | 9 | +Distance traveled by time $t$ with differences computed +::: -```{julia} -#| hold: true -#| echo: false -ts = [0,1,2,3,4,5] -dxs = [0,1,3, 5, 7, 9] -ds = [0,1,4,9,16,25] -d = DataFrame(t=ts, delta=dxs, distance=ds) -table(d) -``` - -A graph of distance versus time could be found by interpolating between the measured points: - +The graph of distance versus time in @fig-plot-of-distance-versus-time-galileo-simple-data is found by interpolating between the measured points. +::: {#fig-plot-of-distance-versus-time-galileo-simple-data} ```{julia} ts = [0,1,2,3,4, 5] xs = [0,1,4,9,16,25] plot(ts, xs) ``` +Plot of distance traveled versus time +::: + The graph looks almost quadratic. What would the following questions have yielded? - * What is the average speed between $0$ and $3$? +* What is the average speed between $0$ and $3$? ```{julia} (9-0) / (3-0) # (xs[4] - xs[1]) / (ts[4] - ts[1]) ``` - * What is the average speed between $2$ and $3$? +* What is the average speed between $2$ and $3$? ```{julia} @@ -188,7 +206,7 @@ xs[2]-xs[1], xs[3] - xs[2], xs[4] - xs[3], xs[5] - xs[4] We see it increments by $2$. The acceleration is the rate of change of speed. We see the rate of change of speed is constant, as the speed increments by $2$ each time unit. -Based on this - and given Galileo's insight - it appears the acceleration for a falling body subject to gravity will be **constant** and the position as a function of time will be quadratic. +Based on this---and given Galileo's insight---it appears the acceleration for a falling body subject to gravity will be **constant** and the position as a function of time will be quadratic. ## The slope of the secant line @@ -196,27 +214,35 @@ Based on this - and given Galileo's insight - it appears the acceleration for a In the above examples, we see that the average speed is computed using the slope formula. This can be generalized for any univariate function $f(x)$: +::: {.definition title="Average rate of change"} -> The average rate of change between $a$ and $b$ is $(f(b) - f(a)) / (b - a)$. It is typical to express this as $\Delta y/ \Delta x$, where $\Delta$ means "change". - - - -Geometrically, this is the slope of the line connecting the points $(a, f(a))$ and $(b, f(b))$. This line is called a [secant](http://en.wikipedia.org/wiki/Secant_line) line, which is just a line intersecting two specified points on a curve. - - -Rather than parameterize this problem using $a$ and $b$, we let $c$ and $c+h$ represent the two values for $x$, then the secant-line-slope formula becomes - +For a continuous function $f$ the average rate of change between $a$ and $b$ is defined by $$ -m = \frac{f(c+h) - f(c)}{h}. +\frac{f(b) - f(a)}{b - a}. $$ +It is typical to express this as $\Delta y/ \Delta x$, where $\Delta$ means "change". +::: + + +Geometrically, the average rate of change is the slope of the line connecting the points $(a, f(a))$ and $(b, f(b))$. This line is called a [secant](http://en.wikipedia.org/wiki/Secant_line) line---a line intersecting two specified points on a curve. + + ## The slope of the tangent line +Before continuing, we re-parameterize with $a=c$ and $b=c+h$ thinking of $c$ as fixed, and $h$ as varying. The secant-line-slope formula becomes: + + +$$ +\text{slope (of a secant line)} = \frac{f(c+h) - f(c)}{h}. +$$ + + The slope of the secant line represents the average rate of change over a given period, $h$. What if this rate is so variable, that it makes sense to take smaller and smaller periods $h$? In fact, what if $h$ goes to $0$? - +::: {#fig-secant-line-tangent-line-animation} ```{julia} #| hold: true #| echo: false @@ -261,14 +287,7 @@ function secant_line_tangent_line_graph(n) plt end -caption = L""" - -The slope of each secant line represents the *average* rate of change between $c$ and $c+h$. As $h$ goes towards $0$, we recover the slope of the tangent line, which represents the *instantatneous* rate of change. - -""" - - - +caption = "" n = 6 anim = @animate for i=0:n secant_line_tangent_line_graph(i) @@ -281,21 +300,41 @@ plotly() ImageFile(imgfile, caption) ``` -The graphic suggests that the slopes of the secant line converge to the slope of a "tangent" line. That is, for a given $c$, this limit exists: +The slope of each secant line represents the *average* rate of change between $c$ and $c+h$. As $h$ goes towards $0$, we recover the slope of the tangent line, which represents the *instantatneous* rate of change. +::: + + +@fig-secant-line-tangent-line-animation suggests that for this function and at the point $x=c$ the slopes of the secant line converge to the slope of a "tangent" line. That is this limit exists: $$ \lim_{h \rightarrow 0} \frac{f(c+h) - f(c)}{h}. $$ -We will define the tangent line at $(c, f(c))$ to be the line through the point with the slope from the limit above - provided that limit exists. Informally, the tangent line is the line through the point that best approximates the function. +With this, we define + +::: {.definition title="Tangent line at c"} + +When the following limit exists, the tangent line to the graph of $f(x)$ at $x=c$ is the line through the point $(c, f(c))$ with slope: + +$$ +m = \lim_{h \rightarrow 0} \frac{f(c+h) - f(c)}{h}. +$$ + +In point-slope form, the line is described by: $y = f(c) + m \cdot (x-c)$. + +Later we will write $f'(c)$ for $m$. +::: + + +Informally, the tangent line is the line through the point that best approximates the function, as in @fig-tangent_line_approx_graph. The tangent line is not just a line that intersects the graph in one point, nor does it need only intersect the line in just one point. ::: {#fig-tangent_line_approx_graph} ```{julia} #| echo: false -gr() let + gr() function make_plot(Δ) f(x) = 1 + sin(x-c) df(x) = cos(x-c) @@ -322,61 +361,16 @@ let end ps = make_plot.((1.5, 1.0, 0.5, 0.1)) + + plotly() plot(ps...) - - end ``` -Illustration that the tangent line is the best linear approximation *near* $c$. +Illustration that the tangent line is the best linear approximation to $f(x)$ *near* $c$. ::: -```{julia} -#| echo: false -plotly() -nothing -``` - -```{julia} -#| hold: true -#| echo: false -#| cache: true -#| eval: false -gr() -function line_approx_fn_graph(n) - f(x) = sin(x) - c = pi/3 - h = round(2.0^(-n) * pi/2, digits=2) - m = cos(c) - - Delta = max(f(c) - f(c-h), f(min(c+h, pi/2)) - f(c)) - - p = plot(f, c-h, c+h, legend=false, xlims=(c-h,c+h), ylims=(f(c)-Delta,f(c)+Delta )) - plot!(p, x -> f(c) + m*(x-c)) - scatter!(p, [c], [f(c)]) - p -end -caption = L""" - -The tangent line is the best linear approximation to the function at the point $(c, f(c))$. As the viewing window zooms in on $(c,f(c))$ we - can see how the graph and its tangent line get more similar. - -""" - -n = 6 -anim = @animate for i=1:n - line_approx_fn_graph(i) -end - -imgfile = tempname() * ".gif" -gif(anim, imgfile, fps = 1) - -plotly() -ImageFile(imgfile, caption) -``` - -The tangent line is not just a line that intersects the graph in one point, nor does it need only intersect the line in just one point. :::{.callout-note} @@ -391,41 +385,47 @@ This last point was certainly not obvious at first. [Barrow](http://www.maa.org/ What is the slope of the tangent line to $f(x) = \sin(x)$ at $c=0$? -We need to compute the limit $(\sin(c+h) - \sin(c))/h$ which is the limit as $h$ goes to $0$ of $\sin(h)/h.$ We know this to be $1.$ - +We need to compute the limit $(\sin(c+h) - \sin(c))/h$ which is the limit as $h$ goes to $0$ of $\sin(h)/h.$ We know this to be $1$. See @fig-plot-sin-tangent-line for a graph of the tangent line to the function. +::: {#fig-plot-sin-tangent-line} ```{julia} #| hold: true +#| echo: false f(x) = sin(x) c = 0 tl(x) = f(c) + 1 * (x - c) -plot(f, -pi/2, pi/2) -plot!(tl, -pi/2, pi/2) +plot(f, -pi/2, pi/2; label="sin(x)") +plot!(tl; label="Tangent line") ``` +Plot of $f(x) = \sin(x)$ with its tangent line ($y = 0 + 1 \cdot (x-0)$) at the origin +::: + ## The derivative -The limit of the slope of the secant line gives an operation: for each $c$ in the domain of $f$ there is a number (the slope of the tangent line) or it does not exist. That is, there is a derived function from $f$. Call this function the *derivative* of $f$. +The limit of the slopes of the secant lines for a fixed $c$ gives an operation: for each $c$ in the domain of $f$ when the limit exists +associate the slope of the tangent line. +::: {.definition title="The derivative"} -There are many notations for the derivative, mostly we use the "prime" notation: - +For each $x$ in the domain of $f(x)$, let $f'(x)$ define a function whose domain is all $x$ for which the following limit exists and whose value is given by: $$ -f'(x) = \lim_{h \rightarrow 0} \frac{f(x+h) - f(x)}{h}. +f'(x) = \lim_{h\rightarrow 0} \frac{f(x+h) - f(x)}{h}. $$ -The limit above is identical, only it uses $x$ instead of $c$ to emphasize that we are thinking of a function now, and not just a value at a point. +The function $f'(x)$ is called the *derivative* of $f(x)$. +::: - -The derivative is related to a function, but at times it is more convenient to write only the expression defining the rule of the function. In that case, we use this notation for the derivative $[\text{expression}]'$. +There are many notations for the derivative, mostly we use the "prime" notation in the definition, but at times it is more convenient to write only the expression defining the rule of the function. In that case, we use this notation for the derivative $[\text{expression}]'$. ### Some basic derivatives +* **The constant rule**. If $f(x) = c$, some constant, then the graph has zero slope, as $f(x+h) - f(x) = 0$ for all $x$ and $h$. That is the derivative is constantly $0$, or $[c]' = 0$. - * **The power rule**. What is the derivative of the monomial $f(x) = x^n$? We need to look at $(x+h)^n - x^n$ for positive, integer-value $n$. Let's look at a case, $n=5$ +* **The power rule**. What is the derivative of the monomial $f(x) = x^n$? We need to look at $(x+h)^n - x^n$ for positive, integer-value $n$. Let's look at a case, $n=5$ ```{julia} @@ -451,10 +451,10 @@ $$ [x^n]' = nx^{n-1}. $$ -It isn't a special case, but when $n=0$, we also have the above formula applies, as $x^0$ is the constant $1$, and all constant functions will have a derivative of $0$ at all $x$. We will see that in general, the power rule applies for any $n$ where $x^n$ is defined. +It isn't a special case, but when $n=0$, we also have the above formula applies, as $x^0$ is the constant $1$. We will see later that in general, the power rule applies for any $n$ where $x^n$ is defined. - * What is the derivative of $f(x) = \sin(x)$? We know that $f'(0)= 1$ by the earlier example with $(\sin(0+h)-\sin(0))/h = \sin(h)/h$, here we solve in general. +* What is the derivative of $f(x) = \sin(x)$? We know that $f'(0)= 1$ by the earlier example with $(\sin(0+h)-\sin(0))/h = \sin(h)/h$, here we solve in general. We need to consider the difference $\sin(x+h) - \sin(x)$: @@ -464,17 +464,14 @@ We need to consider the difference $\sin(x+h) - \sin(x)$: sympy.expand_trig(sin(x+h) - sin(x)) # expand_trig is not exposed in `SymPy` ``` -That used the formula $\sin(x+h) = \sin(x)\cos(h) + \sin(h)\cos(x)$. - - We could then rearrange the secant line slope formula to become: $$ -\cos(x) \cdot \frac{\sin(h)}{h} + \sin(x) \cdot \frac{\cos(h) - 1}{h} +\cos(x) \cdot \frac{\sin(h)}{h} + \sin(x) \cdot \frac{\cos(h) - 1}{h}. $$ -and take a limit. If the answer isn't clear, we can let `SymPy` do this work: +We then take a limit as $h \rightarrow 0$. If the answer isn't clear, we can let `SymPy` do this work: ```{julia} @@ -484,7 +481,7 @@ limit((sin(x+h) - sin(x))/ h, h => 0) From the formula $[\sin(x)]' = \cos(x)$ we can easily get the *slope* of the tangent line to $f(x) = \sin(x)$ at $x=0$ by simply evaluating $\cos(0) = 1$. - * Let's see what the derivative of $\ln(x) = \log(x)$ is (using base $e$ for $\log$ unless otherwise indicated). We have +* Let's see what the derivative of $\ln(x) = \log(x)$ is (using base $e$ for $\log$ unless otherwise indicated). We have $$ @@ -508,37 +505,39 @@ $$ \frac{e^{x+h} - e^x}{h} = \frac{e^x \cdot(e^h -1)}{h}. $$ -Earlier, we saw that $\lim_{h \rightarrow 0}(e^h - 1)/h = 1$. With this, we get $[e^x]' = e^x$, that is it is a function satisfying $f'=f$. +Earlier, we saw that $\lim_{h \rightarrow 0}(e^h - 1)/h = 1$. With this, we get $[e^x]' = e^x$, that is it is a function satisfying $f'(x)=f(x)$. --- -There are several different [notations](http://en.wikipedia.org/wiki/Notation_for_differentiation) for derivatives. Some are historical, some just add flexibility. We use the prime notation of Lagrange: $f'(x)$, $u'$ and $[\text{expr}]'$, where the first emphasizes that the derivative is a function with a value at $x$, the second emphasizes the derivative operates on functions, the last emphasizes that we are taking the derivative of some expression. +There are several [notations](http://en.wikipedia.org/wiki/Notation_for_differentiation) for derivatives. Some are historical, some just add flexibility. We use the prime notation of Lagrange: $f'(x)$, $u'$ and $[\text{expr}]'$, where the first emphasizes that the derivative is a function with a value at $x$, the second emphasizes the derivative operates on functions, the last emphasizes that we are taking the derivative of some expression. -There are many other notations: +Some other notations include: - * The Leibniz notation uses the infinitesimals: $dy/dx$ to relate to $\Delta y/\Delta x$. This notation is very common, and especially useful when more than one variable is involved. `SymPy` uses Leibniz notation in some of its output, expressing somethings such as: +* The Leibniz notation which uses the infinitesimals, $dy/dx$, to relate to $\Delta y/\Delta x$. This notation is very common, and especially useful when more than one variable is involved. `SymPy` uses Leibniz notation in some of its output, expressing somethings such as: $$ f'(x) = \frac{d}{d\xi}(f(\xi)) \big|_{\xi=x}. $$ -The notation - $\big|$ - on the right-hand side separates the tasks of finding the derivative and evaluating the derivative at a specific value. +The notation--$\big|$---on the right-hand side separates the tasks of finding the derivative and evaluating the derivative at a specific value. - * Euler used `D` for the operator `D(f)`. This was initially used by [Argobast](http://jeff560.tripod.com/calculus.html). The notation `D(f)(c)` would be needed to evaluate the derivative at a point. - * Newton used a "dot" above the variable, $\dot{x}(t)$, which is still widely used in physics to indicate a derivative in time. This indicates first taking the derivative and then plugging in $t$. - * The notation $[expr]'(c)$ or $[expr]'\big|_{x=c}$would similarly mean, take the derivative of the expression and **then** evaluate at $c$. +* Euler used `D` for the operator `D(f)`. This was initially used by [Argobast](http://jeff560.tripod.com/calculus.html). The notation `D(f)(c)` would be needed to evaluate the derivative at a point. + +* Newton used a "dot" above the variable, $\dot{x}(t)$, which is still widely used in physics to indicate a derivative in time. This indicates first taking the derivative and then plugging in $t$. + +* The notation $[expr]'(c)$ or $[expr]'\big|_{x=c}$would similarly mean, take the derivative of the expression and **then** evaluate at $c$. ## Rules of derivatives -We could proceed in a similar manner – using limits to find other derivatives, but let's not. If we have a function $f(x) = x^5 \sin(x)$, it would be nice to leverage our previous work on the derivatives of $f(x) =x^5$ and $g(x) = \sin(x)$, rather than derive an answer from scratch. +We could proceed in a similar manner---using limits to find other derivatives, but let's not. If we have a function $f(x) = x^5 \sin(x)$, it would be nice to leverage our previous work on the derivatives of $f(x) =x^5$ and $g(x) = \sin(x)$, rather than derive an answer from scratch. As with limits and continuity, it proves very useful to consider rules that make the process of finding derivatives of combinations of functions a matter of combining derivatives of the individual functions in some manner. @@ -547,20 +546,20 @@ As with limits and continuity, it proves very useful to consider rules that make We already have one such rule: -### Power rule +::: {.relationship title="Power rule"} - -We have seen for integer $n \geq 0$ the formula: +For integer $n \geq 0$ the power rule is: $$ [x^n]' = n x^{n-1}. $$ -This will be shown true for all real exponents. +The power rule will be shown true for all real exponents. +::: -### Sum rule +We now discuss rules which express derivatives of compound expressions in terms of derivatives of different pieces of an expression. Let's consider $k(x) = a\cdot f(x) + b\cdot g(x)$, what is its derivative? That is, in terms of $f$, $g$ and their derivatives, can we express $k'(x)$? @@ -586,8 +585,17 @@ $$ That is $[a\cdot f(x) + b \cdot g(x)]' = a\cdot f'(x) + b\cdot g'(x)$. +::: {.relationship title="Constant multiple and sum rule"} +The sum rule applies to constant multiple and sums of functions: + +$$ +[a f(x) + b g(x)]' = a f'(x) + b g'(x) +$$ + This holds two rules: the derivative of a constant times a function is the constant times the derivative of the function; and the derivative of a sum of functions is the sum of the derivative of the functions. +::: + This example shows a useful template: @@ -601,21 +609,20 @@ $$ \end{align*} $$ +---- -### Product rule +Other rules can be similarly derived. We simply state the rules for products and quotients. + +::: {.relationship title="Product rule"} +The derivative of a product of functions is given by: + +$$ +[f(x) \cdot g(x)]' = f'(x)\cdot g(x) + f(x) \cdot g'(x) +$$ -Other rules can be similarly derived. `SymPy` can give us them as well. Here we define two symbolic functions `u` and `v` and let `SymPy` derive a formula for the derivative of a product of functions: - - -```{julia} -#| hold: true -@syms u() v() -f(x) = u(x) * v(x) -limit((f(x+h) - f(x))/h, h => 0) -``` - -The output uses the Leibniz notation to represent that the derivative of $u(x) \cdot v(x)$ is the $u$ times the derivative of $v$ evaluated at $x$ plus $v$ times the derivative of $u$ evaluated at $x$. A common shorthand is $[uv]' = u'v + uv'$. +A common shorthand is $[uv]' = u'v + uv'$. +::: This example shows a useful template for the product rule: @@ -629,19 +636,21 @@ $$ \end{align*} $$ +---- -### Quotient rule +The derivative of $f(x) = u(x)/v(x)$---a ratio of functions---can be similarly computed. + +::: {.relationship title="Quotient rule"} +The derivative of a ratio of functions can be computed by the quotient rule: + +$$ +\left[\frac{f(x)}{g(x)}\right]' = \frac{f'(x)\cdot g(x) - f(x) \cdot g'(x)}{g(x)^2}. +$$ + +This is often written as $[u/v]' = (u'v - uv')/v^2$. +::: -The derivative of $f(x) = u(x)/v(x)$ - a ratio of functions - can be similarly computed. The result will be $[u/v]' = (u'v - uv')/v^2$: - - -```{julia} -#| hold: true -@syms u() v() -f(x) = u(x) / v(x) -limit((f(x+h) - f(x))/h, h => 0) -``` This example shows a useful template for the quotient rule: @@ -770,7 +779,11 @@ Find the derivative of $x\sin(x)$ evaluated at $\pi$. $$ -[x\sin(x)]'\big|_{x=\pi} = (1\sin(x) + x\cos(x))\big|_{x=\pi} = (\sin(\pi) + \pi \cdot \cos(\pi)) = -\pi. +\begin{align*} +[x\sin(x)]'\big|_{x=\pi} &= (1\sin(x) + x\cos(x))\big|_{x=\pi} \\ +&= (\sin(\pi) + \pi \cdot \cos(\pi)) \\ +&= -\pi. +\end{align*} $$ ### Chain rule @@ -813,17 +826,24 @@ The left-hand side will converge to the derivative of $u(x)$ or $[f(g(x))]'$. The right-most part of the right-hand side would have a limit $g'(x)$, were we to let $h$ go to $0$. -It isn't obvious, but the left part of the right-hand side has the limit $f'(g(x))$. This would be clear if *only* $g(x+h) = g(x) + h$, for then the expression would be exactly the limit expression with $c=g(x)$. But, alas, except to some hopeful students and some special cases, it is definitely not the case in general that $g(x+h) = g(x) + h$ - that right parentheses actually means something. However, it is *nearly* the case that $g(x+h) = g(x) + kh$ for some $k$ and this can be used to formulate a proof (one of the two detailed [here](http://en.wikipedia.org/wiki/Chain_rule#Proofs) and [here](http://kruel.co/math/chainrule.pdf)). +It isn't obvious, but the left part of the right-hand side has the limit $f'(g(x))$. This would be clear if *only* $g(x+h) = g(x) + h$, for then the expression would be exactly the limit expression with $c=g(x)$. But, alas, except to some hopeful students and some special cases, it is definitely not the case in general that $g(x+h) = g(x) + h$---that right parentheses actually means something. However, it is *nearly* the case that $g(x+h) = g(x) + kh$ for some $k$ and this can be used to formulate a proof, like the one sketched below. -Combined, we would end up with: +Combined, we would end up with the chain rule + +::: {.relationship title="Chain rule"} +The *chain rule* is used to find derivatives of compositions: -> *The chain rule*: $[f(g(x))]' = f'(g(x)) \cdot g'(x)$. That is the derivative of the outer function evaluated at the inner function times the derivative of the inner function. +$$ +[f(g(x))]' = f'(g(x)) \cdot g'(x). +$$ + +That is the derivative of the outer function evaluated at the inner function times the derivative of the inner function. +::: - -To see that this works in our specific case, we assume the general power rule that $[x^n]' = n x^{n-1}$ to get: +To see that the chain rule works in our specific case, we assume the general power rule that $[x^n]' = n x^{n-1}$ (for all real $n$) to get: $$ @@ -848,7 +868,7 @@ $$ $$ -This is the same as the derivative of $x$ found by first evaluating the composition. For this problem, the chain rule is not necessary, but typically it is a needed rule to fully differentiate a function. +This is the same as the derivative of $x$ found by first evaluating the composition. For this problem, the chain rule is not necessary, but typically it is the most important rule to fully differentiate a function. ##### Examples @@ -875,7 +895,7 @@ $$ --- -Find the derivative of $\log(2 + \sin(x))$. This is a composition $\log(x)$ – with derivative $1/x$ and $2 + \sin(x)$ – with derivative $\cos(x)$. We get $(1/(2 + \sin(x))) \cos(x)$. +Find the derivative of $\log(2 + \sin(x))$. This is a composition $\log(x)$---with derivative $1/x$ and $2 + \sin(x)$---with derivative $\cos(x)$. We get $(1/(2 + \sin(x))) \cos(x)$. In general, @@ -914,23 +934,29 @@ $$ A function is *differentiable* at $a$ if the following limit exists $\lim_{h \rightarrow 0}(f(a+h)-f(a))/h$. -This is reexpressed as: $f(a+h) - f(a) - f'(a)h = \epsilon_f(h) h$ where as $h\rightarrow 0$, $\epsilon_f(h) \rightarrow 0$. +This is reexpressed as $f$ is differentitable at $a$ is there exists $\epsilon_f(h) \rightarrow 0$ such that: + +$$ +f(a+h) - f(a) - f'(a)h = \epsilon_f(h) h. +$$ -With that in mind, we have: +As $g$ is differentiable, we have: $$ g(a+h) = g(a) + g'(a)h + \epsilon_g(h) h = g(a) + h', $$ -Where $h' = (g'(a) + \epsilon_g(h))h \rightarrow 0$ as $h \rightarrow 0$ will be used to simplify the following: +Where $h' = g'(a)h + \epsilon_g(h)h \rightarrow 0$ as $h \rightarrow 0$ will be used to simplify the following: + $$ \begin{align*} f(g(a+h)) - f(g(a)) &= f(g(a) + g'(a)h + \epsilon_g(h)h) - f(g(a)) \\ -&= f(g(a)) + f'(g(a)) (g'(a)h + \epsilon_g(h)h) + \epsilon_f(h')(h') - f(g(a))\\ -&= f'(g(a)) g'(a)h + f'(g(a))(\epsilon_g(h)h) + \epsilon_f(h')(h'). +&= \textcolor{red}{f(g(a))} + f'(g(a)) (g'(a)h + \epsilon_g(h)h) + \epsilon_f(h')(h') \\ +&\qquad- \textcolor{red}{f(g(a))}\\ +&= f'(g(a)) g'(a)h + \textcolor{blue}{f'(g(a))(\epsilon_g(h)h) + \epsilon_f(h')(h')}. \end{align*} $$ @@ -939,13 +965,13 @@ Rearranging: $$ \begin{align*} f(g(a+h)) &- f(g(a)) - f'(g(a)) g'(a) h\\ -&= f'(g(a))\epsilon_g(h)h + \epsilon_f(h')(h')\\ +&= \textcolor{blue}{f'(g(a))\epsilon_g(h)h + \epsilon_f(h')(h')}\\ &=(f'(g(a)) \epsilon_g(h) + \epsilon_f(h') (g'(a) + \epsilon_g(h)))h \\ &=\epsilon(h)h, \end{align*} $$ -where $\epsilon(h)$ combines the above terms which go to zero as $h\rightarrow 0$ into one. This is the alternative definition of the derivative, showing $(f\circ g)'(a) = f'(g(a)) g'(a)$ when $g$ is differentiable at $a$ and $f$ is differentiable at $g(a)$. +where $\epsilon(h)$ combines the above terms which go to zero as $h\rightarrow 0$ into one. This is the alternative definition of differentiable, showing $(f\circ g)'(a) = f'(g(a)) g'(a)$ when $g$ is differentiable at $a$ and $f$ is differentiable at $g(a)$. ##### The "chain" rule @@ -1025,47 +1051,30 @@ $$ --- -We can verify these with `SymPy`. Rather than take a limit, we will use `SymPy`'s `diff` function to compute derivatives. +We can verify these with `SymPy`. ```{julia} -diff(x^5 * sin(x)) +diff(x^5 * sin(x), x) ``` ```{julia} -diff(x^5/sin(x)) +diff(x^5/sin(x), x) ``` ```{julia} -diff(sin(x^5)) +diff(sin(x^5), x) ``` and finally, ```{julia} -diff(sin(x)^5) +diff(sin(x)^5, x) ``` -:::{.callout-note} -## Note -The `diff` function can be called as `diff(ex)` when there is just one free variable, as in the above examples; as `diff(ex, var)` when there are parameters in the expression. - -::: - --- - -The general product rule: For any $n$ - not just integer values - we can re-express $x^n$ using $e$: $x^n = e^{n \log(x)}$. Now the chain rule can be applied: - - -$$ -[x^n]' = [e^{n\log(x)}]' = e^{n\log(x)} \cdot (n \frac{1}{x}) = n x^n \cdot \frac{1}{x} = n x^{n-1}. -$$ - ---- - - Find the derivative of $f(x) = x^3 (1-x)^2$ using either the power rule or the sum rule. @@ -1081,7 +1090,6 @@ the last by the chain rule. Combining with $u' v + u v'$ we get: $f'(x) = (3x^2) Otherwise, the polynomial can be expanded to give $f(x)=x^5-2x^4+x^3$ which has derivative $f'(x) = 5x^4 - 8x^3 + 3x^2$. - --- @@ -1131,6 +1139,23 @@ $$ e^{-1/2} \cdot (-1) = -e^{-1/2}. $$ +##### Example: the general product rule + +The general product rule: For any real $n$---not just integer values---we can re-express $x^n$ using $e$: + +$$ +x^n = e^{\log(x^n)} = e^{n \log(x)}. +$$ + + +Now the chain rule can be applied: + + +$$ +[x^n]' = [e^{n\log(x)}]' = e^{n\log(x)} \cdot (n \frac{1}{x}) = n x^n \cdot \frac{1}{x} = n x^{n-1}. +$$ + + ##### Example: derivative of inverse functions @@ -1141,15 +1166,15 @@ $$ 1 = (\frac{1}{e^x}) \cdot [e^x]'. $$ -Or solving, $[e^x]' = e^x$. This is a general strategy to find the derivative of an *inverse* function. +Or solving, $[e^x]' = e^x$. This is a general strategy to find the derivative of an *inverse* function in terms of the derivative of the function. The graph of an inverse function is related to the graph of the function through the symmetry $y=x$. -For example, the graph of $e^x$ and $\log(x)$ have this symmetry, emphasized below: - +For example, the graph of $e^x$ and $\log(x)$ have this symmetry, emphasized in @fig-plot-f-inverse-several-points-showing-derivative. +::: {#fig-plot-f-inverse-several-points-showing-derivative} ```{julia} #| hold: true #| echo: false @@ -1172,6 +1197,9 @@ xs′ = @. x₀ + g(y₀) * (ys - y₀) plot!(ys, xs′, linestyle=:dash) ``` +Plot of $f(x) = e^x$ with its inverse function showing the relationship between tangent lines +::: + The point $(1, e)$ on the graph of $e^x$ matches the point $(e, 1)$ on the graph of the inverse function, $\log(x)$. The slope of the tangent line at $x=1$ to $e^x$ is given by $e$ as well. What is the slope of the tangent line to $\log(x)$ at $x=e$? @@ -1203,40 +1231,33 @@ So the reciprocal of the slope of the tangent line of $f$ at the mirror image po This table summarizes the rules of derivatives that allow derivatives of more complicated expressions to be computed with the derivatives of their pieces. +::: {#tbl-rules-of-derivatives .striped .hover} +| Name | Rule | +| --------------:| ---------------------------------------------------------------:| +| Power rule | $[x^n]' = n\cdot x^{n-1}$ | +| constant | $[cf(x)]' = c \cdot f'(x)$ | +| sum/difference | $[f(x) \pm g(x)]' = f'(x) \pm g'(x)$ | +| product | $[f(x) \cdot g(x)]' = f'(x)\cdot g(x) + f(x) \cdot g'(x)$ | +| quotient | $[f(x)/g(x)]' = (f'(x) \cdot g(x) - f(x) \cdot g'(x)) / g(x)^2$ | +| chain | $[f(g(x))]' = f'(g(x)) \cdot g'(x)$ | -```{julia} -#| hold: true -#| echo: false -nm = ["Power rule", "constant", "sum/difference", "product", "quotient", "chain"] -rule = [L"[x^n]' = n\cdot x^{n-1}", - L"[cf(x)]' = c \cdot f'(x)", - L"[f(x) \pm g(x)]' = f'(x) \pm g'(x)", - L"[f(x) \cdot g(x)]' = f'(x)\cdot g(x) + f(x) \cdot g'(x)", - L"[f(x)/g(x)]' = (f'(x) \cdot g(x) - f(x) \cdot g'(x)) / g(x)^2", - L"[f(g(x))]' = f'(g(x)) \cdot g'(x)"] -d = DataFrame(Name=nm, Rule=rule) -table(d) -``` +: Table summarizing different rules for derivatives +::: -This table gives some useful derivatives: +@tbl-derivatives-of-key-functions shows the derivative of a few key functions. +::: {#tbl-derivatives-of-key-functions .striped .hover} -```{julia} -#| hold: true -#| echo: false -fn = [L"x^n (\text{ all } n)", -L"e^x", -L"\log(x)", -L"\sin(x)", -L"\cos(x)"] -a = [L"nx^{n-1}", -L"e^x", -L"1/x", -L"\cos(x)", -L"-\sin(x)"] -d = DataFrame(Function=fn, Derivative=a) -table(d) -``` +| Function | Derivative | +| ----------------------:| ----------:| +| $x^n (\text{ all } n)$ | $nx^{n-1}$ | +| $e^x$ | $e^x$ | +| $\log(x)$ | $1/x$ | +| $\sin(x)$ | $\cos(x)$ | +| $\cos(x)$ | $-\sin(x)$ | + +Derivatives of some functions +::: ## Higher-order derivatives @@ -1251,6 +1272,7 @@ Find the first $3$ derivatives of $f(x) = ax^3 + bx^2 + cx + d$. Differentiating a polynomial is done with the sum rule, here we repeat three times: + $$ \begin{align*} f(x) &= ax^3 + bx^2 + cx + d\\ @@ -1260,7 +1282,7 @@ f'''(x) &= 6a \end{align*} $$ -We can see, the fourth derivative – and all higher order ones – would be identically $0$. This is part of a general phenomenon: an $n$th degree polynomial has only $n$ non-zero derivatives. +We can see, the fourth derivative---and all higher order ones---would be identically $0$. This is part of a general phenomenon: an $n$th degree polynomial has only $n$ non-zero derivatives. --- @@ -1294,8 +1316,12 @@ We need the chain rule *and* the product rule: $$ -[e^{-x^2}]'' = [e^{-x^2} \cdot (-2x)]' = \left(e^{-x^2} \cdot (-2x)\right) \cdot(-2x) + e^{-x^2} \cdot (-2) = -e^{-x^2}(4x^2 - 2). +\begin{align*} +[e^{-x^2}]'' +&= [e^{-x^2} \cdot (-2x)]' \\ +&= \left(e^{-x^2} \cdot (-2x)\right) \cdot(-2x) + e^{-x^2} \cdot (-2) \\ +&= e^{-x^2}(4x^2 - 2). +\end{align*} $$ This can be verified: @@ -1305,15 +1331,24 @@ This can be verified: diff(diff(exp(-x^2))) |> simplify ``` -Having to iterate the use of `diff` is cumbersome. An alternate notation is either specifying the variable twice: `diff(ex, x, x)` or using a number after the variable: `diff(ex, x, 2)`: +::: {.callout-note} +## higher order derivatives with `diff` +Having to iterate the use of `diff` is cumbersome. An alternate notation is either + +* specifying the variable twice: `diff(ex, x, x)` or +* using a number after the variable: `diff(ex, x, 2)`: + +For example, ```{julia} -diff(exp(-x^2), x, x) |> simplify +diff(exp(-x^2), x, x) ``` Higher-order derivatives can become involved when the product or quotient rules becomes involved. +::: + ## Questions @@ -1321,14 +1356,18 @@ Higher-order derivatives can become involved when the product or quotient rules ###### Question -The derivative at $c$ is the slope of the tangent line at $x=c$. Answer the following based on this graph: +The derivative at $c$ is the slope of the tangent line at $x=c$. Answer the following based on @fig-x-expx-sin-pix-over-0-2. +::: {#fig-x-expx-sin-pix-over-0-2} ```{julia} +#| echo: false fn = x -> -x*exp(x)*sin(pi*x) plot(fn, 0, 2) ``` +Plot of function $f(x)$ over $[0,2]$ +::: At which of these points $c= 1/2, 1, 3/2$ is the derivative negative? @@ -1363,15 +1402,18 @@ numericq(0, 1e-2) ###### Question -Consider the graph of the `airyai` function (from `SpecialFunctions`) over $[-5, 5]$. - +Consider the graph of the `airyai` function (from `SpecialFunctions`) over $[-5, 5]$ in @fig-plot-airyai-over-minus5-5-identify-sign-of-derivative. +::: {#fig-plot-airyai-over-minus5-5-identify-sign-of-derivative} ```{julia} #| hold: true #| echo: false plot(airyai, -5, 5) ``` +Plot of `airyai` over $[-5,5]$ +::: + At $x = -2.5$ the derivative is positive or negative? @@ -1669,8 +1711,9 @@ radioq(trig_choices, 4) ###### Question -Consider this picture of composition: +Consider @fig-composition-of-functions-image, a picture of composition. +::: {#fig-composition-of-functions-image} ```{julia} #| hold: true @@ -1699,6 +1742,10 @@ l = @layout [a b] plot(pf, pg, layout=l) ``` +Plot of function $f(x)$ rotated so $x$ axis is pointing to the left; plot of $g(x)$ on right. This picture allows graphical composition to be computed. +::: + + The right graph is of $g(x) = \exp(x)$ at $x=1$, the left graph of $f(x) = \sin(x)$ *rotated* $90$ degrees counter-clockwise. Chasing the arrows shows graphically how $f(g(1))$ can be computed. The nearby values $f(g(1+h))$ are – using the tangent line of $g$ at $x-1$ – approximated by $f(g(1) + g'(1)\cdot h)$, as shown in the graph segment on the left. diff --git a/quarto/derivatives/first_second_derivatives.qmd b/quarto/derivatives/first_second_derivatives.qmd index b60cde9..869d4b4 100644 --- a/quarto/derivatives/first_second_derivatives.qmd +++ b/quarto/derivatives/first_second_derivatives.qmd @@ -14,11 +14,18 @@ using SymPy using Roots ``` +```{julia} +function CalculusWithJulia.plotif(f::Function, g::Function, a::Real, b::Real; kwargs...) + plot(f, a, b; legend=false, line=(1, :black), title="Plot of f colored when g ≥ 0") + gg = x -> g(x) ≥ 0 ? f(x) : NaN + plot!(gg; line=(5, :red, :dot)) +end +``` --- -This section explores properties of a function, $f(x)$, that are described by properties of its first and second derivatives, $f'(x)$ and $f''(x)$. As part of the conversation two tests are discussed that characterize when a critical point is a relative maximum or minimum. (We know that any relative maximum or minimum occurs at a critical point, but it is not true that *any* critical point will be a relative maximum or minimum.) +This section explores properties of a function, $f(x)$, that are described by properties of its first and second derivatives, $f'(x)$ and $f''(x)$. As part of the conversation, two tests are discussed that characterize when a critical point is a relative maximum or minimum. (We know that any relative maximum or minimum occurs at a critical point, but it is not true that *any* critical point will be a relative maximum or minimum.) ## Positive or increasing on an interval @@ -26,18 +33,21 @@ This section explores properties of a function, $f(x)$, that are described by pr We start with some vocabulary: +::: {.definition title="Positive on an interval"} -> A function $f$ is **positive** on an interval $I$ if for any $a$ in $I$ it must be that $f(a) > 0$. +A function $f$ is **positive** on an interval $I$ if for any $a$ in $I$ it must be that $f(a) > 0$. +Of course, we define *negative* in a parallel manner. -Of course, we define *negative* in a parallel manner. The intermediate value theorem says a continuous function can not change from positive to negative without crossing $0$. This is not the case for functions with jumps, of course. +::: + +The intermediate value theorem says a continuous function can not change from positive to negative without crossing $0$. This is not the case for functions with jumps, of course. Next, -::: {.callout-note icon=false} -## Strictly increasing +::: {.definition title="Strictly increasing"} A function, $f$, is (strictly) **increasing** on an interval $I$ if for any $a < b$ it must be that $f(a) < f(b)$. @@ -47,51 +57,61 @@ The word strictly is related to the inclusion of the $<$ precluding the possibil A parallel definition with $a < b$ implying $f(a) > f(b)$ would be used for a *strictly decreasing* function. ::: -We can try and prove these properties for a function algebraically – we'll see both are related to the zeros of some function. However, before proceeding to that it is usually helpful to get an idea of where the answer is using exploratory graphs. + +We introduce a helper function `plotif` from the `CalculusWithJulia` package that highlights the graph of a function $f$ when another function $g(x)$ satisifies $g(x) \geq 0$. This function is called as `plotif(f, g, a, b)`. -We will use a helper function, `plotif(f, g, a, b)` that plots the function `f` over `[a,b]` highlighting the regions in the domain when `g` is non-negative. Such a function is defined for us in the accompanying `CalculusWithJulia` package, which has been previously loaded. - - -To see where a function is positive, we simply pass the function object in for *both* `f` and `g` above. For example, let's look at where $f(x) = \sin(x)$ is positive: - +To see where a function is positive, we simply pass the function object in for *both* `f` and `g` above. For example, in @fig-plotif-sin-sin-minus-2pi-2pi we look at where $f(x) = \sin(x)$ is positive. +::: {#fig-plotif-sin-sin-minus-2pi-2pi} ```{julia} #| hold: true f(x) = sin(x) plotif(f, f, -2pi, 2pi) +plot!(zero) ``` -Let's graph with `cos` in the masking spot and see what happens: +Plot of $f(x) = \sin(x)$ over $[-2\pi,2\pi]$ highlighting where $f(x) \geq 0$ +::: +Let's graph with `cos` in the masking spot and see what happens. +::: {#fig-plotif-sin-cos-minus-2pi-2pi} ```{julia} plotif(sin, cos, -2pi, 2pi) ``` -Maybe surprisingly, we see that the increasing parts of the sine curve are now highlighted. Of course, the cosine is the derivative of the sine function, now we discuss that this is no coincidence. +Plot of $f(x) = \sin(x)$ over $[-2\pi,2\pi]$ highlighting where $f'(x) = \cos(x) \geq 0$ +::: + +Maybe surprisingly, in @fig-plotif-sin-cos-minus-2pi-2pi we see that the increasing parts of the sine curve are now highlighted. Of course, the cosine is the derivative of the sine function, now we discuss that this is no coincidence. For the sequel, we will use `f'` notation to find numeric derivatives, with the notation being defined in the `CalculusWithJulia` package using the `ForwardDiff` package. - -## The relationship of the derivative and increasing +## Positive derivatives and increasing functions The derivative, $f'(x)$, computes the slope of the tangent line to the graph of $f(x)$ at the point $(x,f(x))$. If the derivative is positive, the tangent line will have an increasing slope. Clearly if we see an increasing function and mentally layer on a tangent line, it will have a positive slope. Intuitively then, increasing functions and positive derivatives are related concepts. But there are some technicalities. -Suppose $f(x)$ has a derivative on $I$ . Then +Suppose $f(x)$ has a derivative on $I$. Then +::: {.relationship title="Positive derivative and strictly increasing function"} -> If $f'(x)$ is positive on an interval $I=(a,b)$, then $f(x)$ is strictly increasing on $I$. +If $f'(x)$ is positive on an interval $I=(a,b)$, then $f(x)$ is strictly increasing on $I$. + +::: Meanwhile, +::: {.relationship title="Strictly increasing function and non-negative derivative"} -> If a function $f(x)$ is increasing on $I$, then $f'(x) \geq 0$. +If a function $f(x)$ is increasing on $I$, then $f'(x) \geq 0$. + +::: @@ -110,15 +130,18 @@ The second part, follows from the secant line equation. The derivative can be wr So, to visualize where a function is increasing or flat, we can just pass in the derivative as the masking function in our `plotif` function. -For example, here, with a more complicated function, the intervals where the function is increasing are highlighted by passing in the functions derivative to `plotif`: - +For example, in @fig-plotif-sin-pi-x-times-x-cubed-minus-4-x-squared-plus-2, with a more complicated function, the intervals where the function is increasing are highlighted by passing in the function's derivative to `plotif`. +::: {#fig-plotif-sin-pi-x-times-x-cubed-minus-4-x-squared-plus-2} ```{julia} #| hold: true f(x) = sin(pi*x) * (x^3 - 4x^2 + 2) plotif(f, f', -2, 2) ``` +Plot of $f(x) = \sin(\pi\cdot x) \cdot (x^3 - 4x^2 + 2)$ over $[-2,2]$ highlighting when $f'(x) \geq 0$ +::: + ### First derivative test @@ -128,26 +151,26 @@ When a function changes from increasing to decreasing, or decreasing to increasi When discussing the mean value theorem, we defined *relative extrema* : -> * The function $f(x)$ has a *relative maximum* at $c$ if the value $f(c)$ is an *absolute maximum* for some *open* interval containing $c$. -> * Similarly, $f(x)$ has a *relative minimum* at $c$ if the value $f(c)$ is an absolute minimum for *some* open interval about $c$. +::: {.definition title="Relative maximum, minimum"} + +* The function $f(x)$ has a *relative maximum* at $c$ if the value $f(c)$ is an *absolute maximum* for some *open* interval containing $c$. + +* Similarly, $f(x)$ has a *relative minimum* at $c$ if the value $f(c)$ is an absolute minimum for *some* open interval about $c$. + +::: -We know since [Fermat](http://tinyurl.com/nfgz8fz) that: +We discussed Fermat's theorem which in short says that relative maxima and minima *must* occur at *critical* points of the function. - -> Relative maxima and minima *must* occur at *critical* points. - - - -Fermat says that *critical points* – where the function is defined, but its derivative is either $0$ or undefined – are *interesting* points, however: +However: > A critical point need not indicate a relative maxima or minima. -Again, $f(x)=x^3$ provides the example at $x=0$. This is a critical point, but clearly not a relative maximum or minimum - it is just a slight pause for a strictly increasing function. +Again, $f(x)=x^3$ provides the example at $x=0$. This is a critical point, but clearly not a relative maximum or minimum---it is just a slight pause for a strictly increasing function. This leaves the question: @@ -160,20 +183,21 @@ This leaves the question: This question can be answered by considering the first derivative. -::: {.callout-note icon=false} -## The first derivative test +::: {.relationship title="The first derivative test"} If $c$ is a critical point for $f(x)$ and *if* $f'(x)$ changes sign at $x=c$, then $f(c)$ will be either a relative maximum or a relative minimum. - * $f$ will have a relative maximum at $c$ if the derivative changes sign from $+$ to $-$. - * $f$ will have a relative minimum at $c$ if the derivative changes sign from $-$ to $+$. +* $f$ will have a relative maximum at $c$ if the derivative changes sign from $+$ to $-$. - Further, If $f'(x)$ does *not* change sign at $c$, then $f$ will *not* have a relative maximum or minimum at $c$. +* $f$ will have a relative minimum at $c$ if the derivative changes sign from $-$ to $+$. + + +Further, If $f'(x)$ does *not* change sign at $c$, then $f$ will *not* have a relative maximum or minimum at $c$. ::: -The classification part, should be clear: e.g., if the derivative is positive then negative, the function $f$ will increase to $(c,f(c))$ then decrease from $(c,f(c))$ – so $f$ will have a local maximum at $c$. +The classification part, should be clear: e.g., if the derivative is positive then negative, the function $f$ will increase to $(c,f(c))$ then decrease from $(c,f(c))$---so $f$ will have a local maximum at $c$. Our definition of critical point *assumes* $f(c)$ exists, as $c$ is in the domain of $f$. With this assumption, vertical asymptotes are avoided. However, it need not be that $f'(c)$ exists. The absolute value function at $x=0$ provides an example: this point is a critical point where the derivative changes sign, but $f'(x)$ is not defined at exactly $x=0$. Regardless, it is guaranteed that $f(c)$ will be a relative minimum by the first derivative test. @@ -184,13 +208,16 @@ Our definition of critical point *assumes* $f(c)$ exists, as $c$ is in the domai Consider the function $f(x) = e^{-\lvert x\rvert} \cos(\pi x)$ over $[-3,3]$: - +::: {#fig-plotif-of-exp-abs-x-times-cos-pi-x-over-minus3-3} ```{julia} f(x) = exp(-abs(x)) * cos(pi * x) plotif(f, f', -3, 3) ``` -We can see the first derivative test in action: at the peaks and valleys – the relative extrema – the highlighting changes. This is because $f'$ is changing sign as the function changes from increasing to decreasing or vice versa. +Plot of $f(x) = e^{-\lvert x \rvert \cdot \cos(\pi \cdot x)}$ over $[-3,3]$ highlighting when $f'(x) \geq 0$ +::: + +In @fig-plotif-of-exp-abs-x-times-cos-pi-x-over-minus3-3 we can see the first derivative test in action: at the peaks and valleys---the relative extrema---the highlighting changes. This is because $f'$ is changing sign as the function changes from increasing to decreasing or vice versa. This function has a critical point at $0$, as can be seen. It corresponds to a point where the derivative does not exist. It is still identified through `find_zeros`, which picks up zeros and in case of discontinuous functions, like `f'`, zero crossings: @@ -214,19 +241,23 @@ f(x) = sin(pi*x) * (x^3 - 4x^2 + 2) cps = find_zeros(f', -2, 2) ``` -We should be careful though, as `find_zeros` may miss zeros that are not simple or too close together. A critical point will correspond to a relative maximum if the function crosses the axis, so these can not be "pauses." As this is exactly the case we are screening for, we double check that all the critical points are accounted for by graphing the derivative: - +We should be careful though, as `find_zeros` may miss zeros that are not simple or too close together. A critical point will correspond to a relative maximum if the function crosses the axis, so these can not be "pauses." As this is exactly the case we are screening for, we double check that all the critical points are accounted for by graphing the derivative in @fig-plot-derivative-sin-pix-times-x-cubed-minus-4-x-squared-plus-2. We see the six zeros as stored in `cps` and note that at each the function clearly crosses the $x$ axis. +::: {#fig-plot-derivative-sin-pix-times-x-cubed-minus-4-x-squared-plus-2} ```{julia} +#| echo: false plot(f', -2, 2, legend=false) plot!(zero) scatter!(cps, 0*cps) ``` -We see the six zeros as stored in `cps` and note that at each the function clearly crosses the $x$ axis. +Plot of the derivative of $f(x) = \sin(\pi\cdot x) \cdot (x^3 - 4x^2 + 2)$ over $[-2,2]$ with the critical points of $f(x)$ (in this case the zeros of $f'(x)$ emphasized. At each, the derivative changes sign. +::: -From this last graph of the derivative we can also characterize the graph of $f$: The left-most critical point coincides with a relative minimum of $f$, as the derivative changes sign from negative to positive. The critical points then alternate relative maximum, relative minimum, relative maximum, relative minimum, and finally relative maximum. + + +From the graph of the derivative we can also characterize the graph of $f$: The left-most critical point coincides with a relative minimum of $f$, as the derivative changes sign from negative to positive. The critical points then alternate relative maximum, relative minimum, relative maximum, relative minimum, and finally relative maximum. ##### Example @@ -246,12 +277,11 @@ cps = find_zeros(g', -2, 2) We see the three values $-1$, $0$, $1$ that correspond to the two zeros and the relative maximum of $x^2 - 1$. We could graph things, but instead we characterize these values using a sign chart. A piecewise continuous function can only change sign when it crosses $0$ or jumps over $0$. The derivative will be continuous, except possibly at the three values above, so is piecewise continuous. -A sign chart picks convenient values between crossing points to test if the function is positive or negative over those intervals. When computing by hand, these would ideally be values for which the function is easily computed. On the computer, this isn't a concern; below the midpoint is chosen: +A sign chart picks convenient values between crossing points to test if the function is positive or negative over those intervals. When computing by hand, these would ideally be values for which the function is easily computed. On the computer, this isn't a concern but we want to easily find the $x$ values to evaluate. Below, we specify them directly, but an algorithm to identify the midpoints isn't hard to imagine: ```{julia} -pts = sort(union(-2, cps, 2)) # this includes the endpoints (a, b) and the critical points -test_pts = pts[1:end-1] + diff(pts)/2 # midpoints of intervals between pts +test_pts = [ -1.5, -0.5, 0.5, 1.5] [test_pts sign.(g'.(test_pts))] ``` @@ -269,37 +299,44 @@ Such values are often summarized graphically on a number line using a *sign char Reading this we have: +* the derivative changes sign from negative to positive at $x=-1$, so $g(x)$ will have a relative minimum. - * the derivative changes sign from negative to positive at $x=-1$, so $g(x)$ will have a relative minimum. - * the derivative changes sign from positive to negative at $x=0$, so $g(x)$ will have a relative maximum. - * the derivative changes sign from negative to positive at $x=1$, so $g(x)$ will have a relative minimum. +* the derivative changes sign from positive to negative at $x=0$, so $g(x)$ will have a relative maximum. + +* the derivative changes sign from negative to positive at $x=1$, so $g(x)$ will have a relative minimum. In the `CalculusWithJulia` package there is `sign_chart` function that will do such work for us, though with a different display: ```{julia} -sign_chart(g', -2, 2) +sc = sign_chart(g', -2, 2) ``` -(This function numerically identifies $x$-values for the specified function which are zeros, infinities, or points where the function jumps $0$. It then shows the resulting sign pattern of the function from left to right.) - - -We did this all without graphs. But, let's look at the graph of the derivative: +The `sign_chart` function uses `find_zeros` to search for zeros of $f(x)$, zeroes of $1/f(x)$, and through the use of bisection, will identify other sign changes of each. For identified values, it then indicates the sign on the immediate left and right. In this case, the function identifies $-1$, $0$, and $1$. As the returned value is a named tuple, the command `first.(sc)` can be used to get these identified values. +::: {#fig-plot-of-g-prime-over-minus-2-2-g-is-sqrt-abs-x-squared-minus-1} ```{julia} -plot(g', -2, 2) +#| echo: false +plot(rangeclamp(g', 40), -2, 2; label="g'") ``` -We see asymptotes at $x=-1$ and $x=1$! These aren't zeroes of $f'(x)$, but rather where $f'(x)$ does not exist. The conclusion is correct - each of $-1$, $0$ and $1$ are critical points with the identified characterization - but not for the reason that they are all zeros. +Plot of the derivative of $g(x) = \sqrt{\lvert x^2 - 1 \rvert}$ over $[-2,2]$ +::: +We did this all without graphs. +@fig-plot-of-g-prime-over-minus-2-2-g-is-sqrt-abs-x-squared-minus-1 shows the graph of the derivative of $g(x)$. +We see asymptotes at $x=-1$ and $x=1$! These aren't zeroes of $f'(x)$, but rather where $f'(x)$ does not exist. The conclusion is correct---each of $-1$, $0$ and $1$ are critical points with the identified characterization---but not for the reason that they are all zeros. A plot of $g(x)$ is shown in @fig-plot-g-over-minus-2-2-g-is-sqrt-abs-x-squared-minus-1. +::: {#fig-plot-g-over-minus-2-2-g-is-sqrt-abs-x-squared-minus-1} ```{julia} -plot(g, -2, 2) +#| echo: false +plot(g, -2, 2; label="g") ``` -Finally, why does `find_zeros` find these values that are not zeros of $g'(x)$? As discussed briefly above, it uses the bisection algorithm on bracketing intervals to find zeros which are guaranteed by the intermediate value theorem, but when applied to discontinuous functions, as `f'` is, will also identify values where the function jumps over $0$. +Plot of $g(x) = \sqrt{\lvert x^2 - 1 \rvert}$ over $[-2,2]$ +::: ##### Example @@ -341,13 +378,16 @@ fp(2pi - pi/2), fp(2pi + pi/2) Again, both negative. The function $f(x)$ is just decreasing near $2\pi$, so again the critical point is neither a relative minimum nor maximum. -A graph verifies this: - +A graph of `fx` in @fig-plot-fx-sinx-minus-x-over-minus3-pi-3-pi verifies this. +::: {#fig-plot-fx-sinx-minus-x-over-minus3-pi-3-pi} ```{julia} plot(fx, -3pi, 3pi) ``` +Plot of `fx` ($\sin(x)-x$) over $[-3\pi, 3\pi]$ +::: + We see that at $0$ and $2\pi$ there are "pauses" as the function decreases. We should also see that this pattern repeats. The critical points found by `solve` are only those within a certain domain. Any value that satisfies $\cos(x) - 1 = 0$ will be a critical point, and there are infinitely many of these of the form $n \cdot 2\pi$ for $n$ an integer. @@ -371,13 +411,22 @@ sign_chart((x -> sin(x)-x)', -3pi, 3pi) ##### Example -Suppose you know $f'(x) = (x-1)\cdot(x-2)\cdot (x-3) = x^3 - 6x^2 + 11x - 6$ and $g'(x) = (x-1)\cdot(x-2)^2\cdot(x-3)^3 = x^6 -14x^5 +80x^4-238x^3+387x^2-324x+108$. +Suppose you know the derivatives of two functions $f(x)$ and $g(x)$: + +$$ +\begin{align*} +f'(x) &= (x-1)\cdot(x-2)\cdot (x-3) \\ +&= x^3 - 6x^2 + 11x - 6 \\ +g'(x) &= (x-1)\cdot(x-2)^2\cdot(x-3)^3 \\ +&= x^6 -14x^5 +80x^4-238x^3+387x^2-324x+108. +\end{align*} +$$ How would the graphs of $f(x)$ and $g(x)$ differ, as they share identical critical points? -The graph of $f(x)$ - a function we do not have a formula for - can have its critical points characterized by the first derivative test. As the derivative changes sign at each, all critical points correspond to relative maxima. The sign pattern is negative/positive/negative/positive so we have from left to right a relative minimum, a relative maximum, and then a relative minimum. This is consistent with a $4$th degree polynomial with $3$ relative extrema. +The graph of $f(x)$---a function we do not have a formula for---can have its critical points characterized by the first derivative test. As the derivative changes sign at each, all critical points correspond to relative maxima. The sign pattern is negative/positive/negative/positive so we have from left to right a relative minimum, a relative maximum, and then a relative minimum. This is consistent with a $4$th degree polynomial with $3$ relative extrema. For the graph of $g(x)$ we can apply the same analysis. Thinking for a moment, we see as the factor $(x-2)^2$ comes as a power of $2$, the derivative of $g(x)$ will not change sign at $x=2$, so there is no relative extreme value there. However, at $x=3$ the factor has an odd power, so the derivative will change sign at $x=3$. So, as $g'(x)$ is positive for large *negative* values, there will be a relative maximum at $x=1$ and, as $g'(x)$ is positive for large *positive* values, a relative minimum at $x=3$. @@ -404,12 +453,13 @@ g' + 0 - 0 - 0 + g'-sign ## Concavity -Consider the function $f(x) = x^2$. Over this function we draw some secant lines for a few pairs of $x$ values: - +Consider the function $f(x) = x^2$ plotted in @fig-graph-with-secant-lines-drawn. Over the graph of this function we drew two secant lines for a few pairs of $x$ values: +::: {#fig-graph-with-secant-lines-drawn} ```{julia} #| echo: false let + gr() f(x) = x^2 seca(f,a,b) = x -> f(a) + (f(b) - f(a)) / (b-a) * (x-a) p = plot(f, -2, 3, legend=false, linewidth=5, xlim=(-2,3), ylim=(-2, 9)) @@ -419,49 +469,69 @@ let plot!(p,seca(f, 0, 3/2)) a,b = 0, 3/2; xs = range(a, stop=b, length=50) plot!(xs, seca(f, a, b).(xs), linewidth=5) + plotly() p end ``` +Plot of function $f(x) = x^2$ over $[-2,3]$ with two secant lines drawn. The lines are above the graph of the function between the two points defining them. +::: + The graph attempts to illustrate that for this function the secant line between any two points $a < b$ will lie above the graph over $[a,b]$. This is a special property not shared by all functions. Let $I$ be an open interval. -::: {.callout-note icon=false} -## Concave up +::: {.definition title="Concave up"} -A function $f(x)$ is concave up on $I$ if for any $a < b$ in $I$, the secant line between $a$ and $b$ lies above the graph of $f(x)$ over $[a,b]$. +A function $f(x)$ is *concave up* on $I$ if for any $a < b$ in $I$, the secant line between $a$ and $b$ lies above the graph of $f(x)$ over $[a,b]$. A similar definition exists for *concave down* where the secant lines lie below the graph. ::: -Notationally, concave up says for any $x$ in $[a,b]$: +Notationally, concave up says for any $x$ and $[a,b]$ in $I$ that $$ f(a) + \frac{f(b) - f(a)}{b-a} \cdot (x-a) \geq f(x) \quad\text{ (concave up) } $$ -Replacing $\geq$ with $\leq$ defines *concave down*, and with either $>$ or $<$ will add the prefix "strictly." These definitions are useful for a general definition of [convex functions](https://en.wikipedia.org/wiki/Convex_function). +Similarly, for and $t$ in $[0,1]$ and $[a,b]$ in $I$ this must hold: + +$$ +t \cdot f(a) + (1-t) \cdot f(b) \geq f(t\cdot a + (1-t)\cdot b). +$$ + + +Replacing $\geq$ above with $\leq$ defines *concave down* over $I$. + +Strictly concave up would replace $\geq$ with $>$ for any $x$ in $(a,b)$. These definitions are useful for a general definition of [convex functions](https://en.wikipedia.org/wiki/Convex_function). We won't work with these definitions in this section, rather we will characterize concavity for functions which have either a first or second derivative: +::: {.relationship title="First derivative and concavity"} -> * If $f'(x)$ exists and is *increasing* on $(a,b)$, then $f(x)$ is concave up on $(a,b)$. -> * If $f'(x)$ is *decreasing* on $(a,b)$, then $f(x)$ is concave *down*. +* If $f'(x)$ exists and is *increasing* on $(a,b)$, then $f(x)$ is concave up on $(a,b)$. + +* If $f'(x)$ is *decreasing* on $(a,b)$, then $f(x)$ is concave *down*. + +::: A proof of this makes use of the same trick used to establish the mean value theorem from Rolle's theorem. Assume $f'$ is increasing and let $g(x) = f(x) - (f(a) + M \cdot (x-a))$, where $M$ is the slope of the secant line between $a$ and $b$. By construction $g(a) = g(b) = 0$. If $f'(x)$ is increasing, then so is $g'(x) = f'(x) + M$. By its definition above, showing $f$ is concave up is the same as showing $g(x) \leq 0$. Suppose to the contrary that there is a value where $g(x) > 0$ in $[a,b]$. We show this can't be. Assuming $g'(x)$ always exists, after some work, Rolle's theorem will ensure there is a value where $g'(c) = 0$ and $(c,g(c))$ is a relative maximum, and as we know there is at least one positive value, it must be $g(c) > 0$. The first derivative test then ensures that $g'(x)$ will be positive to the left of $c$ and negative to the right of $c$, since $c$ is at a critical point and not an endpoint. But this can't happen as $g'(x)$ is assumed to be increasing on the interval. -The relationship between increasing functions and their derivatives – if $f'(x) > 0$ on $I$, then $f$ is increasing on $I$ – gives this second characterization of concavity when the second derivative exists: +The relationship between increasing functions and their derivatives – if $f'(x) > 0$ on $I$, then $f$ is increasing on $I$---gives this second characterization of concavity when the second derivative exists: +::: {.relationship title="Second derivative and concavity"} -> * If $f''(x)$ exists and is positive on $I$, then $f(x)$ is concave up on $I$. -> * If $f''(x)$ exists and is negative on $I$, then $f(x)$ is concave down on $I$. +* If $f''(x)$ exists and is positive on $I$, then $f(x)$ is concave up on $I$. + +* If $f''(x)$ exists and is negative on $I$, then $f(x)$ is concave down on $I$. + +::: @@ -471,15 +541,19 @@ This follows, as we can think of $f''(x)$ as just the first derivative of the f ##### Example -Let's look at the function $x^2 \cdot e^{-x}$ for positive $x$. A quick graph shows the function is concave up, then down, then up in the region plotted: - +Let's look at the function $g(x) = x^2 \cdot e^{-x}$ for positive $x$. @fig-plot-g-x-squared-times-exp-minus-x-over-0-8 shows a graph of $g(x)$ using `plotif` to highlight when $g''(x)$ is non-negative. The function is concave up, then down, then up in the region plotted. +::: {#fig-plot-g-x-squared-times-exp-minus-x-over-0-8} ```{julia} +#| echo: false g(x) = x^2 * exp(-x) plotif(g, g'', 0, 8) ``` -From the graph, we would expect that the second derivative - which is continuous - would have two zeros on $[0,8]$: +Plot of $g(x) = x^2 \cdot e^{-x}$ over $[0,8]$ highlighting when $g''(x) \geq 0$ +::: + +From the graph, we would expect that the second derivative---which is continuous---would have two zeros on $[0,8]$: ```{julia} @@ -496,16 +570,17 @@ sign_chart(g'', 0, 8) ### Second derivative test -Concave up functions are "opening" up, and often clearly $U$-shaped, though that is not necessary. At a relative minimum, where there is a $U$-shape, the graph will be concave up; conversely at a relative maximum, where the graph has a downward $\cap$-shape, the function will be concave down. This observation becomes: +Concave up functions are "opening" up, and often clearly $U$-shaped, though that is not necessary. At a relative minimum, where there is a $U$-shape, the graph will be concave up; conversely at a relative maximum, where the graph has a downward ∩-shape, the function will be concave down. This observation becomes: -::: {.callout-note icon=false} -## The second derivative test +::: {.definition title="The second derivative test"} If $c$ is a critical point of $f(x)$ with $f''(c)$ existing in a neighborhood of $c$, then - * $f$ will have a relative minimum at the critical point $c$ if $f''(c) > 0$, - * $f$ will have a relative maximum at the critical point $c$ if $f''(c) < 0$, and - * *if* $f''(c) = 0$ the test is *inconclusive*. +* $f$ will have a relative minimum at the critical point $c$ if $f''(c) > 0$, + +* $f$ will have a relative maximum at the critical point $c$ if $f''(c) < 0$, and + +* the test is *inconclusive* if $f''(c) = 0$, ::: @@ -536,13 +611,17 @@ We can check the sign of the second derivative for each critical point: That $j''(0.6) < 0$ implies that at $0.6$, $j(x)$ will have a relative maximum. As $j''(1) > 0$, the second derivative test says at $x=1$ there will be a relative minimum. That $j''(0) = 0$ says that only that there **may** be a relative maximum or minimum at $x=0$, as the second derivative test does not speak to this situation. (This last check, requiring a function evaluation to be `0`, is susceptible to floating point errors, so isn't very robust as a general tool.) -This should be consistent with this graph, where $-0.25$, and $1.25$ are chosen to capture the zero at $0$ and the two relative extrema: - +This should be consistent with the graph in @fig-plotif-j-j-double-prime-j-is-x-to-the-5-minus-2x-to-the-4-plusx-cubed, where $-0.25$, and $1.25$ are chosen to capture the zero at $0$ and the two relative extrema: +::: {#fig-plotif-j-j-double-prime-j-is-x-to-the-5-minus-2x-to-the-4-plusx-cubed} ```{julia} plotif(j, j'', -0.25, 1.25) ``` +Plot of $j(x) = x^5 - 2x^4 + x^3$ over $[-1/4, 5/4]$ highlighting when $j''(x) \geq 0$ +::: + + For the graph we see that $0$ **is not** a relative maximum or minimum. We could have seen this numerically by checking the first derivative test, and noting there is no sign change: @@ -553,22 +632,30 @@ sign_chart(j', -3, 3) ##### Example -One way to visualize the second derivative test is to *locally* overlay on a critical point a parabola. For example, consider $f(x) = \sin(x) + \sin(2x) + \sin(3x)$ over $[0,2\pi]$. It has $6$ critical points over $[0,2\pi]$. In this graphic, we *locally* layer on $6$ parabolas: - +One way to visualize the second derivative test is to *locally* overlay on a critical point a parabola. For example, consider $f(x) = \sin(x) + \sin(2x) + \sin(3x)$ over $[0,2\pi]$. It has $6$ critical points over $[0,2\pi]$. In @fig-plot-of-sin-x-sin-2x-sin-3x-with-parabola-overlays, we *locally* layer on $6$ parabolas at the critical points: +::: {#fig-plot-of-sin-x-sin-2x-sin-3x-with-parabola-overlays} ```{julia} -#| hold: true -f(x) = sin(x) + sin(2x) + sin(3x) -p = plot(f, 0, 2pi, legend=false, color=:blue, linewidth=3) -cps = find_zeros(f', (0, 2pi)) -Δ = 0.5 -for c in cps - parabola(x) = f(c) + (f''(c)/2) * (x-c)^2 - plot!(parabola, c - Δ, c + Δ, color=:red, linewidth=5, alpha=0.6) +#| echo: false +let + gr() + f(x) = sin(x) + sin(2x) + sin(3x) + parabola_atc(x, c) = f(c) + (f''(c)/2) * (x-c)^2 + p = plot(f, 0, 2pi; legend=false, line=(2, :blue)) + cps = find_zeros(f', (0, 2pi)) + Δ = 0.5 + for c in cps + plot!(Base.Fix2(parabola_atc,c), c - Δ, c + Δ; line=(:red, 5, 0.6)) + end + plotly() + p end -p ``` +Plot of $f(x) = \sin(x) + \sin(2x) + \sin(3x)$ over $[0,2\pi]$ with parabola overlays at the critical points of $f(x)$ +::: + + The graphic shows that for this function near the relative extrema the parabolas *approximate* the function well, so that the relative extrema are characterized by the relative extrema of the parabolas. @@ -582,9 +669,11 @@ $$ The $2$ is a mystery to be answered in the section on [Taylor series](../taylor_series_polynomials.html), the focus here is on the *sign* of $f''(c)$: - * if $f''(c) > 0$ then the approximating parabola opens upward and the critical point is a point of relative minimum for $f$, - * if $f''(c) < 0$ then the approximating parabola opens downward and the critical point is a point of relative maximum for $f$, and - * were $f''(c) = 0$ then the approximating parabola is just a line – the tangent line at a critical point – and is non-informative about extrema. +* if $f''(c) > 0$ then the approximating parabola opens upward and the critical point is a point of relative minimum for $f$, + +* if $f''(c) < 0$ then the approximating parabola opens downward and the critical point is a point of relative maximum for $f$, and + +* were $f''(c) = 0$ then the approximating parabola is just a line---the tangent line at a critical point---and is non-informative about extrema. That is, the parabola picture is just the second derivative test in this light. @@ -592,63 +681,78 @@ That is, the parabola picture is just the second derivative test in this light. ### Inflection points +::: {.definition title="Inflection point"} -An inflection point is a value where the *second* derivative of $f$ changes sign. At an inflection point the derivative will change from increasing to decreasing (or vice versa) and the function will change from concave up to down (or vice versa). +A function $f(x)$ has an inflection point at $x=c$ if $(c, f(c))$is a point on the graph of $f(x)$ where the concavity changes. + +::: + +For a function with a second derivative, an inflection point is a value where the *second* derivative of $f$ changes sign. When the second derivative is continuous, these points will occur when the second derivative is $0$ *and* changes sign. -We can use the `find_zeros` function to identify potential inflection points by passing in the second derivative function. For example, consider the bell-shaped function +We can use the `sign_chart` function to identify potential inflection points by passing in the second derivative function. For example, consider the bell-shaped function: $$ k(x) = e^{-x^2/2}. $$ -A graph suggests relative a maximum at $x=0$, a horizontal asymptote of $y=0$, and two inflection points: - +A graph of $k(x)$ (@fig-plot-of-exp-minus-x-squared-over-2-over-minus3-3) suggests relative a maximum at $x=0$, a horizontal asymptote of $y=0$, and two inflection points. +::: {#fig-plot-of-exp-minus-x-squared-over-2-over-minus3-3} ```{julia} k(x) = exp(-x^2/2) plotif(k, k'', -3, 3) ``` +Plot of $f(x) = e^{-x^2/2}$ over $[-3,3]$ +::: + The inflection points can be found directly, if desired, or numerically with: ```{julia} -find_zeros(k'', -3, 3) +sign_chart(k'', -3, 3) ``` -(The `find_zeros` function may return points which are not inflection points. It primarily returns points where $k''(x)$ changes sign, but *may* also find points where $k''(x)$ is $0$ yet does not change sign at $x$.) + +The `sign_chart` function searches for zeros, infinities, and places where the function changes sign. To check for inflection points, a sign change needs to occur, as it does for both values here. ##### Example -A car travels from a stop for 1 mile in 2 minutes. A graph of its position as a function of time might look like any of these graphs: - +A car travels from a stop for 1 mile in 2 minutes. A graph of its position as a function of time might look like any of the graphs in @fig-three-plots-different-velocities. +::: {#fig-three-plots-different-velocities} ```{julia} #| echo: false let + gr() v(t) = 30/60*t w(t) = t < 1/2 ? 0.0 : (t > 3/2 ? 1.0 : (t-1/2)) y(t) = 1 / (1 + exp(-t)) y1(t) = y(2(t-1)) y2(t) = y1(t) - y1(0) y3(t) = 1/y2(2) * y2(t) - plot(v, 0, 2, label="f1") - plot!(w, label="f2") - plot!(y3, label="f3") + plt = plot(v, 0, 2; label="f1", xlabel="time", ylabel="position") + plot!(plt, w, label="f2") + plot!(plt, y3, label="f3") + plotly() + plt end ``` -All three graphs have the same *average* velocity which is just the $1/2$ miles per minute ($30$ miles an hour). But the instantaneous velocity - which is given by the derivative of the position function) varies. +Plot of three graphs with same average velocity, but different velocities +::: + +All three graphs have the same *average* velocity which is just the $1/2$ miles per minute ($30$ miles an hour). But the instantaneous velocity---which is given by the derivative of the position function) varies. The graph `f1` has constant velocity, so the position is a straight line with slope $v_0$. The graph `f2` is similar, though for first and last 30 seconds, the car does not move, so must move faster during the time it moves. A more realistic graph would be `f3`. The position increases continuously, as do the others, but the velocity changes more gradually. The initial velocity is less than $v_0$, but eventually gets to be more than $v_0$, then velocity starts to increase less. At no point is the velocity not increasing, for `f3`, the way it is for `f2` after a minute and a half. -The rate of change of the velocity is the acceleration. For `f1` this is zero, for `f2` it is zero as well - when it is defined. However, for `f3` we see the increase in velocity is positive in the first minute, but negative in the second minute. This fact relates to the concavity of the graph. As acceleration is the derivative of velocity, it is the second derivative of position - the graph we see. Where the acceleration is *positive*, the position graph will be concave *up*, where the acceleration is *negative* the graph will be concave *down*. The point $t=1$ is an inflection point, and would be felt by most riders. +The rate of change of the velocity is the acceleration. For `f1` this is zero, for `f2` it is zero as well---when it is defined. However, for `f3` we see the increase in velocity is positive in the first minute, but negative in the second minute. This fact relates to the concavity of the graph. As acceleration is the derivative of velocity, it is the second derivative of position - the graph we see. Where the acceleration is *positive*, the position graph will be concave *up*, where the acceleration is *negative* the graph will be concave *down*. The point $t=1$ is an inflection point, and would be felt by most riders. ## Questions @@ -659,12 +763,16 @@ The rate of change of the velocity is the acceleration. For `f1` this is zero, f Consider this graph: - +::: {#fig-plot-airyai-over-minus-5-0-ask-questions} ```{julia} -plot(airyai, -5, 0) # airyai in `SpecialFunctions` loaded with `CalculusWithJulia` +#| echo: false +plot(airyai, -5, 0; legend=false) # airyai in `SpecialFunctions` loaded with `CalculusWithJulia` ``` -On what intervals (roughly) is the function positive? +Plot of $f(x)$ over $[-5, 0]$ +::: + +On what intervals (roughly) is $f(x)$ positive? ```{julia} @@ -675,8 +783,8 @@ choices=[ "``(-5, -4.2)``", "``(-5, -4.2)`` and ``(-2.5, 0)``", "``(-4.2, -2.5)``"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` ###### Question @@ -692,17 +800,18 @@ numericq(0; hint=raw"Any value of $x$ will produce the same answer") ###### Question -Consider this graph: - - +::: {#fig-besselj-1-over-minus5-minus3} ```{julia} #| hold: true #| echo: false import SpecialFunctions: besselj -p = plot(x->besselj(x, 1), -5,-3) +p = plot(x->besselj(x, 1), -5,-3; legend=false) ``` -On what intervals (roughly) is the function negative? +Plot of $f(x)$ over $[-5,3]$ +::: + +Consider the graph in @fig-besselj-1-over-minus5-minus3. On what intervals (roughly) is the function negative? ```{julia} @@ -713,22 +822,24 @@ choices=[ "``(-25.0, 0.0)``", "``(-5.0, -4.0)`` and ``(-4, -3)``", "``(-4.0, -3.0)``"] -answ = 4 -radioq(choices, answ) +answer = 4 +radioq(choices, answer) ``` ###### Question -Consider this graph - +Consider the graph in @fig-besselj-1-over-minus5-minus3-increasing. +::: {#fig-besselj-1-over-minus5-minus3-increasing} ```{julia} #| hold: true #| echo: false plot(x->besselj(x, 21), -5,-3) ``` +Plot of $f(x)$ over $[-5, 3]$ +::: On what interval(s) is this function increasing? @@ -741,12 +852,55 @@ choices=[ "``(-4.7, -3.0)``", "``(-0.17, 0.17)``" ] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` + ###### Question -The function +Consider the following sign chart of a continuous function $f(x)$ over $[0, 2]$ + +```{julia} +#| echo: false +f(x) = 1 + sin(2x) + 4*sin(6x) +``` + +```{julia} +sign_chart(f, 0, 2) +``` + +Is $f(x)$ on the interval $[0, 0.607\cdots]$? + +```{julia} +#| echo: false +choices = ["Yes", "No"] +answer = 1 +buttonq(choices, answer) +``` + +If $f(x)$ positive at $x=1$? + + +```{julia} +#| echo: false +choices = ["Yes", "No"] +answer = 1 +buttonq(choices, answer) +``` + +Do you know $f(x)$ is zero at $0.962916\cdots$? + +```{julia} +#| echo: false +choices = [L"Yes, but only because $f(x)$ is continuous on $[0,2]$ so any sign change must be a zero", + "No, the `find_zero` function may find zeros, infinities, or other places where the sign changes"] +answer = 1 +buttonq(choices, answer) +``` + +###### Question + +@fig-x-over-2-plus-x-square-sin-pi-over-x shows the function $$ f(x) = @@ -756,9 +910,9 @@ f(x) = \end{cases} $$ -is graphed below over $[-1/3, 1/3]$. - +graphed over $[-1/3, 1/3]$. +::: {#fig-x-over-2-plus-x-square-sin-pi-over-x} ```{julia} #| echo: false plt = let @@ -776,32 +930,31 @@ plt = let xs = range(a, b, 10_000) ys = f.(xs) y0,y1 = extrema(ys) - plot(; empty_style..., aspect_ratio=:equal) - plot!([a,b],[0,0]; axis_style...) - plot!([0,0], [y0,y1]; axis_style...) - plot!(xs, f.(xs); line=(:black, 1)) + plt = plot(; empty_style..., aspect_ratio=:equal) + plot!(plt, [a,b],[0,0]; axis_style...) + plot!(plt, [0,0], [y0,y1]; axis_style...) + plot!(plt, xs, f.(xs); line=(:black, 1)) - plot!(xs, x -> x/2 + x^2; line=(:gray, 1, :dot)) - plot!(xs, x -> x/2 - x^2; line=(:gray, 1, :dot)) - plot!(xs, x -> x/2; line=(:gray, 1)) + plot!(plt, xs, x -> x/2 + x^2; line=(:gray, 1, :dot)) + plot!(plt, xs, x -> x/2 - x^2; line=(:gray, 1, :dot)) + plot!(plt, xs, x -> x/2; line=(:gray, 1)) a1 = (1/4 + 1/5)/2 a2 = -(1*1/3 + 4*1/4)/5 - annotate!([ + annotate!(plt, [ (a1, g(a1), text(L"\frac{x}{2} - x^2", 10, :top)), (a1, f(a1), text(L"\frac{x}{2} + x^2", 10, :bottom)), (-1/6, f(1/6), text(L"\frac{x}{2} + x^2\sin(\frac{\pi}{x})", 10, :bottom)) ]) - plot!([-1/6, -1/13.5], [f(1/6), f(-1/13.5)]; axis_style...) + plot!(plt, [-1/6, -1/13.5], [f(1/6), f(-1/13.5)]; axis_style...) + plotly() + plt end plt ``` -```{julia} -#| echo: false -plotly() -nothing -``` +Plot of highly oscillating function near $0$ +::: This function has a derivative at $0$ that is *positive* @@ -823,25 +976,30 @@ diff(f(x), x) ```{julia} #| echo: false choices = ["Yes", "No"] -answer = 1 -buttonq(choices, answer; explanation=raw""" -The slope of the tangent line away from $0$ oscillates from positive to negative at every rational number of the form $1/n$ due to the $\cos(\pi/x)$ term, so it is neither going just up or down around $0$. (This example comes from @Angenent.) -""") +answer = 2 +explanation=raw""" +The slope of the tangent line away from $0$ oscillates from positive to negative at every rational number of the form $1/n$ due to the $\cos(\pi/x)$ term, so it is neither going just up or down around $0$. (This example comes from @Angenent. +""" +buttonq(choices, answer; explanation) ``` ###### Question -Consider this graph +Consider the graph in @fig-plot-1-over-1-plus-x-square-over-minus3-3-question-concave-up. +::: {#fig-plot-1-over-1-plus-x-square-over-minus3-3-question-concave-up} ```{julia} #| hold: true #| echo: false p = plot(x -> 1 / (1+x^2), -3, 3) ``` -On what interval(s) is this function concave up? +Graph of $f(x)$ over $[-3,3]$ +::: + +On what interval(s) is the function concave up? ```{julia} @@ -853,41 +1011,67 @@ choices=[ "``(-0.6, 0.6)``", " ``(-3.0, -0.6)`` and ``(0.6, 3.0)``" ] -answ = 4 -radioq(choices, answ) +answer = 4 +radioq(choices, answer) ``` ###### Question -Consider the following figure of a graph of $f$: +Consider the `sign_chart` of $f''(x)$ over $[0, 4]$: +```{julia} +#| echo: false +f(x) = x^6/30 - 9*x^5/20 + 29*x^4/12 - 13*x^3/2 + 9*x^2 +``` + +```{julia} +sign_chart(f'', 0, 4) +``` + +On what intervals is $f(x)$ *concave up*. Select all that are true? + +```{julia} +#| echo: false +choices = [L"(0,1)", + L"(1, 2)", + L"(2, 3)", + L"(3,4)"] +answer = [1, 3, 4] +multiq(choices, answer; keep_order=true) +``` + + +###### Question + +Consider @fig-x-tanh-exp-x-over-minus5-1 of a graph of $f$ where zeros of the function and its first and second derivative have been marked. + +::: {#fig-x-tanh-exp-x-over-minus5-1} ```{julia} #| echo: false let gr() ex(x) = x * tanh(exp(x)) a, b = -5, 1 - plot(ex, a, b, legend=false, + plt = plot(ex, a, b, legend=false, axis=([], false), line=(:black, 2) ) - plot!([a-.1, b+.1], [0,0], line=(:gray,1), arrow=true, side=:head) + plot!(plt, [a-.1, b+.1], [0,0], line=(:gray,1), arrow=true, side=:head) zs = find_zeros(ex, (a, b)) cps = find_zeros(ex', (a, b)) ips = find_zeros(ex'', (a, b)) - scatter!(zs, ex.(zs), fill=(:black,), marker=(8, :circle)) - scatter!(cps, ex.(cps), fill=(:green,), marker=(8, :diamond)) - scatter!(ips, ex.(ips), fill=(:brown3,), marker=(8,:star5)) + scatter!(plt, zs, ex.(zs), fill=(:black,), marker=(8, :circle)) + scatter!(plt, cps, ex.(cps), fill=(:green,), marker=(8, :diamond)) + scatter!(plt, ips, ex.(ips), fill=(:brown3,), marker=(8,:star5)) + plotly() + plt end ``` -```{julia} -#| echo: false -plotly() -nothing -``` +Plot of $f(x)$ with various points on the curve marked with symbols +::: The black circle denotes what? @@ -897,8 +1081,8 @@ The black circle denotes what? choices = [raw"A zero of $f$", raw"A critical point of $f$", raw"An inflection point of $f$"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` The green diamond denotes what? @@ -909,8 +1093,8 @@ The green diamond denotes what? choices = [raw"A zero of $f$", raw"A critical point of $f$", raw"An inflection point of $f$"] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` @@ -922,8 +1106,8 @@ The red stars denotes what? choices = [raw"Zeros of $f$", raw"Critical points of $f$", raw"Inflection points of $f$"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` @@ -950,8 +1134,8 @@ choices = [ "That the critical point at ``0`` is a relative maximum", "That the critical point at ``0`` is a relative minimum" ] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -969,8 +1153,8 @@ choices = [ " ``f(x)`` is continuous and differentiable at ``2`` and has a critical point", " ``f(x)`` is continuous and differentiable at ``2`` and has a critical point that is a relative minimum by the second derivative test" ] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -1035,8 +1219,8 @@ choices = [ "No, the second derivative test is possibly inconclusive", "Yes" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1053,82 +1237,111 @@ choices = [ "No, the second derivative test is possibly inconclusive if ``c=0``, but otherwise yes", "Yes" ] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question - +::: {#fig-graph-exp-minus-x-times-sin-pi-x-over-0-3} ```{julia} #| echo: false let f(x) = exp(-x) * sin(pi*x) - plot(D(f), 0, 3) + p1 = plot(f, 0, 3; label="f") + p2 = plot(f', 0, 3; label="g") + plot(p1, p2) end ``` -The graph shows $f'(x)$. Is it possible that $f(x) = e^{-x} \sin(\pi x)$? +Plots of $f(x)$ and $g(x)$ over $[0,3]$ +::: + +@fig-graph-exp-minus-x-times-sin-pi-x-over-0-3 shows a graph of $f(x)$ and $g(x)$. Is it possible that $f'(x) = g'(x)$? ```{julia} #| hold: true #| echo: false -yesnoq(true) +choices = ["Yes", "No"] +answer = 1 +explanation = L"Compare features like critical points, increasing decreasing of $f(x)$ to $g(x)$" +buttonq(choices, answer; explanation) ``` -(Plot $f(x)$ and compare features like critical points, increasing decreasing to that indicated by $f'$ through the graph.) + ###### Question - +::: {#fig-plot-second-derivative-of-x-4-minus-3-x-3-minus-2-x-plus-4} ```{julia} #| hold: true #| echo: false f(x) = x^4 - 3x^3 - 2x + 4 -plot(D(f,2), -2, 4) +p1 = plot(f, -2, 4; label="f") +p2 = plot(f'', -2, 4; label="g") +plot(p1, p2) ``` -The graph shows $f'(x)$. Is it possible that $f(x) = x^4 - 3x^3 - 2x + 4$? +Plots of $f(x)$ and $g(x)$ over $[-2, 4]$ +::: + +@fig-plot-second-derivative-of-x-4-minus-3-x-3-minus-2-x-plus-4 shows the graph of $f(x)$ and $g(x)$. Is it possible that $f''(x) = g(x)$? ```{julia} #| hold: true #| echo: false -yesnoq("no") -``` - -###### Question - - -```{julia} -#| hold: true -#| echo: false -f(x) = (1+x)^(-2) -plot(D(f,2), 0,2) -``` - -The graph shows $f''(x)$. Is it possible that $f(x) = (1+x)^{-2}$? - - -```{julia} -#| hold: true -#| echo: false -yesnoq("yes") +choices = ["Yes", "No"] +answer = 1 +explanation = L"Compare features like critical points, increasing decreasing of $f(x)$ with $g(x)$." +buttonq(choices, answer; explanation) +``` + +###### Question + +::: {#fig-plot-f-1-plus-x-to-minus-2} +```{julia} +#| echo: false +let + f(x) = (1+x)^(-2) + p1 = plot(f, 0, 2; label="f") + p2 = plot(f'', 0, 2; label="g") + plot(p1, p2) +end +``` + +Plots of $f(x)$ and $g(x)$ over $[0, 2]$ +::: + +Is it possible that $f''(x) = g(x)$? + + +```{julia} +#| hold: true +#| echo: false +choices = ["Yes", "No"] +answer = 1 +explanation = L"Compare features like critical points, increasing decreasing of $f(x)$ with $g(x)$." +buttonq(choices, answer; explanation) ``` ###### Question +::: {#fig-plot-f-p-x-minus-1-times-x-minus-2-squared-times-x-minus-3-squared} ```{julia} #| hold: true #| echo: false f_p(x) = (x-1)*(x-2)^2*(x-3)^2 -plot(f_p, 0.75, 3.5) +plot(rangeclamp(f_p, 1), 0.75, 3.5; label="f'") ``` -This plot shows the graph of $f'(x)$. What is true about the critical points and their characterization? +Plot of $f'(x)$ over $[0.75, 3.5]$ +::: + +@fig-plot-f-p-x-minus-1-times-x-minus-2-squared-times-x-minus-3-squared shows the graph of $f'(x)$. What is true about the critical points of $f(x)$ and their characterization? ```{julia} @@ -1140,8 +1353,8 @@ choices = [ "The critical points are at ``x=1`` (a relative minimum), ``x=2`` (not a relative extrema), and ``x=3`` (a relative minimum).", "The critical points are at ``x=1`` (a relative minimum), ``x=2`` (a relative minimum), and ``x=3`` (a relative minimum).", ] -answ=1 -radioq(choices, answ) +answer=1 +radioq(choices, answer) ``` ###### Question @@ -1158,8 +1371,8 @@ choices = [ "The function is decreasing over ``(-\\infty, 1)`` and increasing over ``(1, \\infty)``", "The function is negative over ``(-\\infty, 1)`` and positive over ``(1, \\infty)``", ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1174,8 +1387,8 @@ While driving we accelerate to get through a light before it turns red. However, choices = ["A zero of the function", "A critical point for the function", "An inflection point for the function"] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -1194,8 +1407,8 @@ This accurately summarizes how the term is used outside of math books. Does it a #| echo: false choices = ["Yes. Same words, same meaning", """No, but it is close. An inflection point is when the *acceleration* changes from positive to negative, so if "results" are about how a company's rate of change is changing, then it is in the ballpark."""] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question @@ -1209,6 +1422,72 @@ The function $f(x) = x^3 + x^4$ has a critical point at $0$ and a second derivat #| echo: false choices = ["As ``x^3`` has no extrema at ``x=0``, neither will ``f``", "As ``x^4`` is of higher degree than ``x^3``, ``f`` will be ``U``-shaped, as ``x^4`` is."] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) +``` + +##### Question + +Is this a valid alternate definition of concave up on $[a,b]$: + +If for any $\alpha$ in $[0,1]$: + +$$ +f((1-\alpha)a + \alpha b) \geq (1 - \alpha) f(a) + \alpha f(b). +$$ + +```{julia} +#| echo: false +choices = ["Yes", "No"] +answer = 1 +explanation = L"Yes, this is an alternate form where the slope, where the right hand side interpolates linearly between $f(a)$ and $f(b)$, like the secant line" +buttonq(choices, answer; explanation) +``` + + +##### Question + +For a function with many derivatives, we can define the *order* of a zero of $f(c)$ as follows: + +::: {.definition title="Order of a zero of a many times differentiable function"} + +If $f(c) = 0$ and $f^{(j)}(c) = 0$ for all $1 \leq j < k$, but $f^{(k)}(c) \neq 0$ the $k$ is order of the zero, $c$. + +::: + +A *simple* zero is one where $k=1$, or the first derivative is non-zero (and continuous around $c$, but assumption of many times differentiable). + + +Does this statement make sense: + +> Close enough to a simple zero, the derivative either positive or negative, but not zero, so that the function increases or decreases through the zero $c$ + +```{julia} +#| echo: false +choices = ["Yes", "No"] +answer = 1 +buttonq(choices, answer) +``` + +Suppose $f(x) = a_0 + a_1 (x - c) + a_2 (x - c)^2 + a_3 (x-c)^3$ + +If $f(c) = 0$, then what is the value of $a_0$? + +```{julia} +#| echo: false +numericq(0) +``` + +If $f'(c) = 0$ what is the value of $a_1$? + +```{julia} +#| echo: false +numericq(0) +``` + +Under these assumptions, can $f(x)$ be arranged as $f(x) = (x-c)^2 \cdot (a2 + a_3(x-c))$? + +```{julia} +#| echo: false +yesnoq(true) ``` diff --git a/quarto/derivatives/implicit_differentiation.qmd b/quarto/derivatives/implicit_differentiation.qmd index 834381d..f757d8c 100644 --- a/quarto/derivatives/implicit_differentiation.qmd +++ b/quarto/derivatives/implicit_differentiation.qmd @@ -1,6 +1,5 @@ # Implicit differentiation - {{< include ../_common_code.qmd >}} This section uses these add-on packages: @@ -39,23 +38,26 @@ There are a few options for plotting equations in `Julia`. We will use a functi To plot an implicit equation using `implicit_plot` requires expressing the relationship in terms of a function, and then plotting the equation `f(x,y) = 0`. In practice this simply requires all the terms be moved to one side of an equals sign. -To plot the circle of radius $2$, or the equations $x^2 + y^2 = 2^2$ we would move all terms to one side $x^2 + y^2 - 2^2 = 0$ and then express the left hand side through a function: +To plot the circle of radius $2$, or the equations $x^2 + y^2 = 2^2$ we would move all terms to one side $x^2 + y^2 - 2^2 = 0$ and then express the left hand side through a function with two arguments: ```{julia} f(x,y) = x^2 + y^2 - 2^2 ``` -This function is then is passed to the `implicit_plot` function, which works with `Plots` to render the graphic: - +This function is then is passed to the `implicit_plot` function, which works with `Plots` to render the graphic in @fig-implicit-plot-of-ellipse-with-radius-2. +::: {#fig-implicit-plot-of-ellipse-with-radius-2} ```{julia} implicit_plot(f) ``` +Plot of equation involving $f(x,y)$ created with `implicit_plot`. +::: + :::{.callout-note} ## Note -The `f` is a function of *two* variables, used here to express one side of an equation. `Julia` makes this easy to do - just make sure two variables are in the signature of `f` when it is defined. Using functions like this, we can express our equation in the form $f(x,y) = c$ or, more generally, as $f(x,y) = g(x,y)$. The latter of which can be expressed as $h(x,y) = f(x,y) - g(x,y) = 0$. That is, only the form $f(x,y)=0$ is needed to represent an equation. +The `f` is a function of *two* variables, used here to express one side of an equation. `Julia` makes this easy to do---just make sure two variables are in the signature of `f` when it is defined. Using functions like this, we can express our equation in the form $f(x,y) = c$ or, more generally, as $f(x,y) = g(x,y)$. The latter of which can be expressed as $h(x,y) = f(x,y) - g(x,y) = 0$. That is, only the form $f(x,y)=0$ is needed to represent an equation. ::: @@ -65,7 +67,7 @@ There are two different styles in `Julia` to add simple plot recipes. `ImplicitP ::: -Of course, more complicated equations are possible and the steps are similar - only the function definition is more involved. For example, the [Devils curve](http://www-groups.dcs.st-and.ac.uk/~history/Curves/Devils.html) has the form +Of course, more complicated equations are possible and the steps are similar---only the function definition is more involved. For example, the [Devil's curve](http://www-groups.dcs.st-and.ac.uk/~history/Curves/Devils.html) has the form $$ @@ -74,7 +76,7 @@ $$ Here we draw the curve for a particular choice of $a$ and $b$. For illustration purposes, a narrower viewing window is specified below using `xlims` and `ylims`: - +::: {#fig-implicit-plot-of-devils-curve} ```{julia} #| hold: true a,b = -1,2 @@ -82,6 +84,9 @@ f(x,y) = y^4 - x^4 + a*y^2 + b*x^2 implicit_plot(f; xlims=(-3,3), ylims=(-3,3), legend=false) ``` +Plot of Devil's curve for a certain choice of parameter +::: + ## Tangent lines, implicit differentiation @@ -91,7 +96,7 @@ The graph $x^2 + y^2 = 1$ has well-defined tangent lines at all points except $( In general though, we may not be able to solve for $y$ in terms of $x$. What then? -The idea is to *assume* that $y$ is representable by some function of $x$. This makes sense, moving on the curve from $(x,y)$ to some nearby point, means changing $x$ will cause some change in $y$. This assumption is only made *locally* - basically meaning a complicated graph is reduced to just a small, well-behaved, section of its graph. +The idea is to *assume* that $y$ is representable by some function of $x$. This makes sense, moving on the curve from $(x,y)$ to some nearby point, means changing $x$ will cause some change in $y$. This assumption is only made *locally*---basically meaning a complicated graph is reduced to just a small, well-behaved, section of its graph. ::: {#fig-well-behaved-section} @@ -153,10 +158,10 @@ $$ This says the slope of the tangent line depends on the point $(x,y)$ through the formula $-x/y$. -As a check, we compare to what we would have found had we solved for $y= \sqrt{1 - x^2}$ (for $(x,y)$ with $y \geq 0$). We would have found: $dy/dx = 1/2 \cdot 1/\sqrt{1 - x^2} \cdot (-2x)$. Which can be simplified to $-x/y$. This should show that the method above - assuming $y$ is a function of $x$ and differentiating - is not only more general, but can even be easier. +As a check, we compare to what we would have found had we solved for $y= \sqrt{1 - x^2}$ (for $(x,y)$ with $y \geq 0$). We would have found: $dy/dx = 1/2 \cdot 1/\sqrt{1 - x^2} \cdot (-2x)$. Which can be simplified to $-x/y$. This should show that the method above---assuming $y$ is a function of $x$ and differentiating---is not only more general, but can even be easier. -The name - *implicit differentiation* - comes from the assumption that $y$ is implicitly defined in terms of $x$. According to the [Implicit Function Theorem](http://en.wikipedia.org/wiki/Implicit_function_theorem) the above method will work provided the curve has sufficient smoothness near the point $(x,y)$. (Continuously differentiable and non vanishing derivative in $y$.) +The name---*implicit differentiation*---comes from the assumption that $y$ is implicitly defined in terms of $x$. According to the [Implicit Function Theorem](http://en.wikipedia.org/wiki/Implicit_function_theorem) the above method will work provided the curve has sufficient smoothness near the point $(x,y)$. (Continuously differentiable and non vanishing derivative in $y$.) ##### Examples @@ -169,20 +174,23 @@ $$ x^2y + a\cdot b \cdot y - a^2 \cdot x = 0, \quad a\cdot b > 0. $$ -For $a = 2, b=1$ we have the graph: - +For $a = 2, b=1$ we have the graph in @fig-serpentine-a-2-b-1. +::: {#fig-serpentine-a-2-b-1} ```{julia} #| hold: true a, b = 2, 1 f(x,y) = x^2*y + a * b * y - a^2 * x implicit_plot(f; legend=false) -x₀, y₀ = 0, 0 -m = (a^2 - 2x₀*y₀) / (a*b + x₀^2) -plot!(x -> y₀ + m*(x - x₀), -1, 1) +x0, y0 = 0, 0 +m = (a^2 - 2x0*y0) / (a*b + x0^2) +plot!(x -> y0 + m*(x - x0), -1, 1) ``` +Plot of serpentine equation $x^2y + 2 \cdot 1 \cdot y - 2^2 \cdot x$ along with a tangent line. +::: + To the plot we added a tangent line at $(0,0)$. We can see that at each point in the viewing window the tangent line exists due to the smoothness of the curve. To find the slope of the tangent line at a point $(x,y)$ the tangent line will have slope $dy/dx$ satisfying: @@ -214,7 +222,7 @@ $$ A graph for $a=3$ shows why it has the name it does: - +::: {#fig-plot-eight-curve{ ```{julia} #| hold: true a = 3 @@ -222,6 +230,9 @@ f(x,y) = x^4 - a^2*(x^2 - y^2) implicit_plot(f; xticks=-5:5) ``` +Plot of eight curve when $a=3$ +::: + The tangent line at $(x,y)$ will have slope, $dy/dx$ satisfying: @@ -277,7 +288,7 @@ Let's see how to add a graph of a tangent line to the graph of an equation. Tang Returning to the equation for a circle, $x^2 + y^2 = 1$, let's look at $(\sqrt{2}/2, - \sqrt{2}/2)$. The derivative is $-x/y$, so the slope at this point is $1$. The line itself has equation $y = b + m \cdot (x-a)$. The following represents this in `Julia`: - +::: {#fig-adding-tangent-line-as-layer} ```{julia} #| hold: true F(x,y) = x^2 + y^2 - 1 @@ -291,7 +302,10 @@ implicit_plot(F, xlims=(-2, 2), ylims=(-2, 2), aspect_ratio=:equal) plot!(tl) ``` -We added *both* the implicit plot of $F$ and the tangent line to the graph at the given point. +Implicit plot of $F(x,y)$ along with a tangent line +::: + +We added *both* the implicit plot of $F$ and the tangent line to @fig-adding-tangent-line-as-layer at the given point. ##### Example @@ -310,19 +324,27 @@ $$ Setting $a=1$ we have the graph: +::: {#fig-folium-of-descartes-plotted-implicitly} ```{julia} a = 1 G(x,y) = x^3 + y^3 - 3*a*x*y implicit_plot(G) ``` +Plot of folium of Descartes when $a=1$ +::: + We can solve for the lower curve, $y$, as a function of $x$, as follows: +::: {#fig-folium-of-descartes-plotted-by-finding-minimum} ```{julia} y1(x) = minimum(find_zeros(y->G(x,y), -10, 10)) # find_zeros from `Roots` ``` +Numerically finding a function for a part of an equation +::: + This gives the lower part of the curve, which we can plot with: @@ -407,7 +429,7 @@ end Let $a = b = c = d = 1$, then $(1,4)$ is a point on the curve. We can draw a tangent line to this point with these commands: - +::: {#fig-implicit-plot-of-trident-of-newton} ```{julia} H = ex(a=>1, b=>1, c=>1, d=>1) x0, y0 = 1, 4 @@ -416,7 +438,10 @@ implicit_plot(lambdify(H); xlims=(-5,5), ylims=(-5,5), legend=false) plot!(y0 + m * (x-x0)) ``` -Basically this includes all the same steps as if done "by hand." Some effort could have been saved in plotting, had values for the parameters been substituted initially, but not doing so shows their dependence in the derivative. +Plot of trident of Newton along with a tangent line for a given point +::: + +Basically this includes all the same steps as if done "by hand". Some effort could have been saved in plotting, had values for the parameters been substituted initially, but not doing so shows their dependence in the derivative. :::{.callout-warning} @@ -444,7 +469,7 @@ $$ 3x^2 - (3y^2 \frac{dy}{dx}) = 0. $$ -We could solve for $dy/dx$ at this point - it always appears as a linear factor - to get: +We could solve for $dy/dx$ at this point---it always appears as a linear factor---to get: $$ @@ -475,18 +500,21 @@ $$ It isn't so pretty, but that's all it takes. -To visualize, we plot implicitly and notice that: +To visualize, we plot implicitly in @fig-implicit-plot-K and notice that: * as we change quadrants from the third to the fourth to the first the concavity changes from down to up to down, as the sign of the second derivative changes from negative to positive to negative; * and that at these inflection points, the "tangent" line is vertical when $y=0$ and flat when $x=0$. - +::: {#fig-implicit-plot-K} ```{julia} K(x,y) = x^3 - y^3 - 3 implicit_plot(K, xlims=(-3, 3), ylims=(-3, 3)) ``` +Implicit plot of cubic equation, $K$ +::: + The same problem can be done symbolically. The steps are similar, though the last step (replacing $x^3 - y^3$ with $3$) isn't done without explicitly asking. @@ -496,8 +524,8 @@ The same problem can be done symbolically. The steps are similar, though the las eqn = K(x,y) eqn1 = eqn(y => u(x)) -dydx = solve(diff(eqn1,x), diff(u(x), x))[1] # 1 solution -d2ydx2 = solve(diff(eqn1, x, 2), diff(u(x),x, 2))[1] # 1 solution +dydx = only(solve(diff(eqn1,x), diff(u(x), x))) # 1 solution +d2ydx2 = only(solve(diff(eqn1, x, 2), diff(u(x),x, 2))) # 1 solution eqn2 = subs(d2ydx2, diff(u(x), x) => dydx, u(x) => y) simplify(eqn2) ``` @@ -514,7 +542,7 @@ In short, both $f \circ g$ and $g \circ f$ are identify functions on their respe The chain rule can be used to give the derivative of an inverse function when applied to $f(f^{-1}(x)) = x$. Solving gives, $[f^{-1}(x)]' = 1 / f'(f^{-1}(x))$. -This is great - if we can remember the rules. If not, sometimes implicit differentiation can also help. +This is great---if we can remember the rules. If not, sometimes implicit differentiation can also help. Consider the inverse function for the tangent, which exists when the domain of the tangent function is restricted to $(-\pi/2, \pi/2)$. The function solves $y = \tan^{-1}(x)$ or $\tan(y) = x$. Differentiating this yields: @@ -566,7 +594,7 @@ This problem illustrates one best done with implicit derivatives. A video showin Well, suppose you hold the rope in two places, which we can take to be $(0,0)$ and $(a,b)$. Then let $(x,y)$ be all the possible positions of the ring that hold the rope taught. Then we have this picture: - +::: {#fig-string-with-a-weight} ```{julia} #| hold: true #| echo: false @@ -594,18 +622,17 @@ let (1/2, 0, text(L"x",:top)), (5/2, 1, text(L"a-x", :top)), (1, -1, text(L"|y|",:right)), - (1+Δ, -1, text(L"b-y",:left)), + (1+Δ, -1/2, text(L"b-y",:left)), (1+2a, -3, text(L"(x,y)",:left)) ]) + plotly() + current() end ``` -```{julia} -#| echo: false -plotly() -nothing -``` +Picture of string with fixed end points weighted down +::: Since the length of the rope does not change, we must have for any admissible $(x,y)$ that: @@ -638,11 +665,14 @@ F₀(x, y) = F₀(x, y, a, b) Our values $(x,y)$ must satisfy $f(x,y) = L$. Let's graph: - +::: {#fig-implicit-plot-of-Fzero-is-an-ellipse} ```{julia} implicit_plot((x,y) -> F₀(x,y) - L, xlims=(-5, 7), ylims=(-5, 7)) ``` +Implicit plot of $F_0 -L$ is an ellipse +::: + The graph is an ellipse, though slightly tilted. @@ -680,7 +710,7 @@ The values of `dydx` depend on any pair $(x,y)$, but our solution must also sati ```{julia} -eqn1 = eqn(x => cps[2]) +eqn1 = eqn(x => last(cps)) ``` We would try to solve `eqn1` for `y` with `solve(eqn1, y)`, but `SymPy` can't complete this problem. Instead, we will approach this numerically using `find_zero` from the `Roots` package. We make the above a function of `y` alone @@ -700,7 +730,7 @@ xstar = N(cps[2](y => ystar, a =>3, b => 3)) Our minimum is at `(xstar, ystar)`, as this graphic shows: - +::: {#fig-implicit-plot-F0-10} ```{julia} tl(x) = ystar + 0 * (x- xstar) @@ -708,6 +738,10 @@ implicit_plot((x,y) -> F₀(x,y,3,3) - 10, xlims=(-4, 7), ylims=(-10, 10)) plot!(tl) ``` +Implicit plot of $F_0 - L$ for a sest of values. + +::: + If you watch the video linked to above, you will see that the surprising fact here is the resting point is such that the angles formed by the rope are the same. Basically this makes the tension in both parts of the rope equal, so there is a static position (if not static, the ring would move and not end in the final position). We can verify this fact numerically by showing the arctangents of the two triangles are the same up to a sign: @@ -731,12 +765,15 @@ Here we try the same problem numerically, using a zero-finding approach to ident Starting with $F(x,y) = \sqrt{x^2 + y^2} + \sqrt{(x-1)^2 + (y-2)^2}$ and $L=3$, we have: - +::: {@fig-implicit-plot-of-F0-L-with-values-found-numerically} ```{julia} F₁(x,y) = F₀(x,y, 1, 2) - 3 # a,b,L = 1,2,3 implicit_plot(F₁) ``` +Plot of $F - L$ +::: + Trying to find the lowest $y$ value we have from the graph it is near $x=0.1$. We can do better. @@ -762,12 +799,15 @@ This shows the smallest value is around $-0.414$ and occurs in the $33$rd positi With this slight modification, we have: - +::: {#fig-plot-of-inverse-F1-numerically-found} ```{julia} f₁(x) = find_zero(y -> F₁(x, y), zero(x)) plot(f₁', -0.5, 1.5) ``` +Plot of $f_1$ found by solving $F_1(x,y(x)) = 0$. +::: + The zero of `f'` is a bit to the right of $0$, say $0.2$; we use `find_zero` again to find it: @@ -891,8 +931,8 @@ choices = [ "``b \\cdot (1 - (x/a)^n)^{1/n}``", "``-(x/a)^n / (y/b)^n``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1013,8 +1053,8 @@ choices = [ "``2xy / (x^2 + a^2)``", "``a^3/(x^2 + a^2)``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` In @fig-witch-agnesi for a given $\theta$ the point $P = (x,y)$ where $x$ is the $x$ value of the intersection of the drawn line with the line $y=a$ and $y$ is the $y$ value of the intersection of the drawn line with the circle $x^2 + y^2 = a^2$. @@ -1026,8 +1066,9 @@ Suppose $O=(0,0)$ and $A=(u,v)$. Which of these formulas is true: choices = [ L"(v+a)/u = 2a/u = \tan(\theta)", L"v/u = a/u = \tan(\theta)" -], -radioq(choices, 1) +] +answer = 1 +radioq(choices, answer) ``` Suppose $B=(u,v)$. Which of these is true: @@ -1055,10 +1096,13 @@ Image number 35 from L'Hospitals calculus book (the first). Given a description #ImageFile(:derivatives, imgfile, caption) nothing ``` +::: {#fig-image-35-lhospital} +![](figures/fcarc-may2016-fig35-350.png){fig-alt="image 35 of L'Hospital's book"} -![Image number 35 from L'Hospitals calculus book (the first). Given a description of the curve, identify the point ``E`` which maximizes the height.](figures/fcarc-may2016-fig35-350.png) +Image number 35 from L'Hospitals calculus book (the first). Given a description of the curve, identify the point ``E`` which maximizes the height. +::: -The figure above shows a problem appearing in L'Hospital's first calculus book. Given a function defined implicitly by $x^3 + y^3 = axy$ (with $AP=x$, $PM=y$ and $AB=a$) find the point $E$ that maximizes the height. In the [AMS feature column](http://www.ams.org/samplings/feature-column/fc-2016-05) this problem is illustrated and solved in the historical manner, with the comment that the concept of implicit differentiation wouldn't have occurred to L'Hospital. +@fig-image-35-lhospital shows a problem appearing in L'Hospital's first calculus book. Given a function defined implicitly by $x^3 + y^3 = axy$ (with $AP=x$, $PM=y$ and $AB=a$) find the point $E$ that maximizes the height. In the [AMS feature column](http://www.ams.org/samplings/feature-column/fc-2016-05) this problem is illustrated and solved in the historical manner, with the comment that the concept of implicit differentiation wouldn't have occurred to L'Hospital. Using Implicit differentiation, find when $dy/dx = 0$. @@ -1068,8 +1112,8 @@ Using Implicit differentiation, find when $dy/dx = 0$. #| hold: true #| echo: false choices = ["``y^2 = 3x/a``", "``y=3x^2/a``", "``y=a/(3x^2)``", "``y^2=a/(3x)``"] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` Substituting the correct value of $y$, above, into the defining equation gives what value for $x$: @@ -1084,8 +1128,8 @@ choices=[ "``x=(1/2) a^3 3^{1/3}``", "``x=(1/3) a^2 2^{1/2}``" ] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question @@ -1118,8 +1162,8 @@ If $y>0$ is the sign positive or negative? #| hold: true #| echo: false choices = ["positive", "negative", "Can be both"] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` If $x>0$ is the sign positive or negative? @@ -1129,8 +1173,8 @@ If $x>0$ is the sign positive or negative? #| hold: true #| echo: false choices = ["positive", "negative", "Can be both"] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` When $x>0$, the graph of the equation is... @@ -1140,6 +1184,6 @@ When $x>0$, the graph of the equation is... #| hold: true #| echo: false choices = ["concave up", "concave down", "both concave up and down"] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` diff --git a/quarto/derivatives/jsxgraph-newton.qmd b/quarto/derivatives/jsxgraph-newton.qmd new file mode 100644 index 0000000..b895212 --- /dev/null +++ b/quarto/derivatives/jsxgraph-newton.qmd @@ -0,0 +1,95 @@ +XXX do we include this? XXX + +--- + + +This interactive graphic (built using [JSXGraph](https://jsxgraph.uni-bayreuth.de/wp/index.html)) allows the adjustment of the point `x0`, initially at $0.85$. Five iterations of Newton's method are illustrated. Different positions of `x0` clearly converge, others will not. + +::: {#fig-jsxgraph-newtons-method} +```{=html} +
+``` + +```{ojs} +//| echo: false +//| output: false + +JXG = require("jsxgraph"); + +// newton's method + +b = JXG.JSXGraph.initBoard('jsxgraph', { + boundingbox: [-3,5,3,-5], axis:true +}); + + +f = function(x) {return x*x*x*x*x - x - 1}; +fp = function(x) { return 4*x*x*x*x - 1}; +x0 = 0.85; + +nm = function(x) { return x - f(x)/fp(x);}; + +l = b.create('point', [-1.5,0], {name:'', size:0}); +r = b.create('point', [1.5,0], {name:'', size:0}); +xaxis = b.create('line', [l,r]) + + +P0 = b.create('glider', [x0,0,xaxis], {name:'x0'}); +P0a = b.create('point', [function() {return P0.X();}, + function() {return f(P0.X());}], {name:''}); + +P1 = b.create('point', [function() {return nm(P0.X());}, + 0], {name:''}); +P1a = b.create('point', [function() {return P1.X();}, + function() {return f(P1.X());}], {name:''}); + +P2 = b.create('point', [function() {return nm(P1.X());}, + 0], {name:''}); +P2a = b.create('point', [function() {return P2.X();}, + function() {return f(P2.X());}], {name:''}); + +P3 = b.create('point', [function() {return nm(P2.X());}, + 0], {name:''}); +P3a = b.create('point', [function() {return P3.X();}, + function() {return f(P3.X());}], {name:''}); + +P4 = b.create('point', [function() {return nm(P3.X());}, + 0], {name:''}); +P4a = b.create('point', [function() {return P4.X();}, + function() {return f(P4.X());}], {name:''}); +P5 = b.create('point', [function() {return nm(P4.X());}, + 0], {name:'x5', strokeColor:'black'}); + + + + + +P0a.setAttribute({fixed:true}); +P1.setAttribute({fixed:true}); +P1a.setAttribute({fixed:true}); +P2.setAttribute({fixed:true}); +P2a.setAttribute({fixed:true}); +P3.setAttribute({fixed:true}); +P3a.setAttribute({fixed:true}); +P4.setAttribute({fixed:true}); +P4a.setAttribute({fixed:true}); +P5.setAttribute({fixed:true}); + +sc = '#000000'; +b.create('segment', [P0,P0a], {strokeColor:sc, strokeWidth:1}); +b.create('segment', [P0a, P1], {strokeColor:sc, strokeWidth:1}); +b.create('segment', [P1,P1a], {strokeColor:sc, strokeWidth:1}); +b.create('segment', [P1a, P2], {strokeColor:sc, strokeWidth:1}); +b.create('segment', [P2,P2a], {strokeColor:sc, strokeWidth:1}); +b.create('segment', [P2a, P3], {strokeColor:sc, strokeWidth:1}); +b.create('segment', [P3,P3a], {strokeColor:sc, strokeWidth:1}); +b.create('segment', [P3a, P4], {strokeColor:sc, strokeWidth:1}); +b.create('segment', [P4,P4a], {strokeColor:sc, strokeWidth:1}); +b.create('segment', [P4a, P5], {strokeColor:sc, strokeWidth:1}); + +b.create('functiongraph', [f, -1.5, 1.5]) + +``` + +Interactive means to explore Newton's method for varying starting points +::: diff --git a/quarto/derivatives/lhospitals_rule.qmd b/quarto/derivatives/lhospitals_rule.qmd index 25d5218..57bb1ce 100644 --- a/quarto/derivatives/lhospitals_rule.qmd +++ b/quarto/derivatives/lhospitals_rule.qmd @@ -17,7 +17,7 @@ using SymPy --- -Let's return to limits of the form $\lim_{x \rightarrow c}f(x)/g(x)$ which have an indeterminate form of $0/0$ if both are evaluated at $c$. The typical example being the limit considered by Euler: +Let's return to limits of the form $\lim_{x \rightarrow c}f(x)/g(x)$ which have an indeterminate form of $0/0$ when are evaluated at $c$. The typical example being the limit considered by Euler: $$ @@ -41,7 +41,7 @@ $$ This is because we know $\sin(\xi) \frac{x}{2}$ has a limit of $0$, when $|\xi| \leq |x|$. -That doesn't look any easier, as we worried about the error term, but if just mentally replaced $\sin(x)$ with $x$ - which it basically is near $0$ - then we can see that the limit should be the same as $x/x$ which we know is $1$ without thinking. +That doesn't look any easier, as we worried about the error term, but if just mentally replaced $\sin(x)$ with $x$---which it basically is near $0$---then we can see that the limit should be the same as $x/x$ which we know is $1$ without thinking. Basically, we found that in terms of limits, if both $f(x)$ and $g(x)$ are $0$ at $c$, that we *might* be able to just take this limit: $(f(c) + f'(c) \cdot(x-c)) / (g(c) + g'(c) \cdot (x-c))$ which is just $f'(c)/g'(c)$. @@ -50,16 +50,17 @@ Basically, we found that in terms of limits, if both $f(x)$ and $g(x)$ are $0$ a Wouldn't that be nice? We could find difficult limits just by differentiating the top and the bottom at $c$ (and not use the messy quotient rule). -Well, in fact that is more or less true, a fact that dates back to [L'Hospital](http://en.wikipedia.org/wiki/L%27H%C3%B4pital%27s_rule) - who wrote the first textbook on differential calculus - though this result is likely due to one of the Bernoulli brothers. +Well, in fact that is more or less true, a fact that dates back to [L'Hospital](http://en.wikipedia.org/wiki/L%27H%C3%B4pital%27s_rule) - who wrote the first textbook on differential calculus---though this result is likely due to one of the Bernoulli brothers. -::: {.callout-note icon=false} -## L'Hospital's rule +::: {.definition title="L'Hospital's rule"} -Suppose: +Suppose for $f$ and $g$ that: - * that $\lim_{x\rightarrow c+} f(c) =0$ and $\lim_{x\rightarrow c+} g(c) =0$, - * that $f$ and $g$ are differentiable in $(c,b)$, and - * that $g(x)$ exists and is non-zero for *all* $x$ in $(c,b)$, +* $\lim_{x\rightarrow c+} f(c) =0$ and $\lim_{x\rightarrow c+} g(c) =0$, + +* $f$ and $g$ are differentiable in $(c,b)$, and + +* $g(x)$ exists and is non-zero for *all* $x$ in $(c,b)$, then **if** the following limit exists: $\lim_{x\rightarrow c+}f'(x)/g'(x)=L$ it follows that $\lim_{x \rightarrow c+}f(x)/g(x) = L$. @@ -94,7 +95,7 @@ In [Gruntz](http://www.cybertester.com/data/gruntz.pdf), in a reference attribut ##### Examples - * Consider this limit at $0$: $(a^x - 1)/x$. We have $f(x) =a^x-1$ has $f(0) = 0$, so this limit is indeterminate of the form $0/0$. The derivative of $f(x)$ is $f'(x) = a^x \log(a)$ which has $f'(0) = \log(a)$. The derivative of the bottom is also $1$ at $0$, so we have: +* Consider this limit at $0$: $(a^x - 1)/x$. We have $f(x) =a^x-1$ has $f(0) = 0$, so this limit is indeterminate of the form $0/0$. The derivative of $f(x)$ is $f'(x) = a^x \log(a)$ which has $f'(0) = \log(a)$. The derivative of the bottom is also $1$ at $0$, so we have: $$ @@ -104,11 +105,11 @@ $$ :::{.callout-note} ## Note -Why rewrite in the "opposite" direction? Because the theorem's result – $L$ is the limit – is only true if the related limit involving the derivative exists. We don't do this in the following, but did so here to emphasize the need for the limit of the ratio of the derivatives to exist. +Why rewrite in the "opposite" direction? Because the theorem's result---$L$ is the limit---is only true if the related limit involving the derivative exists. We don't do this in the following, but did so here to emphasize the need for the limit of the ratio of the derivatives to exist to apply the theorem. ::: - * Consider this limit: +* Consider this limit: $$ @@ -118,7 +119,7 @@ $$ It too is of the indeterminate form $0/0$. The derivative of the top is $e^x + e^{-x}$, which is $2$ when $x=0$, so the ratio of $f'(0)/g'(0)$ is seen to be $2$. By continuity, the limit of the ratio of the derivatives is $2$. Then by L'Hospital's rule, the limit above is $2$. - * Sometimes, L'Hospital's rule must be applied twice. Consider this limit: +* Sometimes, L'Hospital's rule must be applied twice. Consider this limit: $$ @@ -142,7 +143,7 @@ $$ As $L = 1/2$ for this related limit, it must also be the limit of the original problem, by L'Hospital's rule. - * Our "intuitive" limits can bump into issues. Take for example the limit of $(\sin(x)-x)/x^2$ as $x$ goes to $0$. Using $\sin(x) \approx x$ makes this look like $0/x^2$ which is still indeterminate. (Because the difference is higher order than $x$.) Using L'Hospitals, says this limit will exist (and be equal) if the following one does: +* Our "intuitive" limits can bump into issues. Take for example the limit of $(\sin(x)-x)/x^2$ as $x$ goes to $0$. Using $\sin(x) \approx x$ makes this look like $0/x^2$ which is still indeterminate. (Because the difference is higher order than $x$.) Using L'Hospitals, says this limit will exist (and be equal) if the following one does: $$ @@ -159,7 +160,7 @@ $$ So as this limit exists, working backwards, the original limit in question will also be $0$. - * This example comes from the Wikipedia page. It "proves" a discrete approximation for the second derivative. +* This example comes from the Wikipedia page. It "proves" a discrete approximation for the second derivative. Show if $f''(x)$ exists at $c$ and is continuous at $c$, then @@ -180,8 +181,11 @@ We have to be careful, as we differentiate in the $h$ variable, not the $c$ one, $$ -\lim_{h \rightarrow 0} \frac{f''(c+h) - 0 - (-f''(c-h))}{2} = -\lim_{c \rightarrow 0}\frac{f''(c+h) + f''(c-h)}{2} = f''(c). +\begin{align*} +\lim_{h \rightarrow 0} \frac{f''(c+h) - 0 - (-f''(c-h))}{2} +&= \lim_{c \rightarrow 0}\frac{f''(c+h) + f''(c-h)}{2}\\ +&= f''(c). +\end{align*} $$ That last equality follows, as it is assumed that $f''(x)$ exists at $c$ and is continuous, that is, $f''(c \pm h) \rightarrow f''(c)$. @@ -190,7 +194,7 @@ That last equality follows, as it is assumed that $f''(x)$ exists at $c$ and is The expression above finds use when second derivatives are numerically approximated. (The middle expression is the basis of the central-finite difference approximation to the derivative.) - * L'Hospital himself was interested in this limit for $a > 0$ ([math overflow](http://mathoverflow.net/questions/51685/how-did-bernoulli-prove-lh%C3%B4pitals-rule)) +* L'Hospital himself was interested in this limit for $a > 0$ ([math overflow](http://mathoverflow.net/questions/51685/how-did-bernoulli-prove-lh%C3%B4pitals-rule)) $$ @@ -241,9 +245,9 @@ limit(f(x)/g(x), x => a) A first proof of L'Hospital's rule takes advantage of Cauchy's [generalization](http://en.wikipedia.org/wiki/Mean_value_theorem#Cauchy.27s_mean_value_theorem) of the mean value theorem to two functions. Suppose $f(x)$ and $g(x)$ are continuous on $[c,b]$ and differentiable on $(c,b)$. On $(c,x)$, $c < x < b$ there exists a $\xi$ with $f'(\xi) \cdot (g(x) - g(c)) = g'(\xi) \cdot (f(x) - f(c))$. In our formulation, both $f(c)$ and $g(c)$ are zero, so we have, provided we know that $g(x)$ is non zero, that $f(x)/g(x) = f'(\xi)/g'(\xi)$ for some $\xi$, $c < \xi < x$. That the right-hand side has a limit as $x \rightarrow c+$ is true by the assumption that the limit of the ratio of the derivatives exists. (The $\xi$ part can be removed by considering it as a composition of a function going to $c$.) Thus the right limit of the ratio $f/g$ is known. ---- +::: {#fig-lhospitals-picture-graph-animation} ```{julia} #| echo: false #| cache: true @@ -277,14 +281,6 @@ function lhopitals_picture_graph(n) end caption = raw""" -Geometric interpretation of -``L=\lim_{x \rightarrow 0} x^2 / (\sqrt{1 + x} - 1 - x^2)``. -At ``0`` this limit is indeterminate of the form -``0/0``. The value for a fixed ``x`` can be seen as the slope of a secant -line of a parametric plot of the two functions, plotted as -``(g, f)``. In this figure, the limiting "tangent" line has ``0`` slope, -corresponding to the limit ``L``. In general, L'Hospital's rule is -nothing more than a statement about slopes of tangent lines. """ @@ -302,7 +298,18 @@ gif(anim, imgfile, fps = 1) end ``` -## Generalizations +Geometric interpretation of +$L=\lim_{x \rightarrow 0} x^2 / (\sqrt{1 + x} - 1 - x^2)$. +At $0$ this limit is indeterminate of the form +$0/0$. The value for a fixed $x$ can be seen as the slope of a secant +line of a parametric plot of the two functions, plotted as +$(g, f)$. In this figure, the limiting "tangent" line has $0$ slope, +corresponding to the limit $L$. In general, L'Hospital's rule is +nothing more than a statement about slopes of tangent lines. + +::: + +## Direct generalizations to the $\infty/\infty$ case and the $c=\infty$ case L'Hospital's rule generalizes to other indeterminate forms, in particular the indeterminate form $\infty/\infty$ can be proved at the same time as $0/0$ with a more careful [proof](http://en.wikipedia.org/wiki/L%27H%C3%B4pital%27s_rule#General_proof). diff --git a/quarto/derivatives/linearization.qmd b/quarto/derivatives/linearization.qmd index f2c1410..f316209 100644 --- a/quarto/derivatives/linearization.qmd +++ b/quarto/derivatives/linearization.qmd @@ -20,7 +20,7 @@ using DualNumbers --- -The derivative of $f(x)$ has the interpretation as the slope of the tangent line. The tangent line is the line that best approximates the function at the point. +The derivative of $f(x)$, when it exists, is the slope of the tangent line; the tangent line being the line that best approximates the function at the point. Using the point-slope form of a line, we see that the tangent line to the graph of $f(x)$ at $(c,f(c))$ is given by: @@ -48,17 +48,20 @@ tangent(sin, pi/4) ``` -We make some graphs with tangent lines: - +In @fig-plot-x-squared-two-tangent-lines-at-minus-1-2 we make some graphs with tangent lines. +::: {#fig-plot-x-squared-two-tangent-lines-at-minus-1-2} ```{julia} #| hold: true f(x) = x^2 -plot(f, -3, 3) -plot!(tangent(f, -1)) -plot!(tangent(f, 2)) +plot(f, -3, 3; label="f", ylims=(-10, 10)) +plot!(tangent(f, -1); label="tangent line at -1") +plot!(tangent(f, 2); label="tangent line at 2") ``` +Plot of $f(x) = x^2$ over $[-3,3]$ with tangent lines drawn at $x=2$ and $x=-1$ +::: + The graph shows that near the point, the line and function are close, but this need not be the case away from the point. We can express this informally as @@ -69,10 +72,13 @@ $$ with the understanding this applies for $x$ "close" to $c$. -Usually for the applications herein, instead of $x$ and $c$ the two points are $x+\Delta_x$ and $x$. This gives: +Usually for the applications herein, instead of $x$ and $c$ the two points are labeled $x+\Delta_x$ and $x$. This gives: +::: {.definition title="Linearization"} -> *Linearization*: $\Delta_y = f(x +\Delta_x) - f(x) \approx f'(x) \Delta_x$, for small $\Delta_x$. +$\Delta_y = f(x +\Delta_x) - f(x) \approx f'(x) \Delta_x$, for small $\Delta_x$. + +::: @@ -85,15 +91,18 @@ This section gives some implications of this fact and quantifies what "close" ca There are several approximations that are well known in physics, due to their widespread usage: - * That $\sin(x) \approx x$ around $x=0$: - +* That $\sin(x) \approx x$ around $x=0$: +::: {#fig-plot-sin-tangent-line-at-0} ```{julia} #| hold: true -plot(sin, -pi/2, pi/2) -plot!(tangent(sin, 0)) +plot(sin, -pi/2, pi/2; label="sin") +plot!(tangent(sin, 0); label="tangent line at 0") ``` +Plot of $\sin(x)$ and its tangent line at $0$ +::: + Symbolically: @@ -105,16 +114,19 @@ f(x) = sin(x) f(c) + diff(f(x),x)(c) * (x - c) ``` - * That $\log(1 + x) \approx x$ around $x=0$: - +* That $\log(1 + x) \approx x$ around $x=0$: +::: {#fig-plot-log1p-x-and-tangent-at-0} ```{julia} #| hold: true f(x) = log(1 + x) -plot(f, -1/2, 1/2) -plot!(tangent(f, 0)) +plot(f, -1/2, 1/2; label="sin") +plot!(tangent(f, 0); label="tangent line at 0") ``` +Plot of $f(x) = log(1 + x)$ and its tangent line at $x=0$ +::: + Symbolically: @@ -129,16 +141,19 @@ f(c) + diff(f(x),x)(c) * (x - c) (The `log1p` function implements a more accurate version of this function when numeric values are needed.) - * That $1/(1-x) \approx 1+x$ around $x=0$: - +* That $1/(1-x) \approx 1+x$ around $x=0$: +::: {#fig-plot-1-over-1-minus-x-over-minus-one-half-one-half} ```{julia} #| hold: true f(x) = 1/(1-x) -plot(f, -1/2, 1/2) -plot!(tangent(f, 0)) +plot(f, -1/2, 1/2; label="f") +plot!(tangent(f, 0); label="tangent line at 0") ``` +Plot of $1/(1-x)$ and its tangent line at $x=0$ +::: + Symbolically: @@ -150,17 +165,21 @@ f(x) = 1 / (1 - x) f(c) + diff(f(x),x)(c) * (x - c) ``` - * That $(1+x)^n \approx 1 + nx$ around $x = 0$. For example, with $n=5$ - +* That $(1+x)^n \approx 1 + nx$ around $x = 0$. For example, with $n=5$ +::: {#fig-plot-1-plus-x-to-the-n-and-tangent-line} ```{julia} #| hold: true n = 5 f(x) = (1+x)^n # f'(0) = n = n(1+x)^(n-1) at x=0 -plot(f, -1/2, 1/2) -plot!(tangent(f, 0)) +plot(f, -1/2, 1/2; label="f") +plot!(tangent(f, 0); label="tangent line at 0") ``` +Plot of $f(x) = (1 + x)^5$ and its tangent line at $0$ +::: + + Symbolically: @@ -227,7 +246,7 @@ plotly() nothing ``` -The plot in @fig-tangent-dy-dx shows a tangent line with slope $dy/dx$ and the actual change in $y$, $\Delta y$, for some specified $\Delta x$ at a point $(c,f(c))$. The small gap above the sine curve is the error were the value of the sine approximated using the drawn tangent line. We can see that approximating the value of $\Delta y = \sin(c+\Delta x) - \sin(c)$ with the often easier to compute $(dy/dx) \cdot \Delta x = f'(c)\Delta x$ - for small enough values of $\Delta x$ - is not going to be too far off provided $\Delta x$ is not too large. +The plot in @fig-tangent-dy-dx shows a tangent line with slope $dy/dx$ and the actual change in $y$, $\Delta y$, for some specified $\Delta x$ at a point $(c,f(c))$. The small gap above the sine curve is the error were the value of the sine approximated using the drawn tangent line. We can see that approximating the value of $\Delta y = \sin(c+\Delta x) - \sin(c)$ with the often easier to compute $(dy/dx) \cdot \Delta x = f'(c)\Delta x$---for small enough values of $\Delta x$---is not going to be too far off provided $\Delta x$ is not too large. This approximation is known as linearization. It can be used both in theoretical computations and in practical applications. To see how effective it is, we look at some examples. @@ -389,7 +408,7 @@ Vₛ(r) = 4/3 * pi * r^3 A *simple* pendulum is comprised of a massless "bob" on a rigid "rod" of length $l$. The rod swings back and forth making an angle $\theta$ with the perpendicular. At rest $\theta=0$, here we have $\theta$ swinging with $\lvert\theta\rvert \leq \theta_0$ for some $\theta_0$. -According to [Wikipedia](http://tinyurl.com/yz5sz7e) - and many introductory physics book - while swinging, the angle $\theta$ varies with time following this equation: +According to [Wikipedia](http://tinyurl.com/yz5sz7e)---and many introductory physics book---while swinging, the angle $\theta$ varies with time following this equation: $$ @@ -405,7 +424,7 @@ This would be much easier if the second derivative were proportional to the angl [Huygens](http://en.wikipedia.org/wiki/Christiaan_Huygens) used the approximation of $\sin(x) \approx x$, noted above, to say that when the angle is not too big, we have the pendulum's swing obeying $\theta''(t) = -g/l \cdot \theta(t)$. Without getting too involved in why, we can verify by taking two derivatives that $\theta_0\sin(\sqrt{g/l}\cdot t)$ will be a solution to this modified equation. -With this solution, the motion is periodic with constant amplitude (assuming frictionless behaviour), as the sine function is. More surprisingly, the period is found from $T = 2\pi/(\sqrt{g/l}) = 2\pi \sqrt{l/g}$. It depends on $l$ - longer "rods" take more time to swing back and forth - but does not depend on the how wide the pendulum is swinging between (provided $\theta_0$ is not so big the approximation of $\sin(x) \approx x$ fails). This latter fact may be surprising, though not to Galileo who discovered it. +With this solution, the motion is periodic with constant amplitude (assuming frictionless behaviour), as the sine function is. More surprisingly, the period is found from $T = 2\pi/(\sqrt{g/l}) = 2\pi \sqrt{l/g}$. It depends on $l$---longer "rods" take more time to swing back and forth---but does not depend on the how wide the pendulum is swinging between (provided $\theta_0$ is not so big the approximation of $\sin(x) \approx x$ fails). This latter fact may be surprising, though not to Galileo who discovered it. ## Differentials @@ -414,11 +433,11 @@ With this solution, the motion is periodic with constant amplitude (assuming fri The Leibniz notation for a derivative is $dy/dx$ indicating the change in $y$ as $x$ changes. It proves convenient to decouple this using *differentials* $dx$ and $dy$. What do these notations mean? They measure change along the tangent line in same way $\Delta_x$ and $\Delta_y$ measure change for the function. The differential $dy$ depends on both $x$ and $dx$, it being defined by $dy=f'(x)dx$. As tangent lines locally represent a function, $dy$ and $dx$ are often associated with an *infinitesimal* difference. -Taking $dx = \Delta_x$, as in the previous graphic, we can compare $dy$ – the change along the tangent line given by $dy/dx \cdot dx$ – and $\Delta_y$ – the change along the function given by $f(x + \Delta_x) - f(x)$. The linear approximation, $f(x + \Delta_x) - f(x)\approx f'(x)dx$, says that +Taking $dx = \Delta_x$, as in the previous graphic, we can compare $dy$---the change along the tangent line given by $dy/dx \cdot dx$---and $\Delta_y$---the change along the function given by $f(x + \Delta_x) - f(x)$. The linear approximation, $f(x + \Delta_x) - f(x)\approx f'(x)dx$, says that $$ -\Delta_y \approx dy; \quad \text{ when } \Delta_x = dx +\Delta_y \approx dy; \quad \text{ when } \Delta_x = dx. $$ ## The error in approximation @@ -426,8 +445,15 @@ $$ How good is the approximation? Graphically we can see it is pretty good for the graphs we choose, but are there graphs out there for which the approximation is not so good? Of course. However, we can say this (the [Lagrange](http://en.wikipedia.org/wiki/Taylor%27s_theorem) form of a more general Taylor remainder theorem): +::: {.definition title="Lagrange remainder form"} -> Let $f(x)$ be twice differentiable on $I=(a,b)$, $f$ is continuous on $[a,b]$, and $a < c < b$. Then for any $x$ in $I$, there exists some value $\xi$ between $c$ and $x$ such that $f(x) = f(c) + f'(c)(x-c) + (f''(\xi)/2)\cdot(x-c)^2$. +Let $f(x)$ be twice differentiable on $I=(a,b)$, $f$ is continuous on $[a,b]$, and $a < c < b$. Then for any $x$ in $I$, there exists some value $\xi$ between $c$ and $x$ such that + +$$ +f(x) = f(c) + f'(c)(x-c) + \frac{f''(\xi)}{2}\cdot(x-c)^2. +$$ + +::: @@ -437,9 +463,9 @@ That is, the error is basically a constant depending on the concavity of $f$ tim For $\sin(x)$ at $c=0$ we get $\lvert\sin(x) - x\rvert = \lvert-\sin(\xi)\cdot x^2/2\rvert$. Since $\lvert\sin(\xi)\rvert \leq 1$, we must have this bound: $\lvert\sin(x) - x\rvert \leq x^2/2$. -Can we verify? Let's do so graphically: - +Can we verify? We do so graphically in @fig-graph-abs-sin-x-minus-x-and-x-squared-over-2. +::: {#fig-graph-abs-sin-x-minus-x-and-x-squared-over-2} ```{julia} #| hold: true h(x) = abs(sin(x) - x) @@ -448,6 +474,9 @@ plot(h, -2, 2, label="h") plot!(g, -2, 2, label="f") ``` +Plot of $h(x) = \lvert \sin(x) - x \vert$ and $g(x) = x^2/2$ over $[-2, 2]$ +::: + The graph shows a tight bound near $0$ and then a bound over this viewing window. @@ -479,9 +508,9 @@ $$ \lvert f(x) - x\rvert \leq \lvert f''(0)\rvert \cdot \frac{x^2}{2} = x^2/2. $$ -Plotting we verify the bound on $|\log(1+x)-x|$: - +In @fig-abs-log1p-minus-x-and-bound we verify the bound on $|\log(1+x)-x|$. +::: {#fig-abs-log1p-minus-x-and-bound} ```{julia} #| hold: true h(x) = abs(log(1+x) - x) @@ -490,6 +519,9 @@ plot(h, -0.5, 2, label="h") plot!(g, -0.5, 2, label="g") ``` +Plot of $f(x) = \lvert \log(1 + x) - x \rvert$ and a derived upper bound +::: + Again, we see the very close bound near $0$, which widens at the edges of the viewing window. @@ -500,15 +532,17 @@ To see formally why the remainder is as it is, we recall the mean value theorem $$ -\text{error} = h(x) - h(0) = (g(x) - g(0)) \frac{h'(e)}{g'(e)} = -(x^2 - 0) \cdot \frac{f'(e) - f'(0)}{2e} = -x^2 \cdot \frac{1}{2} \cdot f''(\xi). +\begin{align*} +\text{error} &= h(x) - h(0) = (g(x) - g(0)) \frac{h'(e)}{g'(e)} \\ +&= (x^2 - 0) \cdot \frac{f'(e) - f'(0)}{2e} \\ +&= x^2 \cdot \frac{1}{2} \cdot f''(\xi). +\end{align*} $$ The value of $\xi$, from the mean value theorem applied to $f'(x)$, satisfies $0 < \xi < e < x$, so is in $[0,x].$ -### The big (and small) "oh" +### The big---and small--""oh" `SymPy` can find the tangent line expression as a special case of its `series` function (which implements [Taylor series](../taylor_series_polynomials.html)). The `series` function needs an expression to approximate; a variable specified, as there may be parameters in the expression; a value $c$ for *where* the expansion is taken, with default $0$; and a number of terms, for this example $2$ for a constant and linear term. (There is also an optional `dir` argument for one-sided expansions.) @@ -681,165 +715,6 @@ x = Dual(1, 1) We again see `log(x)` being evaluated in line `%6`. The derivative evaluated at `x` is done in line `%11` and this is multiplied by `xp` in line `%12`. -## Curvature - -The curvature of a function will be a topic in a later section on differentiable vector calculus, but the concept of linearization can be used to give an earlier introduction. - - -The tangent line linearizes the function, it being the best linear approximation to the graph of the function at the point. The slope of the tangent line is the limit of the slopes of different secant lines. Consider now, the orthogonal concept, the *normal line* at a point. This is a line perpendicular to the tangent line that goes through the point on the curve. - -At a point $(c,f(c))$ the slope of the normal line is $-1/f'(c)$. - -Following [Kirby C. Smith](https://doi.org/10.2307/2687102), consider two nearby points on the curve of $f$ and suppose we take the two normal lines at $x=c$ and $x=c+h$. These two curves will intersect if the lines are not parallel. To ensure this, assueme that in some neighborhood of $c$, $f'(c)$ is increasing. - -The two normal lines are: - -$$ -\begin{align*} -y &= f(c) - \frac{1}{f'(c)}(x-c)\\ -y &= f(c+h) - \frac{1}{f'(c+h)}(x-(c+h))\\ -\end{align*} -$$ - -Rearranging, we have - -$$ -\begin{align*} --f'(c)(y-f(c)) &= x-c\\ --f'(c+h)(y-f(c+h)) &= x-(c+h) -\end{align*} -$$ - - -Call $R$ the intersection point of the two normal lines: - -```{julia} -#| echo: false -using Roots -let - gr() - f(x) = x^4 - fp(x) = 4x^3 - c = 1/4 - h = 1/4 - nlc(x) = f(c) - 1/fp(c) * (x - c) - nlch(x) = f(c+h) - 1/fp(c+h) * (x-(c+h)) - canvas() = plot(axis=([],false), legend=false, aspect_ratio=:equal) - canvas() - plot!(f, 0, 3/4; line=(3,)) - plot!(nlc; ylim=(-1/4, 1)) - plot!(nlch; ylim=(-1/4, 1)) - Rx = find_zero(x -> nlc(x) - nlch(x), (-10, 10)) - scatter!([c,c+h], f.([c, c+h])) - scatter!([Rx], [nlc(Rx)]) - annotate!([(c, f(c), L"(c,f(c))",:top), - (c+h, f(c+h), L"(c+h, f(c+h))",:bottom), - (Rx, nlc(Rx), L"R",:left)]) -end -``` - -```{julia} -#| echo: false -plotly() -nothing -``` - - -What happens to $R$ as $h \rightarrow 0$? - -We can symbolically solve to see: - -```{julia} -@syms 𝑓() 𝑓p() 𝑓pp() x y c ℎ -n1 = -𝑓p(c)*(y-𝑓(c)) ~ x - c -n2 = -𝑓p(c+ℎ)*(y-𝑓(c+ℎ)) ~ x - (c+ℎ) -R = solve((n1, n2), (x, y)) -``` - - -Taking limits of each term as $h$ goes to zero we have after some notation-simplifying substitution: - -```{julia} -R = Dict(k => limit(R[k], ℎ=>0) for k in (x,y)) -Rx = R[x](limit((𝑓(c+ℎ)-𝑓(c))/ℎ, ℎ=>0) => 𝑓p(c), - limit((𝑓p(c+ℎ)-𝑓p(c))/ℎ, ℎ=>0) => 𝑓pp(c)) -``` - - -and - -```{julia} -Ry = R[y](limit((𝑓(c+ℎ)-𝑓(c))/ℎ, ℎ=>0) => 𝑓p(c), - limit((𝑓p(c+ℎ)-𝑓p(c))/ℎ, ℎ=>0) => 𝑓pp(c)) -``` - -The squared distance, $r^2$, of $R$ to $(c,f(c))$ is then: - -```{julia} -simplify((Rx-c)^2 + (Ry-𝑓(c))^2) -``` - -Or - -$$ -r^2 = \frac{(f'(c)^2 + 1)^3}{f''(c)^2}. -$$ - - -This formula for $r$ is known as the radius of curvature of $f$ -- the radius of the *circle* that best approximates the function at the point. That is, this value reflects the curvature of $f$ supplementing the tangent line or best *linear* approximation to the graph of $f$ at the point. - -```{julia} -#| echo: false -let - gr() - f(x) = x^4 - fp(x) = 4x^3 - fpp(x) = 12x^2 - c = 1/4 - h = 1/4 - nlc(x) = f(c) - 1/fp(c) * (x - c) - nlch(x) = f(c+h) - 1/fp(c+h) * (x-(c+h)) - canvas() = plot(axis=([],false), legend=false, aspect_ratio=:equal) - canvas() - plot!(f, -1/4, 3/4; line=(3,)) - tl(x) = f(c) + f'(c)*(x-c) - plot!(tl, ylim=(-1/4, 3/2); line=(2, :dot)) - - - Rx, Ry = c - fp(c)^3 / fpp(c) - fp(c)/fpp(c), f(c) + (fp(c)^2+1)/fpp(c) - r = (fp(c)^2 + 1)^(3/2) / abs(fpp(c)) - - scatter!([c], f.([c])) - scatter!([Rx], [nlc(Rx)]) - annotate!([(c, f(c), L"(c,f(c))",:top), - (Rx, nlc(Rx), L"R",:left)]) - - - Delta = pi/10 - theta = range(3pi/2 - Delta, 2pi - 3Delta, length=100) - xs, ys = cos.(theta), sin.(theta) - - - plot!(Rx .+ r.*xs, Ry .+ r.*ys) - - x0s, y0s = [Rx,Rx .+ r * first(xs)],[Ry,Ry .+ r * first(ys)] - xns, yns = [Rx,Rx .+ r * last(xs)],[Ry,Ry .+ r * last(ys)] - xcs, ycs = [Rx,c],[Ry,f(c)] - sty = (2, :0.25, :dash) - plot!(x0s, y0s; line=sty); - plot!(xcs, ycs; line=sty); - plot!(xns, yns; line=sty) - -end -``` - -```{julia} -#| echo: false -plotly() -nothing -``` - - ## Questions @@ -857,8 +732,8 @@ choices = [ "``1 + x^{1/2}``", "``1 + (1/2) \\cdot x``", "``1 - (1/2) \\cdot x``"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` ###### Question @@ -875,8 +750,8 @@ choices = [ "``1 + x^k``", "``1 + k \\cdot x``", "``1 - k \\cdot x``"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` ###### Question @@ -894,8 +769,8 @@ choices = [ "``x``", "``1 - x^2/2``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -913,8 +788,8 @@ choices = [ "``1 + x``", "``1 - x``" ] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question @@ -932,8 +807,8 @@ choices = [ "``1 + x``", "``25``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -946,8 +821,8 @@ Let $f(x) = \sqrt{x}$. Find the actual error in approximating $f(26)$ by the val #| hold: true #| echo: false tgent(x) = 5 + x/10 -answ = tgent(1) - sqrt(26) -numericq(answ) +answer = tgent(1) - sqrt(26) +numericq(answer) ``` ###### Question @@ -961,8 +836,8 @@ An estimate of some quantity was $12.34$ the actual value was $12$. What was the #| echo: false est = 12.34 act = 12.0 -answ = (est -act)/act * 100 -numericq(answ) +answer = (est -act)/act * 100 +numericq(answer) ``` ###### Question @@ -978,8 +853,8 @@ tl(x) = x x0 = 5 * pi/180 est = x0 act = sin(x0) -answ = (est -act)/act * 100 -numericq(answ) +answer = (est -act)/act * 100 +numericq(answer) ``` ###### Question @@ -992,8 +867,8 @@ The side length of a square is measured roughly to be $2.0$ cm. The actual lengt #| hold: true #| echo: false tl(x) = 4 + 4x -answ = tl(.2) - 4 -numericq(abs(answ)) +answer = tl(.2) - 4 +numericq(abs(answer)) ``` ###### Question diff --git a/quarto/derivatives/mean_value_theorem.qmd b/quarto/derivatives/mean_value_theorem.qmd index 3700d1a..352f35d 100644 --- a/quarto/derivatives/mean_value_theorem.qmd +++ b/quarto/derivatives/mean_value_theorem.qmd @@ -1,4 +1,4 @@ -# The mean value theorem for differentiable functions +# Implications of Differentiability {{< include ../_common_code.qmd >}} @@ -24,7 +24,21 @@ nothing --- -A function is *continuous* at $c$ if $f(c+h) - f(c) \rightarrow 0$ as $h$ goes to $0$. We can write that as $f(c+h) - f(c) = \epsilon_h$, with $\epsilon_h$ denoting a function going to $0$ as $h \rightarrow 0$. With this notion, differentiability could be written as $f(c+h) - f(c) - f'(c)h = \epsilon_h \cdot h$. This is clearly a more demanding requirement than mere continuity at $c$. +Supposed $\epsilon_h$ is some function going to $0$ as $h \rightarrow 0$ that may be different from line to line. Then we have two somewhat similar characterizations of continuity and differentiability: + + + +A function is *continuous* at $c$ if + +$$ +f(c+h) - f(c) = \epsilon_h. +$$ + +A function is *differentiable* at $c$ if + +$$ +f(c+h) - f(c) - f'(c)h = \epsilon_h \cdot h. +$$ We defined a function to be *continuous* on an interval $I=(a,b)$ if it was continuous at each point $c$ in $I$. Similarly, we define a function to be *differentiable* on the interval $I$ if it is differentiable at each point $c$ in $I$. @@ -33,67 +47,79 @@ We defined a function to be *continuous* on an interval $I=(a,b)$ if it was cont This section looks at properties of differentiable functions. As there is a more stringent definition, perhaps more properties are a consequence of the definition. -## Differentiable is more restrictive than continuous. +## Differentiability implies continuity Let $f$ be a differentiable function on $I=(a,b)$. We see that $f(c+h) - f(c) = f'(c)h + \epsilon_h\cdot h = h(f'(c) + \epsilon_h)$. The right hand side will clearly go to $0$ as $h\rightarrow 0$, so $f$ will be continuous. In short: +::: {.relationship title="Differentiable implies continuous"} -> A differentiable function on $I=(a,b)$ is continuous on $I$. +A differentiable function on $I=(a,b)$ is continuous on $I$. + +::: Is it possible that all continuous functions are differentiable? -The fact that the derivative is related to the tangent line's slope might give an indication that this won't be the case - we just need a function which is continuous but has a point with no tangent line. The usual suspect is $f(x) = \lvert x\rvert$ at $0$. - +The fact that the derivative is related to the tangent line's slope might give an indication that this won't be the case - we just need a function which is continuous but has a point with no tangent line. The usual suspect is $f(x) = \lvert x\rvert$ at $0$, plotted around $0$ in @fig-plot-abs-over-minus1-1-not-diff-at-0. +::: {#fig-plot-abs-over-minus1-1-not-diff-at-0} ```{julia} -#| hold: true +#| echo: false f(x) = abs(x) plot(f, -1,1) ``` +Plot of $f(x) = \lvert x \rvert$ over $[-1, 1]$. This function does not have a tangent line at $x=0$. +::: + We can see formally that the secant line expression will not have a limit when $c=0$ (the left limit is $-1$, the right limit $1$). But more insight is gained by looking at the shape of the graph. At the origin, the graph always is vee-shaped. There is no linear function that approximates this function well. The function is just not smooth enough, as it has a kink. -There are other functions that have kinks. These are often associated with powers. For example, at $x=0$ this function will not have a derivative: - +There are other functions that have kinks. These are often associated with powers. For example, at $x=0$ the function $f(x) = x^{2/3}$ (@fig-plot-x-2-thirds-over-minus1-1) will not have a derivative at $x=0$. +::: {#fig-plot-x-2-thirds-over-minus1-1} ```{julia} -#| hold: true +#| echo: false f(x) = (x^2)^(1/3) plot(f, -1, 1) ``` -Other functions have tangent lines that become vertical. The natural slope would be $\infty$, but this isn't a limiting answer (except in the extended sense we don't apply to the definition of derivatives). A candidate for this case is the cube root function: +Plot of $f(x) = x^{2/3}$ over $[-1, 1]$. This function does not have a tangent line at $x=0$. +::: +Other functions have tangent lines that become vertical. The natural slope would be $\infty$, but this isn't a limiting answer (except in the extended sense we don't apply to the definition of derivatives). A candidate for this case is the cube root function, shown in @fig-cbrt-over-minus1-1-not-tangent-line-at-0. +::: {#fig-cbrt-over-minus1-1-not-tangent-line-at-0} ```{julia} +#| echo: false plot(cbrt, -1, 1) ``` +Plot of `cbrt` over $[-1,1]$. The "tangent" line at $x=0$ is vertical; the function is not differentiable at $0$ +::: + The derivative at $0$ would need to be $+\infty$ to match the graph. This is implied by the formula for the derivative from the power rule: $f'(x) = 1/3 \cdot x^{-2/3}$, which has a vertical asymptote at $x=0$. :::{.callout-note} ## Note -The `cbrt` function is used above, instead of `f(x) = x^(1/3)`, as the latter is not defined for negative `x`. Though it can be for the exact power `1/3`, it can't be for an exact power like `1/2`. This means the value of the argument is important in determining the type of the output - and not just the type of the argument. Having type-stable functions is part of the magic to making `Julia` run fast, so `x^c` is not defined for negative `x` and most floating point exponents. +The `cbrt` function is used to plot @fig-cbrt-over-minus1-1-not-tangent-line-at-0}, instead of `f(x) = x^(1/3)`, as the latter is not defined for negative `x`. Though it can be for the exact power `1/3`, it can't be for an exact power like `1/2`. This means the value of the argument is important in determining the type of the output - and not just the type of the argument. Having type-stable functions is part of the magic to making `Julia` run fast, so `x^c` is not defined for negative `x` and most floating point exponents. ::: -Lest you think that continuous functions always have derivatives except perhaps at exceptional points, this isn't the case. The functions used to [model](http://tinyurl.com/cpdpheb) the stock market are continuous but have no points where they are differentiable. +Lest you think that continuous functions always have derivatives except perhaps at exceptional points, this isn't the case. The functions used to [model](http://tinyurl.com/cpdpheb) the stock market are continuous but have **no** points where they are differentiable. -## Derivatives and maxima. +## Fermat's theorem We have defined an *absolute maximum* of $f(x)$ over an interval to be a value $f(c)$ for a point $c$ in the interval that is as large as any other value in the interval. Just specifying a function and an interval does not guarantee an absolute maximum, but specifying a *continuous* function and a *closed* interval does, by the extreme value theorem. -::: {.callout-note icon=false} -## A relative maximum +::: {.definition title="A relative maximum"} We say $f(x)$ has a *relative maximum* at $c$ if there exists *some* interval $I=(a,b)$ with $a < c < b$ for which $f(c)$ is an absolute maximum for $f$ and $I$. @@ -104,46 +130,36 @@ The difference is a bit subtle, for an absolute maximum the interval must also b :::{.callout-note} ## Note -A hiker can appreciate the difference. A relative maximum would be the crest of any hill, but an absolute maximum would be the summit. +A hiker can appreciate the difference. A relative maximum would be the crest of any hill, but an absolute maximum would often be the summit. ::: What does this have to do with derivatives? +A theorem attributed to [Fermat](https://digitalcommons.ursinus.edu/cgi/viewcontent.cgi?params=/context/triumphs_calculus/article/1011/&path_info=M05_Fermats_Method_for_Finding_Maxima_and_Minima_2022_05_17.pdf) says something about where a relative or absolute maximum (or minimum) can occur under assumptions: -[Fermat](http://science.larouchepac.com/fermat/fermat-maxmin.pdf), perhaps with insight from Kepler, was interested in maxima of polynomial functions. As a warm up, he considered a line segment $AC$ and a point $E$ with the task of choosing $E$ so that $(E-A) \times (C-E)$ being a maximum. We might recognize this as finding the maximum of $f(x) = (x-A)\cdot(C-x)$ for some $A < C$. Geometrically, we know this to be at the midpoint, as the equation is a parabola, but Fermat was interested in an algebraic solution that led to more generality. +::: {.theorem title="Fermat's theorem"} +If a differentiable function on $(a,b)$ has a maximum at $c$ with $a < c < b$ then $f'(c) = 0$ -He takes $b=AC$ and $a=AE$. Then the product is $a \cdot (b-a) = ab - a^2$. He then perturbs this writing $AE=a+e$, then this new product is $(a+e) \cdot (b - a - e)$. Equating the two, and canceling like terms gives $be = 2ae + e^2$. He cancels the $e$ and basically comments that this must be true for all $e$ even as $e$ goes to $0$, so $b = 2a$ and the value is at the midpoint. - - -In a more modern approach, this would be the same as looking at this expression: - - -$$ -\frac{f(x+e) - f(x)}{e} = 0. -$$ - -Working on the left hand side, for non-zero $e$ we can cancel the common $e$ terms, and then let $e$ become $0$. This becomes a problem in solving $f'(x)=0$. Fermat could compute the derivative for any polynomial by taking a limit, a task we would do now by the power rule and the sum and difference of function rules. - - -This insight holds for other types of functions: - - -> If $f(c)$ is a relative maximum then either $f'(c) = 0$ or the derivative at $c$ does not exist. +::: -When the derivative exists, this says the tangent line is flat. (If it had a slope, then the function would increase by moving left or right, as appropriate, a point we pursue later.) +Relaxing differentibility to continuity, we have + +::: {.relationship title="The derivative at a relative maximum"} + +If a continuous function on $(a,b)$ has a maximum at $c$ with $a < c < b$ then $f'(c) = 0$ or the derivative of $f$ at $c$ does not exist. + +::: For a continuous function $f(x)$, call a point $c$ in the domain of $f$ where either $f'(c)=0$ or the derivative does not exist a **critical** **point**. - We can combine Bolzano's extreme value theorem with Fermat's insight to get the following: -::: {.callout-note icon=false} -## Absolute maxima characterization +::: {.relationship title="Absolute maxima characterization"} A continuous function on $[a,b]$ has an absolute maximum that occurs at a critical point $c$, $a < c < b$, or an endpoint, $a$ or $b$. @@ -151,10 +167,9 @@ A similar statement holds for an absolute minimum. ::: -The above gives a restricted set of places to look for absolute maximum and minimum values - all the critical points and the endpoints. +The above gives a restricted set of places to look for absolute maximum and minimum values---all the critical points and the endpoints, but no where else. - -It is also the case that all relative extrema occur at a critical point, *however* not all critical points correspond to relative extrema. We will see *derivative tests* that help characterize when that occurs. +It is the case that all relative extrema occur at a critical point, *however* it is *not* the case that all critical points correspond to relative extrema. We will see *derivative tests* that help characterize when a critical point corresponds to a relative extrema. ```{julia} @@ -163,7 +178,7 @@ It is also the case that all relative extrema occur at a critical point, *howeve ### {{{lhopital_32}}} imgfile = "figures/lhopital-32.png" caption = L""" -Image number ``32`` from L'Hopitals calculus book (the first) showing that +Image number ``32`` from L'Hospitals calculus book (the first) showing that at a relative minimum, the tangent line is parallel to the $x$-axis. This of course is true when the tangent line is well defined by Fermat's observation. @@ -172,23 +187,15 @@ by Fermat's observation. nothing ``` -![Image number $32$ from L'Hopitals calculus book (the first) showing that +::: {#fig-lhospital-image-number-32} +![](./figures/lhopital-32.png){fig-alt="Image number 32 of L'Hospital's book"} + +Image number $32$ from L'Hospitals calculus book (the first) showing that at a relative minimum, the tangent line is parallel to the $x$-axis. This of course is true when the tangent line is well defined -by Fermat's observation.](./figures/lhopital-32.png) +by Fermat's observation. +::: -### Numeric derivatives - - -The `ForwardDiff` package provides a means to numerically compute derivatives without approximations at a point. In `CalculusWithJulia` this is extended to find derivatives of functions and the `'` notation is overloaded for function objects. Hence these two give nearly identical answers, the difference being only the type of number used: - - -```{julia} -#| hold: true -f(x) = 3x^3 - 2x -fp(x) = 9x^2 - 2 -f'(3), fp(3) -``` ##### Example @@ -196,7 +203,7 @@ f'(3), fp(3) For the function $f(x) = x^2 \cdot e^{-x}$ find the absolute maximum over the interval $[0, 5]$. -We have that $f(x)$ is continuous on the closed interval of the question, and in fact differentiable on $(0,5)$, so any critical point will be a zero of the derivative. We can check for these with: +We have that $f(x)$ is continuous on the closed interval of the question, and in fact differentiable on $(0,5)$. By differentiability, any critical point will be a zero of the derivative. We can check for these using `f'` to compute the derivative automatically: ```{julia} @@ -270,16 +277,16 @@ Here the maximum occurs at an endpoint. The critical point $c=0.67\dots$ does no Let $f(x)$ be differentiable on $(a,b)$ and continuous on $[a,b]$. Then the absolute maximum occurs at an endpoint or where the derivative is $0$ (as the derivative is always defined). This gives rise to: -::: {.callout-note icon=false} -## [Rolle's](http://en.wikipedia.org/wiki/Rolle%27s_theorem) theorem +::: {.theorem title="Rolle's theorem"} -For $f$ differentiable on $(a,b)$ and continuous on $[a,b]$, if $f(a)=f(b)$, then there exists some $c$ in $(a,b)$ with $f'(c) = 0$. +[Rolle's](http://en.wikipedia.org/wiki/Rolle%27s_theorem) theorem states that +if $f$ differentiable on $(a,b)$ and continuous on $[a,b]$ and if $f(a)=f(b)$, then there exists some $c$ in $(a,b)$ with $f'(c) = 0$. ::: ::: {#fig-l-hospital-144} -![Figure from L'Hospital's calculus book](figures/lhopital-144.png) +![](figures/lhopital-144.png){ig-alt="Figure from L'Hospital's calculus book"} Figure from L'Hospital's calculus book showing Rolle's theorem where $c=E$ in the labeling. ::: @@ -287,7 +294,7 @@ Figure from L'Hospital's calculus book showing Rolle's theorem where $c=E$ in th This modest observation opens the door to many relationships between a function and its derivative, as it ties the two together in one statement. -To see why Rolle's theorem is true, we assume that $f(a)=0$, otherwise consider $g(x)=f(x)-f(a)$. By the extreme value theorem, there must be an absolute maximum and minimum. If $f(x)$ is ever positive, then the absolute maximum occurs in $(a,b)$ - not at an endpoint - so at a critical point where the derivative is $0$. Similarly if $f(x)$ is ever negative. Finally, if $f(x)$ is just $0$, then take any $c$ in $(a,b)$. +To see why Rolle's theorem is true, we assume that $f(a)=0$, otherwise consider $g(x)=f(x)-f(a)$. By the extreme value theorem, there must be an absolute maximum and minimum. If $f(x)$ is ever positive, then the absolute maximum occurs in $(a,b)$---not at an endpoint---so at a critical point where the derivative is $0$. Similarly if $f(x)$ is ever negative. Finally, if $f(x)$ is just $0$, then take any $c$ in $(a,b)$. The statement in Rolle's theorem speaks to existence. It doesn't give a recipe to find $c$. It just guarantees that there is *one* or *more* values in the interval $(a,b)$ where the derivative is $0$ if we assume differentiability on $(a,b)$ and continuity on $[a,b]$. @@ -296,7 +303,8 @@ The statement in Rolle's theorem speaks to existence. It doesn't give a recipe t ##### Example -Let $j(x) = e^x \cdot x \cdot (x-1)$. We know $j(0)=0$ and $j(1)=0$, so on $[0,1]$. Rolle's theorem guarantees that we can find *at* *least* one answer (unless numeric issues arise): +Let $j(x) = e^x \cdot x \cdot (x-1)$. We know $j(0)=0$ and $j(1)=0$, so on $[0,1]$. Rolle's theorem guarantees that we can find *at* *least* one answer to $j'(x) = 0$ between $0$ and $1$. We see there is only the one numerically. @fig-plot-expx-times-x-times-x-minus-1-over-0-1 also illustrates graphically the lone value for $c$ in $[a,b]$ for this problem. + ```{julia} @@ -304,21 +312,29 @@ j(x) = exp(x) * x * (x-1) find_zeros(j', 0, 1) ``` -The following graph illustrates the lone value for $c$ in $[a,b]$ for -this problem: - +::: {#fig-plot-expx-times-x-times-x-minus-1-over-0-1} ```{julia} #| echo: false -x0 = find_zero(j', (0, 1)) -j₀ = j(x0) -plot([j, x->j₀ + 0*(x-x0)], 0, 1; legend=false) -scatter!([0,x0,1], [j₀, j₀, j₀]) -annotate!([(0,j₀,text("a", :bottom)), - (x0, j₀, text("c", :bottom)), - (1, j₀, text("b", :bottom))]) +let + gr() + x0 = find_zero(j', (0, 1)) + j₀ = j(x0) + plt = plot(; legend=false, framestyle=:origin) + plot!(plt, [j, x->j₀ + 0*(x-x0)], 0, 1) + scatter!(plt, [(x0, j₀)]) + annotate!(plt, [ + (0,0,text(L"a", :bottom, :left)), + (x0, j₀, text(L"(c, f(c))", :bottom, :left)), + (1, 0, text(L"b", :bottom))]) + plotly() + plt +end ``` +Plot of $f(x) = e^x \cdot x \cdot (x-1)$ over $[0,1]$ showing a single value $c$ satisfying Rolle's theorem +::: + ## The mean value theorem @@ -327,27 +343,25 @@ We are driving south and in one hour cover 70 miles. If the speed limit is 65 mi The mean value theorem is a direct generalization of Rolle's theorem. -::: {.callout-note icon=false} -## Mean value theorem +::: {.theorem title="Mean value theorem"} Let $f(x)$ be differentiable on $(a,b)$ and continuous on $[a,b]$. Then there exists a value $c$ in $(a,b)$ where $$ -f'(c) = (f(b) - f(a)) / (b - a). +f'(c) = \frac{f(b) - f(a)}{b - a}. $$ ::: -This says for any secant line between $a < b$ there will be a parallel tangent line at some $c$ with $a < c < b$ (all provided $f$ is differentiable on $(a,b)$ and continuous on $[a,b]$). +This theorem says appropriate functions the secant line between $a < b$ will have at least one parallel tangent line at a value $c$ with $a < c < b$. @fig-mean-value-theorem illustrates the theorem. The secant line between $a$ and $b$ is dashed. For this function there are two values of $c$ where the slope of the tangent line is seen to be the same as the slope of this secant line. At least one is guaranteed by the theorem. - +::: {#fig-mean-value-theorem} ```{julia} #| hold: true #| echo: false -#| label: fig-mean-value-theorem let # mean value theorem gr() @@ -397,6 +411,9 @@ plotly() nothing ``` +Figure illustrating the mean value theorem. The secant line from $(a,f(a))$ to $(b, f(b))$ is matched by two points $c$ in $(a,b)$ with parallel tangent lines +::: + Like Rolle's theorem this is a guarantee that something exists, not a recipe to find it. In fact, the mean value theorem is just Rolle's theorem applied to: @@ -406,7 +423,7 @@ $$ That is the function $f(x)$, minus the secant line between $(a,f(a))$ and $(b, f(b))$. - +::: {#fig-jsxgraph-mvt} ```{julia} #| hold: true #| echo: false @@ -456,9 +473,28 @@ board.create('tangent', [r], {strokeColor:'#ff0000'}); line = board.create('line',[p[0],p[1]],{strokeColor:'#ff0000',dash:1}); ``` -This interactive example can also be found at [jsxgraph](http://jsxgraph.uni-bayreuth.de/wiki/index.php?title=Mean_Value_Theorem). It shows a cubic polynomial fit to the $4$ adjustable points labeled A through D. The secant line is drawn between points A and B with a dashed line. A tangent line – with the same slope as the secant line – is identified at a point $(\alpha, f(\alpha))$ where $\alpha$ is between the points A and B. That this can always be done is a consequence of the mean value theorem. +Interactive graphic showing a parallel tangent line to $f(x) at $a < c < b$ can always be found that has the same slope as the secant line between $(a, f(a))$ and $(b, f(b))$ +::: +The interactive example of @fig-jsxgraph-mvt can also be found at [jsxgraph](http://jsxgraph.uni-bayreuth.de/wiki/index.php?title=Mean_Value_Theorem). It shows a cubic polynomial fit to the $4$ adjustable points labeled A through D. The secant line is drawn between points A and B with a dashed line. A tangent line---with the same slope as the secant line---is identified at a point $(\alpha, f(\alpha))$ where $\alpha$ is between the points A and B. That this can always be done is a consequence of the mean value theorem. + +##### Example + +The function $f(x) = e^{-x^2/2}$ is continuously differentiable on $[0,1]$. That means the mean value theorem applies. Find a value $c$ satisfying the theorem. + +The pattern is the same, we use `Roots` to solve an equation as follows: + +```{julia} +f(x) = exp(-x^2/2) +a, b = 0, 1 +m = (f(b) - f(a)) / (b - a) # slope of secant line +h(x) = f'(x) - m # solving f'(x) = m +find_zeros(h, (a, b)) +``` + +The call to `find_zeros` returns just one value for $c$. + ##### Example @@ -468,7 +504,9 @@ The mean value theorem is an extremely useful tool to relate properties of a fun For example, suppose we have a function $f(x)$ and we know that the derivative is **always** $0$. What can we say about the function? -Well, constant functions have derivatives that are constantly $0$. But do others? We will see the answer is no: If a function has a zero derivative in $(a,b)$ it must be a constant. We can readily see that if $f$ is a polynomial function this is the case, as we can differentiate a polynomial function and this will be zero only if **all** its coefficients are $0$, which would mean there is no non-constant leading term in the polynomial. But polynomials are not representative of all functions, and so a proof requires a bit more effort. +Well, constant functions have derivatives that are constantly $0$. But do others? We will see the answer is no: If a function has a zero derivative in $(a,b)$ it must be a constant. + +We can readily see that if $f$ is a polynomial function this is the case, as we can differentiate a polynomial function and this will be zero only if **all** its coefficients are $0$, which would mean there is no non-constant leading term in the polynomial. But polynomials are not representative of all functions, and so a proof requires a bit more effort. Suppose it is known that $f'(x)=0$ on some interval $I$ and we take any $a < b$ in $I$. Since $f'(x)$ always exists, $f(x)$ is always differentiable, and hence always continuous. So on $[a,b]$ the conditions of the mean value theorem apply. That is, there is a $c$ in $(a,b)$ with $(f(b) - f(a)) / (b-a) = f'(c) = 0$. But this would imply $f(b) - f(a)=0$. That is $f(x)$ is a constant, as for any $a$ and $b$, we see $f(a)=f(b)$. @@ -477,10 +515,9 @@ Suppose it is known that $f'(x)=0$ on some interval $I$ and we take any $a < b$ ### The Cauchy mean value theorem -[Cauchy](http://en.wikipedia.org/wiki/Mean_value_theorem#Cauchy.27s_mean_value_theorem) offered an extension to the mean value theorem above. +[Cauchy](http://en.wikipedia.org/wiki/Mean_value_theorem#Cauchy.27s_mean_value_theorem) offered an extension to the mean value theorem. -::: {.callout-note icon=false} -## Cauchy mean value theorem +::: {.theorem title="Cauchy mean value theorem"} Suppose both $f$ and $g$ satisfy the conditions of the mean value theorem on $[a,b]$ with $g(b)-g(a) \neq 0$, then there exists at least one $c$ with $a < c < b$ such that @@ -513,12 +550,12 @@ For some $c$ in $[0,x]$. If $\lim_{x \rightarrow 0} f'(x)/g'(x) = L$, then the r This could be used to prove the limit of $\sin(x)/x$ as $x$ goes to $0$ just by showing the limit of $\cos(x)/1$ is $1$, as is known by continuity. -### Visualizing the Cauchy mean value theorem +##### Example: visualizing the Cauchy mean value theorem The Cauchy mean value theorem can be visualized in terms of a tangent line and a *parallel* secant line in a similar manner as the mean value theorem as long as a *parametric* graph is used. A parametric graph plots the points $(g(t), f(t))$ for some range of $t$. That is, it graphs *both* functions at the same time. The following illustrates the construction of such a graph: - +::: {#fig-illustrate-cauchy-mean-value-theore} ```{julia} #| hold: true #| echo: false @@ -540,15 +577,7 @@ function parametric_fns_graph(n) val = @sprintf("% 0.2f", ts[end]) annotate!(plt, [(0, 1, L"t = %$val")]) end -caption = L""" - -Illustration of parametric graph of $(g(t), f(t))$ for $-\pi/2 \leq t -\leq \pi/2$ with $g(x) = \sin(x)$ and $f(x) = x$. Each point on the -graph is from some value $t$ in the interval. We can see that the -graph goes through $(0,0)$ as that is when $t=0$. As well, it must go -through $(1, \pi/2)$ as that is when $t=\pi/2$ - -""" +caption = "" n = 10 @@ -562,9 +591,17 @@ plotly() ImageFile(imgfile, caption) ``` -With $g(x) = \sin(x)$ and $f(x) = x$, we can take $I=[a,b] = [0, \pi/2]$. In the figure below, the *secant line* is drawn in red which connects $(g(a), f(a))$ with the point $(g(b), f(b))$, and hence has slope $\Delta f/\Delta g$. The parallel lines drawn show the *tangent* lines with slope $f'(c)/g'(c)$. Two exist for this problem, the mean value theorem guarantees at least one will. +Illustration of parametric graph of $(g(t), f(t))$ for $-\pi/2 \leq t +\leq \pi/2$ with $g(x) = \sin(x)$ and $f(x) = x$. Each point on the +graph is from some value $t$ in the interval. We can see that the +graph goes through $(0,0)$ as that is when $t=0$. As well, it must go +through $(1, \pi/2)$ as that is when $t=\pi/2$ +::: +With $g(x) = \sin(x)$ and $f(x) = x$, we can take $I=[a,b] = [0, \pi/2]$. In the @fig-mvt-two-c-exists-for-sinx, the *secant line* is drawn in red which connects $(g(a), f(a))$ with the point $(g(b), f(b))$, and hence has slope $\Delta f/\Delta g$. The parallel lines drawn show the *tangent* lines with slope $f'(c)/g'(c)$. Two exist for this problem, the mean value theorem guarantees at least one will. + +::: {#fig-mvt-two-c-exists-for-sinx} ```{julia} #| hold: true #| echo: false @@ -587,6 +624,9 @@ end p ``` +Illustration of the Cauchy mean value theorem +::: + ## Questions @@ -659,6 +699,44 @@ numericq(float(val)) ###### Question +Let $f(x) = 1/x$. For $0 < a < b$, find $c$ so that $f'(c) = (f(b) - f(a)) / (b-a)$. + + +```{julia} +#| hold: true +#| echo: false +choices = [ +"``c = (a+b)/2``", +"``c = \\sqrt{ab}``", +"``c = 1 / (1/a + 1/b)``", +"``c = a + (\\sqrt{5} - 1)/2 \\cdot (b-a)``" +] +answ = 2 +radioq(choices, answ) +``` + +###### Question + + +Let $f(x) = x^2$. For $0 < a < b$, find $c$ so that $f'(c) = (f(b) - f(a)) / (b-a)$. + + +```{julia} +#| hold: true +#| echo: false +choices = [ +"``c = (a+b)/2``", +"``c = \\sqrt{ab}``", +"``c = 1 / (1/a + 1/b)``", +"``c = a + (\\sqrt{5} - 1)/2 \\cdot (b-a)``" +] +answ = 1 +radioq(choices, answ) +``` + +###### Question + + Will the function $f(x) = x + 1/x$ satisfy the conditions of the mean value theorem over $[-1/2, 1/2]$? @@ -705,43 +783,6 @@ answ = 3 radioq(choices, answ) ``` -###### Question - - -Let $f(x) = 1/x$. For $0 < a < b$, find $c$ so that $f'(c) = (f(b) - f(a)) / (b-a)$. - - -```{julia} -#| hold: true -#| echo: false -choices = [ -"``c = (a+b)/2``", -"``c = \\sqrt{ab}``", -"``c = 1 / (1/a + 1/b)``", -"``c = a + (\\sqrt{5} - 1)/2 \\cdot (b-a)``" -] -answ = 2 -radioq(choices, answ) -``` - -###### Question - - -Let $f(x) = x^2$. For $0 < a < b$, find $c$ so that $f'(c) = (f(b) - f(a)) / (b-a)$. - - -```{julia} -#| hold: true -#| echo: false -choices = [ -"``c = (a+b)/2``", -"``c = \\sqrt{ab}``", -"``c = 1 / (1/a + 1/b)``", -"``c = a + (\\sqrt{5} - 1)/2 \\cdot (b-a)``" -] -answ = 1 -radioq(choices, answ) -``` ###### Question @@ -774,3 +815,85 @@ L"The squeeze theorem applies, as $0 < g(x) < x$", answ = 3 radioq(choices, answ) ``` + + +##### Question + +[Fermat](https://digitalcommons.ursinus.edu/cgi/viewcontent.cgi?params=/context/triumphs_calculus/article/1011/&path_info=M05_Fermats_Method_for_Finding_Maxima_and_Minima_2022_05_17.pdf) didn't exactly prove his theorem in the language of today. Rather, following Monks, we quote + +> Let a be the desired unknown, whether it be a length, a plane region or a solid, depending +> on what the given magnitude equals, and let its maximum or minimum be found in terms of +> $a$, involving whatever degree. Replace this first quantity with $a + e$, and the maximum or +> minimum will be found in terms of $a$ and $e$, with coefficients of whatever degree. These two +> representations of the maximum or minimum are adequated, to use Diophantus’ term, +> and the common terms are subtracted. Having done this, all terms from either part (affected by +> $e$ or its powers) are divided each by $e$, or by a higher power of the same, until some term of +> one or the other of the expressions is altogether freed from being affected by $e$. +> +> All terms involving $e$ or one of its powers are then eliminated and the remaining terms +> are equated; or, should one of the expressions be left as nothing, then the positive terms +> are equated with the negatives, which reduces to the same thing. The solution to this last +> equation will yield the value of $a$, which will reveal knowledge of the maximum or minimum + +Huh? As an example, he considered a line segment $AC$ and a point $E$ with the task of choosing $E$ so that $(E-A) \times (C-E)$ being a maximum. + +::: {#fig-fermat-line-segment} +```{julia} +#| echo: false + +let + gr() + A, E, C = (0,0), (1, 0), (3, 0) + plt = plot(; empty_style...) + plot!(plt, [A,C]; line=(1, :black)) + tck = (0, 0.1) + for P ∈ (A, E, C) + plot!(plt, [P, P .+ tck], line=(1, :black)) + end + annotate!(plt, [ + (A..., text(L"A", :top)), + (E..., text(L"E", :top)), + (C..., text(L"C", :top))]) + plotly() + plt +end +``` + +$AC$ is a line divided at $E$ so that $AE \times EC$ is maximum +:::: + + +Set $b=AC$ and $a = AE$ then the product is $a \cdot (b-a)$. the point was at $a + e$, then the product would be $(a+e) \cdot (b - a - e)$. The term *adequated* means approximately equal gives what? + +```{julia} +#| echo: false +choices = [L"a \cdot(b-a) - a \cdot (b-a) = 0", + L"a \cdot(b-a) - (a + e) \cdot (b- (a - e)) \approx 0"] +answer = 2 +buttonq(choices, answer) +``` + +Next we divide by $e$---or a higher power of $e$---and simplify so that some term has not $e$ in it. + +For this case, does this satisfy the above? + +$$ +\frac{(a \cdot (b-a) - (a + e)\cdot(b - (a + e))}{e} = 2a - b + e +$$ + +```{julia} +#| echo: false +choices = ["Yes", "No"] +answer = 1 +buttonq(choices, answer) +``` + +The value $b - 2a = 0$ gives $a = $b/2$. Is this true: geometrically, we know this to be at the midpoint, as the equation is a parabola. + + +```{julia} +#| echo: false +choices = ["Yes", "No"] +answer = 1 +buttonq(choices, answer) +``` diff --git a/quarto/derivatives/more_zeros.qmd b/quarto/derivatives/more_zeros.qmd index 0644ed3..9aeb79c 100644 --- a/quarto/derivatives/more_zeros.qmd +++ b/quarto/derivatives/more_zeros.qmd @@ -1,5 +1,4 @@ -# Derivative-free alternatives to Newton's method - +# Other zero-finding algorithms {{< include ../_common_code.qmd >}} @@ -17,379 +16,537 @@ using SymPy --- -Newton's method is not the only algorithm of its kind for identifying zeros of a function. In this section we discuss some alternatives. +There are *numerous* zero-finding methods in addition to the secant method and Newton's method. This section shows a few different directions. It then discusses the topic of when to terminate an algorithm. + +This section is entirely optional, none of the algorithms discussed below are utilized in the sequel. + +## Other methods + +We discuss variations of both Newton's method and the bisection method. + +### Estimating the derivative + +[Sidi](https://www.math.nthu.edu.tw/~amen/2008/070227-1.pdf) starts with Newton's method with its update step + +$$ +x_{i+1} = x_i - \frac{f(x_i)}{f'(x_i)} +$$ + +and notes that the secant method just uses the slope of the secant line between $x_i$ and $x_{i-1}$ to estimate $f'(x_i)$. The secant line is the *linear* polynomial interpolating the two points $(x_i, f(x_i))$ and $(x_{i-1}, f(x_{i-1}))$ and the slope the derivative of this polynomial. Sidi generalizes this to approximate the function $f'(x_i)$ by using more points from the algorithm to interpolate a polynomial at $x_i, x_{i-1}, \dots, x_{i-k}$ and then using the derivative of this polynomial to estimate the derivative of $f$ at $x_i$. -## The `find_zero(f, x0)` function +Let's consider a case with three points or $k=2$. Here we use formula (10) of Sidi computing the derivative of the interpolating polynomial at $x_n$ specialized for $k=2$. +```{julia} +function pprime_n2(xn_2, xn_1, xn, yn_2, yn_1, yn) + m = (yn - yn_1) / (xn - xn_1) + m_1 = (yn_1 - yn_2) / (xn_1 - xn_2) + m + (m - m_1)/(xn - xn_2) * (xn - xn_1) +end +``` -The function `find_zero` from the `Roots` packages provides several different algorithms for finding a zero of a function, including some derivative-free algorithms for finding zeros when started with a nearby initial guess. The default method is similar to Newton's method in that only a good, initial guess is needed. However, the algorithm, while possibly slower in terms of function evaluations and steps, is engineered to be a bit more robust to the choice of initial estimate than Newton's method. (If it finds a bracket, it will use a bisection algorithm which is guaranteed to converge, but can be slower to do so.) Here we see how to call the function: +Typically, we would start with two initial points and apply the secant method to produce a third, but for this example we will use an initial three points. + +```{julia} +f(x) = cos(x) - x/2 +xs = [0, pi/2, pi/4] +ys = f.(xs) # y₀, y₁, y₂ + +xx = xs[end] - ys[end] / pprime_n2(xs[end-2:end]..., ys[end-2:end]...) +yy = f(xx) +push!(xs, xx); push!(ys, yy) # y₃ +``` + +This method generalizes the secant method with a convergence rate of $1.83928\cdots$. For this problem we see it takes 5 iterations to converge to machine tolerance: + +```{julia} +xx = xs[end] - ys[end] / pprime_n2(xs[end-2:end]..., ys[end-2:end]...) +yy = f(xx) +push!(xs, xx); push!(ys, yy) # y₄ = 0.0004182051168989398 + +xx = xs[end] - ys[end] / pprime_n2(xs[end-2:end]..., ys[end-2:end]...) +yy = f(xx) +push!(xs, xx); push!(ys, yy) # y₅ = -3.6161489780361933e-7 + +xx = xs[end] - ys[end] / pprime_n2(xs[end-2:end]..., ys[end-2:end]...) +yy = f(xx) +push!(xs, xx); push!(ys, yy) # y₆ = 3.609335053056384e-13 + +xx = xs[end] - ys[end] / pprime_n2(xs[end-2:end]..., ys[end-2:end]...) +yy = f(xx) +push!(xs, xx); push!(ys, yy) # y₇ = 0.0 + +xx +``` + +### Estimating the inverse function + +Suppose $f^{-1}$ exists in a neighborhood of $\alpha$ and we have generated steps in our algorithm $x_0, x_1, \dots, x_n$. We can find $\alpha$ from $f^{-1}(0)$. Typically though, we wouldn't have the inverse function, but we can use facts about functions such as linearization, which *near* $0$ has: + +$$ +f^{-1}(y) \approx f^{-1}(0) + (f^{-1})'(0)\cdot(y) = \alpha + (f^{-1})'(0)\cdot y +$$ + +Solving for $\alpha$ gives for $(x_i, f(x_i))$ + +$$ +\alpha \approx f^{-1}(f(x_i)) - f^{-1}(0) f(x_i). +$$ + +Replacing $f^{-1}(0) \approx (f^{-1})'(y_i) = (f^{-1})'(f(x_i)) = 1/f'(x_i)$ we get: + +$$ +\alpha \approx x_i - \frac{f(x_i)}{f'(x_i)}. +$$ + +Which is basically Newton's method. + +The above uses one point, $(x_i, f(x_i))$ and the fact that $f$ is differentiable. What if two (or more) points were used, would that give some insight? + +Here is some code that interpolates a *polynomial* for $f^{-1}(y)$ through $k$ points and then solves for it's value at $0$, which is $a_0$. ```{julia} -f(x) = cos(x) - x -x₀ = 1 -find_zero(f, x₀) +@syms x[1:5] y[1:5] a[0:5] +a₀ = first(a) +function I(k) + eqs = Tuple(sum(a[j] * y[i]^(j-1) for j in 1:k) ~ x[i] for i in 1:k) + sols = solve(eqs, Tuple(a[1:k])) + sols[a₀] # intercept +end ``` -Compare to this related call which uses the bisection method: +We use $k=2$ and see what comes: ```{julia} -find_zero(f, (0, 1)) ## [0,1] must be a bracketing interval +I(2) ``` -For this example both give the same answer, but the bisection method is a bit less convenient as a bracketing interval must be pre-specified. - - -## The secant method - - -The default `find_zero` method above uses a secant-like method unless a bracketing method is found. The secant method is historic, dating back over $3000$ years. Here we discuss the secant method in a more general framework. - - -One way to view Newton's method is through the inverse of $f$ (assuming it exists): if $f(\alpha) = 0$ then $\alpha = f^{-1}(0)$. - - -If $f$ has a simple zero at $\alpha$ and is locally invertible (that is some $f^{-1}$ exists) then the update step for Newton's method can be identified with: - - - * fitting a polynomial to the local inverse function of $f$ going through the point $(f(x_0),x_0)$, - * and matching the slope of $f$ at the same point. - - -That is, we can write $g(y) = h_0 + h_1 (y-f(x_0))$. Then $g(f(x_0)) = x_0 = h_0$, so $h_0 = x_0$. From $g'(f(x_0)) = 1/f'(x_0)$, we get $h_1 = 1/f'(x_0)$. That is, $g(y) = x_0 + (y-f(x_0))/f'(x_0)$. At $y=0,$ we get the update step $x_1 = g(0) = x_0 - f(x_0)/f'(x_0)$. - - -A similar viewpoint can be used to create derivative-free methods. - - -For example, the [secant method](https://en.wikipedia.org/wiki/Secant_method) can be seen as the result of fitting a degree-$1$ polynomial approximation for $f^{-1}$ through two points $(f(x_0),x_0)$ and $(f(x_1), x_1)$. - - -Again, expressing this approximation as $g(y) = h_0 + h_1(y-f(x_1))$ leads to $g(f(x_1)) = x_1 = h_0$. Substituting $f(x_0)$ gives $g(f(x_0)) = x_0 = x_1 + h_1(f(x_0)-f(x_1))$. Solving for $h_1$ leads to $h_1=(x_1-x_0)/(f(x_1)-f(x_0))$. Then $x_2 = g(0) = x_1 - (x_1-x_0)/(f(x_1)-f(x_0)) \cdot f(x_1)$. This is the first step of the secant method: - - -$$ -x_{n+1} = x_n - f(x_n) \frac{x_n - x_{n-1}}{f(x_n) - f(x_{n-1})}. -$$ - -That is, where the next step of Newton's method comes from the intersection of the tangent line at $x_n$ with the $x$-axis, the next step of the secant method comes from the intersection of the secant line defined by $x_n$ and $x_{n-1}$ with the $x$ axis. That is, the secant method simply replaces $f'(x_n)$ with the slope of the secant line between $x_n$ and $x_{n-1}$. - - -We code the update step as `λ2`: - +We can see this is a rewriting of the secant method through: ```{julia} -λ2(f0,f1,x0,x1) = x1 - f1 * (x1-x0) / (f1-f0) +x1, x2 = x[1:2]; y1, y2 = y[1:2] +m = (y2 - y1) / (x2 - x1) +secant_method = x2 - (1/m) * y2 +simplify(I(2) - secant_method) ``` +An inverse quadratic step ($k=2$) is utilized by Brent's method, as possible, to yield a rapidly convergent bracketing algorithm implemented as a default zero finder in many software languages. `Julia`'s `Roots` package implements the method in `Roots.Brent()`. -Then we can run a few steps to identify the zero of sine starting at $3$ and $4$ - +To see an example of inverse quadratic, we first make a function to compute the next $x$ value, given three previous $x$ and $f(x)$ values. ```{julia} -#| hold: true -#| term: true -x0,x1 = 4,3 -f0,f1 = sin.((x0,x1)) -@show x1,f1 - -x0,x1 = x1, λ2(f0,f1,x0,x1) -f0,f1 = f1, sin(x1) -@show x1,f1 - -x0,x1 = x1, λ2(f0,f1,x0,x1) -f0,f1 = f1, sin(x1) -@show x1,f1 - -x0,x1 = x1, λ2(f0,f1,x0,x1) -f0,f1 = f1, sin(x1) -@show x1,f1 - -x0,x1 = x1, λ2(f0,f1,x0,x1) -f0,f1 = f1, sin(x1) -x1,f1 +u = lambdify(I(3), (x[1:3]..., y[1:3]...)) ``` -Like Newton's method, the secant method converges quickly for this problem (though its rate is less than the quadratic rate of Newton's method). - - -This method is included in `Roots` as `Secant()` (or `Order1()`): - +Let's try initial values $(x_0, x_1, x_2) = (0, \pi/2, \pi/4)$: ```{julia} -find_zero(sin, (4,3), Secant()) +f(x) = cos(x) - x/2 +xs = [0, pi/2, pi/4] +ys = f.(xs) ``` -Though the derivative is related to the slope of the secant line, that is in the limit. The convergence of the secant method is not as fast as Newton's method, though at each step of the secant method, only one new function evaluation is needed, so it can be more efficient for functions that are expensive to compute or differentiate. +Now we do a step. The new values is "pushed" to the vector of values. + +```{julia} +xx = u(xs[end-2:end]..., ys[end-2:end]...) +yy = f(xx) +push!(xs, xx) +push!(ys, yy) +xx, yy +``` + +We know do a few more steps, the value of `yy` is shown as a comment. + +```{julia} +xx = u(xs[end-2:end]..., ys[end-2:end]...); yy = f(xx) +push!(xs, xx); push!(ys, yy) # yy = 0.0011053827937966831 + +xx = u(xs[end-2:end]..., ys[end-2:end]...); yy = f(xx) +push!(xs, xx); push!(ys, yy) # yy = -2.113895167021873e-6 + +xx = u(xs[end-2:end]..., ys[end-2:end]...); yy = f(xx) +push!(xs, xx); push!(ys, yy) # yy = 1.1904810470753091e-11 + +xx = u(xs[end-2:end]..., ys[end-2:end]...); yy = f(xx) +push!(xs, xx); push!(ys, yy) # yy = 0.0 + +xx +``` + +The convergence happens quite quickly with this well-behaved problem. + +An inverse cubic interpolation is utilized by [Alefeld, Potra, and Shi](https://dl.acm.org/doi/10.1145/210089.210111) which gives an asymptotically even more rapidly convergent algorithm than Brent's (implemented in `Roots.AlefeldPotraShi()` and also `Roots.A42()`). This is used as a finishing step in many cases by the default hybrid `Order0()` method of `find_zero`. -Let $\epsilon_{n+1} = x_{n+1}-\alpha$, where $\alpha$ is assumed to be the *simple* zero of $f(x)$ that the secant method converges to. A [calculation](https://math.okstate.edu/people/binegar/4513-F98/4513-l08.pdf) shows that - - -$$ -\begin{align*} -\epsilon_{n+1} &\approx \frac{x_n-x_{n-1}}{f(x_n)-f(x_{n-1})} \frac{(1/2)f''(\alpha)(\epsilon_n-\epsilon_{n-1})}{x_n-x_{n-1}} \epsilon_n \epsilon_{n-1}\\ -& \approx \frac{f''(\alpha)}{2f'(\alpha)} \epsilon_n \epsilon_{n-1}\\ -&= C \epsilon_n \epsilon_{n-1}. -\end{align*} -$$ - - -The constant `C` is similar to that for Newton's method, and reveals potential troubles for the secant method similar to those of Newton's method: a poor initial guess (the initial error is too big), the second derivative is too large, the first derivative too flat near the answer. - - -Assuming the error term has the form $\epsilon_{n+1} = A|\epsilon_n|^\phi$ and substituting into the above leads to the equation - - -$$ -\frac{A^{1+1/\phi}}{C} = |\epsilon_n|^{1 - \phi +1/\phi}. -$$ - -The left side being a constant suggests $\phi$ solves: $1 - \phi + 1/\phi = 0$ or $\phi^2 -\phi - 1 = 0$. The solution is the golden ratio, $(1 + \sqrt{5})/2 \approx 1.618\dots$. ### Steffensen's method +Another alternative to the secant method is Steffensen's method. -Steffensen's method is a secant-like method that converges with $|\epsilon_{n+1}| \approx C |\epsilon_n|^2$. The secant is taken between the points $(x_n,f(x_n))$ and $(x_n + f(x_n), f(x_n + f(x_n))$. Like Newton's method this requires $2$ function evaluations per step. Steffensen's is implemented through `Roots.Steffensen()`. Steffensen's method is more sensitive to the initial guess than other methods, so in practice must be used with care, though it is a starting point for many higher-order derivative-free methods. +The secant method has super-linear convergence, but not quadratic convergence. It uses these points to evaluate the values $(x_i, f(x_i)$ and $(x_{i-1}, f(x_{i-1}))$. When $x_i$ converges to $\alpha$, $x_i - x_{i-1}$ will converge to $0$. The secant lines used are eventually "converging" to tangent lines. + +Steffensen's method takes a different pair of points to use for a secant line, these being $(x_n,f(x_n))$ and $(x_n + f(x_n), f(x_n + f(x_n)))$. When $x_i \rightarrow \alpha$ it follows for a continuous $f(x)$ that $f(x_i) \rightarrow 0$, so the secant lines used by Steffensen's method will also be close to the tangent line. + +[This note](https://fractal.math.unr.edu/~ejolson/701-12/code/hw2sol/hw2sol.pdf) shows that with $\eta_i$ and $\xi_i$ being values that *converge* to $\alpha$, that + +$$ +e_{i+1} = -e_{i}^2 \cdot +\left(\frac{f''(\xi_i)\left(f'(x_i) - \frac{1}{2} f''(\eta_i) e_i\right) +f''(\eta_i)}{ + 2f'(x_i) + f''(\xi_i)f(x_i)}\right) +$$ + +As the following ratio converges to something non zero under assumptions, the Steffensen method has quadratic convergence. + +$$ +e_{i+1}/e_i^2 \rightarrow \frac{f''(\alpha)(1 + f'(\alpha))}{2f'(\alpha)} +$$ -## Inverse quadratic interpolation +Like Newton's method this method requires $2$ function evaluations per step, but unlike Newton's method is derivative free. Steffensen's is implemented in the `Roots` package through `Roots.Steffensen()`. Steffensen's method is more sensitive to the initial guess than other methods, so in practice must be used with care, though it is a starting point for many higher-order derivative-free methods. -Inverse quadratic interpolation fits a quadratic polynomial through three points, not just two like the Secant method. The third being $(f(x_2), x_2)$. +### Alternative bracketing methods -For example, here is the inverse quadratic function, $g(y)$, going through three points marked with red dots. The blue dot is found from $(g(0), 0)$. +The bisection method has several advantages, primarily it is guaranteed to converge regardless of any assumptions on the shape of the function. It's implementation in `Roots` can handle any $x$ values as long as the function value has a sign (not `NaN` and not an error). However, it is slow---linearly convergent. There can be improvements. +#### Regula falsi + +One alternative is the (modified) *regula falsi* method which replaces the midpoint ($x_i/2 + x_{i-1}/2$) with the intersection point of the line between two bracketing points $(x_i, f(x_i))$ and $(x_{i-1}, f(x_{i-1}))$ given by solving the following, which comes from similar triangles: ```{julia} -#| hold: true +@syms xᵢ xᵢ₋₁ yᵢ yᵢ₋₁ x +only(solve(yᵢ / (x - xᵢ) ~ -yᵢ₋₁ / (xᵢ₋₁ - x), x)) +``` + + +As seen earlier, this formula is a single step of the secant method, but unlike the secant method, for this method the two points chosen to continue are picked to ensure $x_i, x_{i-1}$ form a bracketing interval. + +Despite being related to the secant method, the convergence rate of *regula falsi* is only linear. Some function shapes preference a certain endpoint, whereas the secant method chooses the last two values. + +@fig-regula-false-convex show that some function shapes result in one end point being fixed as the algorithm progresses which can lead to linear convergence. + +::: {#fig-regula-false-convex} +```{julia} #| echo: false - -a,b,c = 1,2,3 -fa,fb,fc = -1,1/4,1 -g(y) = (y-fb)*(y-fa)/(fc-fb)/(fc-fa)*c + (y-fc)*(y-fa)/(fb-fc)/(fb-fa)*b + (y-fc)*(y-fb)/(fa-fc)/(fa-fb)*a -ys = range(-2,2, length=100) -xs = g.(ys) -plot(xs, ys, legend=false) -scatter!([a,b,c],[fa,fb,fc], color=:red, markersize=5) -scatter!([g(0)],[0], color=:blue, markersize=5) -plot!(zero, color=:blue) -``` - -Here we use `SymPy` to identify the degree-$2$ polynomial as a function of $y$, then evaluate it at $y=0$ to find the next step: - - -```{julia} -@syms y hs[0:2] xs[0:2] fs[0:2] -H(y) = sum(hᵢ*(y - fs[end])^i for (hᵢ,i) ∈ zip(hs, 0:2)) - -eqs = tuple((H(fᵢ) ~ xᵢ for (xᵢ, fᵢ) ∈ zip(xs, fs))...) -ϕ = solve(eqs, hs) -hy = subs(H(y), ϕ) -``` - -The value of `hy` at $y=0$ yields the next guess based on the past three, and is given by: - - -```{julia} -q⁻¹ = hy(y => 0) -``` - -Though the above can be simplified quite a bit when computed by hand, here we simply make this a function with `lambdify` which we will use below. - - -```{julia} -λ3 = lambdify(q⁻¹) # fs, then xs -``` - -(`SymPy`'s `lambdify` function, by default, picks the order of its argument lexicographically, in this case they will be the `f` values then the `x` values.) - - -An inverse quadratic step is utilized by Brent's method, as possible, to yield a rapidly convergent bracketing algorithm implemented as a default zero finder in many software languages. `Julia`'s `Roots` package implements the method in `Roots.Brent()`. An inverse cubic interpolation is utilized by [Alefeld, Potra, and Shi](https://dl.acm.org/doi/10.1145/210089.210111) which gives an asymptotically even more rapidly convergent algorithm than Brent's (implemented in `Roots.AlefeldPotraShi()` and also `Roots.A42()`). This is used as a finishing step in many cases by the default hybrid `Order0()` method of `find_zero`. - - -In a bracketing algorithm, the next step should reduce the size of the bracket, so the next iterate should be inside the current bracket. However, quadratic convergence does not guarantee this to happen. As such, sometimes a substitute method must be chosen. - - -[Chandrapatla's](https://www.google.com/books/edition/Computational_Physics/cC-8BAAAQBAJ?hl=en&gbpv=1&pg=PA95&printsec=frontcover) method, is a bracketing method utilizing an inverse quadratic step as the centerpiece. The key insight is the test to choose between this inverse quadratic step and a bisection step. This is done in the following based on values of $\xi$ and $\Phi$ defined within: - - -```{julia} -function chandrapatla(f, u, v, λ; verbose=false) - a,b = promote(float(u), float(v)) - fa,fb = f(a),f(b) - @assert fa * fb < 0 - - if abs(fa) < abs(fb) - a,b,fa,fb = b,a,fb,fa +let + gr() + Δ = 0.2 + function add_line!(xs, ys, f, a) + x = (xs[1]*ys[2] - xs[2]*ys[1])/(ys[2] - ys[1]) + scatter!(collect(zip(xs, ys)); marker=(5, :orange)) + plot!(collect(zip(xs, ys)); line=(1, :gray50)) + scatter!([(x, 0)]; marker=(5, :blue)) + annotate!([(x, 0, text(a, :top))]) + plot!(plt, [(x, 0), (x, Δ)]; line=(1, :gray50)) + xs[2] = x + ys[2] = f(x) end - c, fc = a, fa + f(x) = 10*log(x)/x^3 + plt = plot(; empty_style..., xlims=(0.7, 3.2)) + plt = plot!(plt, f, 0.8, 3) + plot!([(0.8, 0), (3.2, 0)]; line=(1, :black), arrow=true, side=:right) - maxsteps = 100 - for ns in 1:maxsteps + x₀ = 2.75 + x₁ = 0.85 + α = 1.0 + xs = [x₁, x₀] + ys = f.(xs) - Δ = abs(b-a) - m, fm = (abs(fa) < abs(fb)) ? (a, fa) : (b, fb) - ϵ = eps(m) - if Δ ≤ 2ϵ - return m - end - @show m,fm - iszero(fm) && return m + add_line!(xs, ys, f, L"x_2") + add_line!(xs, ys, f, L"x_3") + add_line!(xs, ys, f, L"x_4") + add_line!(xs, ys, f, L"x_5") + add_line!(xs, ys, f, L"x_6") + add_line!(xs, ys, f, L"x_7") - ξ = (a-b)/(c-b) - Φ = (fa-fb)/(fc-fb) + plot!(plt, [(x₀, 0), (x₀, Δ)]; line=(1, :gray50)) + plot!(plt, [(x₁, 0), (x₁, Δ)]; line=(1, :gray50)) + annotate!(plt, [(x₀, 0, text(L"x_0", :top)), + (x₁, 0, text(L"x_1", :top)), + (α, 0, text(L"\alpha", :left, :top))]) + scatter!(plt, [(α, 0)]; marker=(5, :green)) - if Φ^2 < ξ < 1 - (1-Φ)^2 - xt = λ(fa,fc,fb, a,c,b) # inverse quadratic - else - xt = a + (b-a)/2 - end - - ft = f(xt) - - isnan(ft) && break - - if sign(fa) == sign(ft) - c,fc = a,fa - a,fa = xt,ft - else - c,b,a = b,a,xt - fc,fb,fa = fb,fa,ft - end - - verbose && @show ns, a, fa - - end - error("no convergence: [a,b] = $(sort([a,b]))") + plotly() + plt end ``` -Like bisection, this method ensures that $a$ and $b$ is a bracket, but it moves $a$ to the newest estimate, so does not maintain that $a < b$ throughout. +Plot illustrating that the *regula falsi* method may have a fixed endpoint for some convex functions +::: -We can see it in action on the sine function. Here we pass in $\lambda$, but in a real implementation (as in `Roots.Chandrapatla()`) we would have programmed the algorithm to compute the inverse quadratic value. +#### Modified regula falsi +@fig-modified-regula-falsi shows a scenario where the secant line between $(x_{i-1}, f(x_{i-1}))$ and $(x_i, f(x_i))$ crosses the $x$ axis at $x_{i+1}$ which is to the *right* of the zero $\alpha$, as it always will be for this curve and these points. A modified *regula falsi* method modifies the fixed end by using $\tilde{f}(x_i)$ and not $f(x_i)$ to compute the secant line, where $\tilde{f}$ is some multiple, $\gamma$, of $f$. In the figure, $\gamma$ is shown so that the *next* choice ($x_{i+2}$ would be its label) is exactly $\alpha$. And value for the multiplier less than this $\gamma$ will shift the intersection point to the other side of $\alpha$. The value of $\gamma$ is the ratio of the secant line slopes between $x_{i+1}$ and $\alpha$ and between $x_{i-1}$ and $\alpha$. Some choices for $\gamma$ lead to super-linear convergence. + +::: {#fig-modified-regula-falsi} +```{julia} +#| echo: false +let + gr() + dd(f, a, b) = (f(a) - f(b)) / (a - b) + f(x) = (x-3)^2 - 1 + plt = plot(; xlims=(0.5, 3.25), empty_style...) + plot!(plt, [(1,0), (3.25,0)]; arrow=true, side=:right, line=(1, :gray)) + plot!(f, 1, 3.25; line=(1, :black)) + xᵢ₋₁ , xᵢ = 1.25, 2.75 + xᵢ₊₁ = (xᵢ₋₁ * f(xᵢ) - xᵢ * f( xᵢ₋₁)) / (f(xᵢ) - f(xᵢ₋₁)) + + α = 2 + γ = dd(f, xᵢ₊₁, α) / dd(f, xᵢ₋₁, α) + + plot!(plt, [(xᵢ₋₁, 0), (xᵢ₋₁, f(xᵢ₋₁))]; line=(1, :dot)) + plot!(plt, [(xᵢ₋₁, f(xᵢ₋₁)) , (xᵢ, f(xᵢ))]; line=(1, :dash, :blue)) + plot!(plt, [(xᵢ₋₁, γ*f(xᵢ₋₁)) , (xᵢ₊₁, f(xᵢ₊₁))]; line=(1, :dash, :blue)) + + scatter!(plt, [(xᵢ₋₁, γ*f(xᵢ₋₁)), (xᵢ₋₁, f(xᵢ₋₁)), + (α, 0), (xᵢ₊₁, 0), + (xᵢ₊₁, f(xᵢ₊₁)), (xᵢ, f(xᵢ)) + ]; marker=(3, :orange)) + + annotate!(plt, [ + (xᵢ₋₁, 0, text(L"x_{i-1}", :top)), + (xᵢ₋₁, γ*f(xᵢ₋₁), text(L"\gamma \cdot f(x_{i-1})", :right)), + (xᵢ₋₁, f(xᵢ₋₁), text(L"f(x_{i-1})", :right)), + (α, 0, text(L"\alpha", :bottom)), + (xᵢ₊₁, 0, text(L"x_{i+1}", :top)), + (xᵢ, 0, text(L"x_{i}", :top)) + ]) + + plotly() + plt +end +``` + +Modified *regula falsi* method illustration. When midpoint $x_{i+1}$ is on same side of zero $\alpha$ as $x_i$ the *next* step will be between $x_{i-1}$ and $x_{i+1}$. *Were* $x_{i-1}$ modified by $\gamma$ the next midpoint would be an exact zero. If multiplied by a value less, then the midpoint moves to other side of $\alpha$ and would break the repeated choice of a fixed side when keeping a bracketing interval. +::: + + +#### Anderson Bjork + +There are numerous modifications of the *regula falsi* algorithm that +employ different scaling values, we discuss one now. The +[Anderson-Bjork](https://iopscience.iop.org/article/10.1088/1757-899X/1276/1/012010/pdf) +method is a modification of the *regula falsi* method that avoids the +linear convergence when one endpoint is always fixed. + +The modification works as follows, suppose the bracketing interval is $[a,b]$ and $c$ is the point found by the secant line. Then if $f(a)$ and $f(c)$ have the same sign **and** the previous step kept the right side point ($b$) fixed, then instead of using $(c, f(c))$ and $(b, f(c))$ as the new points (as $[c,b]$ is a bracket) use $(b, \gamma \cdot f(b))$ where $\gamma = 1 - f(c)/f(a)$ if $\gamma$ is positive and $\gamma=1/2$ if not. This will modify the next step in the algorithm. The $\gamma$ factors are multiplied each time, so that eventually the $y$ value used at the fixed side should lead to a midpoint on the other side of the zero, as happens when the modified value of $f(x_1)$ and $f(x_5)$ are used to find the midpoint in @fig-anderson-bjork-trajectory. + +::: {#fig-anderson-bjork-trajectory} +```{julia} +#| echo: false +let + gr() + + midpt(a,b,fa,fb) = (a*fb - b*fa)/(fb-fa) + side = nothing + function ABstep!(xs, ys, side, label) + a, b = xs; fa, fb = ys + c = midpt(a,b,fa,fb) + fc = f(c) + + if sign(fa) == sign(fc) + xs[1] = c + ys[1] = fc + if side == :right + m = 1 - fc/fa + m = m < 0 ? 1/2 : m + ys[2] *= m + else + side = :right + end + else + xs[2] = c + ys[2] = fc + if side == :left + m = 1 - fc/fb + m = m < 0 ? 1/2 : m + ys[1] *= m + else + side = :left + end + end + + plot!([(c,0), (c,fc)]; line=(1, :dash, :gray50)) + scatter!([(c,0)]; marker=(5, :blue)) + scatter!(collect(zip(xs, ys)); marker=(5, :orange)) + plot!(collect(zip(xs, ys)); line=(1, :gray50)) + + annotate!([(c, 0, text(label, :top, :left))]) + side + end + + f(x) = 10*log(x)/x^3 + plt = plot(f, 0.8, 3; xlims=(0.7, 3.2), empty_style...) + plot!([(0.8, 0), (3.2, 0)]; line=(1, :black), arrow=true, side=:right) + + α = 1 + x₀, x₁ = 2.75, 0.85 + xs = [x₀, x₁] + ys = f.(xs) + scatter!(plt, collect(zip(xs, ys)); marker=(5, :orange)) + plot!(plt, collect(zip(xs, ys)); line=(1, :gray50)) + plot!(plt, [(x₀,0), (x₀, f(x₀))]; line=(1, :dash, :gray50)) + plot!(plt, [(x₁,0), (x₁, f(x₁))]; line=(1, :dash, :gray50)) + + + side = ABstep!(xs, ys, side, L"x_2") + side = ABstep!(xs, ys, side, L"x_3") + side = ABstep!(xs, ys, side, L"x_4") + side = ABstep!(xs, ys, side, L"x_5") + side = ABstep!(xs, ys, side, L"x_6") + + c = midpt(xs..., ys...) + scatter!([(c,0)]; marker=(5, :blue)) + scatter!([(α,0)]; marker=(5, :green)) + annotate!([(c,0, text(L"x_7", :top, :left)), + (x₀, 0, text(L"x_0", :top, )), + (x₁, 0, text(L"x_1", :top, :right)), + (α, 0, text(L"\alpha", :top, :left)) + ]) + + plotly() + current() +end +``` + +Illustration of Anderson-Bjork algorithm. The point $x_1$ stays as the left-hand endpoint up until $x_6$, but by modifying $f(x_0)$ the algorithm converges super-linearly towards $\alpha$, as compared to @fig-regula-false-convex. + +::: + + +::: {.callout-note} +## Hybrid algorithms + +There are a few, newer, *hybrid algorithms* where some dynamic choice is made as to what update step should be chosen. One due to Chandrapatla (implemented in `Roots.Chandrapatla`) is a bracketing algorithm which chooses between an inverse quadratic step or a bisection step using a certain inequality. We note another, a bracketing algorithm due to Ganchovski and Traykov (with improvements by some `Julia` programmers since its inclusion in the `NonLinearSolve.jl` package) that chooses between bisection or the Anderson-Bjork update based on an estimate of how "straight" the curve is. This is implemented in `Roots.ModAB`. The latter is quite efficient over a wide range of problems. +::: + + + + +##### Examples + +The function $f(x) = (x^2 + 1) \cdot \sin(x) - e^{\sqrt{\lvert x\rvert}} \cdot (x - 1) \cdot (x^2 - 5)$ has a zero *between* $0$ and $1$, and *near* $0.8$. We see how to find it using various algorithms implemented in `Roots`. ```{julia} -#| term: true -chandrapatla(sin, 3, 4, λ3, verbose=true) +f(x) = (x^2 + 1) * sin(x) - exp(sqrt(abs(x))) * (x - 1) * (x^2 - 5) +x0 = 0.8 +xs = (0, 1) ``` +The Steffensen method needs a *nearby* estimate: + +```{julia} +find_zero(f, x0, Roots.Steffensen()) # 5 iterations, 12 function evaluations +``` + +The `Sidi(2)` method needs a *nearby* estimate or an initial two points for a secant line which it bootstraps to get a third point. We use the bracketing interval below: + +```{julia} +find_zero(f, xs, Roots.Sidi(2)) # 3 iterations, 6 function evaluations +``` + +For some steps, `Brent` and `Chandrapatla` use a quadratic inverse calculation, whereas `A42` uses a cubic inverse calculation: + +```{julia} +find_zero(f, xs, Roots.Brent()) # 16 iterations, 18 function evaluations +find_zero(f, xs, Roots.Chandrapatla()) # 20 iterations, 22 function evaluations +find_zero(f, xs, Roots.A42()) # 4 iterations, 10 function evaluations +``` + +Finally, we compare *regula falsi* variants: + +```{julia} +find_zero(f, xs, Roots.RegulaFalsi(:classic)) # 10 iterations, 13 function evaluations +find_zero(f, xs, Roots.RegulaFalsi(:AndersonBjork)) # 6 iterations, 9 function evaluations +find_zero(f, xs, Roots.ModAB()) # 6 iterations, 8 function evaluations +``` + +For this problem, all methods converge to the same zero, but from the counts of iterations and function evaluations they differ in how the get there. + + ## Tolerances +Iterative zero-finding algorithms may mathematically converge, but when implemented on the computer a stopping rule must be articulated. Typically these involve the following: -The `chandrapatla` algorithm typically waits until `abs(b-a) <= 2eps(m)` (where $m$ is either $b$ or $a$ depending on the size of $f(a)$ and $f(b)$) is satisfied. Informally this means the algorithm stops when the two bracketing values are no more than a small amount apart. What is a "small amount?" - - -To understand, we start with the fact that floating point numbers are an approximation to real numbers. - - -Floating point numbers effectively represent a number in scientific notation in terms of - - - * a sign (plus or minus) , - * a *mantissa* (a number in $[1,2)$, in binary ), and - * an exponent (to represent a power of $2$). - - -The mantissa is of the form `1.xxxxx...xxx` where there are $m$ different `x`s each possibly a `0` or `1`. The `i`th `x` indicates if the term `1/2^i` should be included in the value. The mantissa is the sum of `1` plus the indicated values of `1/2^i` for `i` in `1` to `m`. So the last `x` represents if `1/2^m` should be included in the sum. As such, the mantissa represents a discrete set of values, separated by `1/2^m`, as that is the smallest difference possible. - - -For example if `m=2` then the possible value for the mantissa are `11 => 1 + 1/2 + 1/4 = 7/4`, `10 => 1 + 1/2 = 6/4`, `01 => 1 + 1/4 = 5/4`. and `00 => 1 = 4/4`, values separated by `1/4 = 1/2^m`. - - -For $64$-bit floating point numbers `m=52`, so the values in the mantissa differ by `1/2^52 = 2.220446049250313e-16`. This is the value of `eps()`. - - -However, this "gap" between numbers is for values when the exponent is `0`. That is the numbers in `[1,2)`. For values in `[2,4)` the gap is twice, between `[1/2,1)` the gap is half. That is the gap depends on the size of the number. The gap between `x` and its next largest floating point number is given by `eps(x)` and that always satisfies `eps(x) <= eps() * abs(x)`. - - -One way to think about this is the difference between `x` and the next largest floating point values is *basically* `x*(1+eps()) - x` or `x*eps()`. - - -For the specific example, `abs(b-a) <= 2eps(m)` means that the gap between `a` and `b` is essentially 2 floating point values from the $x$ value with the smallest $f(x)$ value. +* stop (and fail) if too many steps are taken +* stop when $\lvert x_i - x_{i-1} \rvert$ is quite small (as then the algorithm stops improving) +* stop when $f(x_i)$ is quite small, as it is close to being zero. + +Small on the computer is a *relative* term and requires a bit of discussion. + +When $\lvert x_i - x_{i-1} \rvert$ is small, we have to recall that the gap between floating point numbers depends on the size of the number, and doubles in going from $[2^{i-1}, 2^i)$ to $[2^i, 2^{i+1})$. As such, a relative tolerance is often chosen so that *small* really means that for some $\epsilon$ + +$$ +\lvert x_i - x_{i-1} \rvert \leq \max(\lvert x_i \rvert, \lvert x_{i-1} \rvert) \cdot \epsilon. +$$ +In code, this might be `abs(b-a) <= 2eps(m)`, which means that the "gap" between `a` and `b` is essentially no more than $2$ floating point values from the $x$ value with the smallest $f(x)$ value. For bracketing methods that is about as good as you can get. However, once floating point values are understood, the absolute best you can get for a bracketing interval would be - * along the way, a value `f(c)` is found which evaluates *exactly* to `0.0` - * the endpoints of the bracketing interval are *adjacent* floating point values, meaning the interval can not be bisected and `f` changes sign between the two values. +* along the way, a value `f(c)` is found which evaluates *exactly* to `0.0` + +* the endpoints of the bracketing interval are *adjacent* floating point values, meaning the interval can not be bisected and `f` changes sign between the two values. -There can be problems when the stopping criteria is `abs(b-a) <= 2eps(m))` and the answer is `0.0` that require engineering around. For example, the algorithm above for the function `f(x) = -40*x*exp(-x)` does not converge when started with `[-9,1]`, even though `0.0` is an obvious zero. +There can be problems when the stopping criteria is `abs(b-a) <= 2eps(m))` and the answer is `0.0` that require engineering around. As such, an *absolute* tolerance might be needed, one where $\lvert x_i - x_{i-1} \rvert \leq \delta$. + +For bracketing algorithms, consideration of $\lvert x_i - x_{i-1} \rvert$ might be all that matters, but not for algorithms like Newton's or the secant algorithm. In Newton's method the update step is $f(x_{i-1})/f'(x_{i-1})$. Naturally when $f(x_i)$ is close to $0$, the update step is small and $\lvert x_{i} - x_{i-1}\rvert = \Delta$ will be close to $0$. *However*, should $f'(x_i)$ be large, then $\Delta$ can also be small and the algorithm will possibly stop, as $x_{i} \approx x_{i-1}$---but not necessarily $x_{i} \approx \alpha$. So termination on $\Delta$ alone can be off. Checking if $f(x_{i})$ is an approximate zero---as it should be if $f$ is continuous---is also useful to include in a stopping criteria. + +However, there may never be a value with `f(x_i)` exactly `0.0`. (The value of `sin(1pi)` is not zero, for example, as `1pi` is an approximation to $\pi$, as well the `sin` of values adjacent to `float(pi)` do not produce `0.0` exactly.) -```{julia} -#| hold: true -#| error: true -fu(x) = -40*x*exp(-x) -chandrapatla(fu, -9, 1, λ3) -``` - -Here the issue is `abs(b-a)` is tiny (of the order `1e-119`) but `eps(m)` is even smaller. - -> For checking if $x_n \approx x_{n+1}$ both a relative and absolute error should be used unless something else is known. +Suppose `x_i` is the closest floating point number to $\alpha$, the mathematical zero. Then the relative rounding error, $($ `x_i` $- \alpha)/\alpha$, will be a value $\delta$ with $\delta$ less than `eps()`. -For non-bracketing methods, like Newton's method or the secant method, different criteria are useful. There may not be a bracketing interval for `f` (for example `f(x) = (x-1)^2`) so the second criteria above might need to be restated in terms of the last two iterates, $x_n$ and $x_{n-1}$. Calling this difference $\Delta = |x_n - x_{n-1}|$, we might stop if $\Delta$ is small enough. As there are scenarios where this can happen, but the function is not at a zero, a check on the size of $f$ is needed. - - -However, there may be no floating point value where $f$ is exactly `0.0` so checking the size of `f(x_n)` requires some agreement. - - -First if `f(x_n)` is `0.0` then it makes sense to call `x_n` an *exact zero* of $f$, even though this may hold even if `x_n`, a floating point value, is not mathematically an *exact* zero of $f$. (Consider `f(x) = x^2 - 2x + 1`. Mathematically, this is identical to `g(x) = (x-1)^2`, but `f(1 + eps())` is zero, while `g(1+eps())` is `4.930380657631324e-32`. - - -However, there may never be a value with `f(x_n)` exactly `0.0`. (The value of `sin(1pi)` is not zero, for example, as `1pi` is an approximation to $\pi$, as well the `sin` of values adjacent to `float(pi)` do not produce `0.0` exactly.) - - -Suppose `x_n` is the closest floating point number to $\alpha$, the zero. Then the relative rounding error, $($ `x_n` $- \alpha)/\alpha$, will be a value $\delta$ with $\delta$ less than `eps()`. - - -How far then can `f(x_n)` be from $0 = f(\alpha)$? +How far then can `f(x_i)` be from $0 = f(\alpha)$? Consider: $$ -f(x_n) = f(x_n - \alpha + \alpha) = f(\alpha + \alpha \cdot \delta) = f(\alpha \cdot (1 + \delta)), +f(x_i) = f(x_i - \alpha + \alpha) = f(\alpha + \alpha \cdot \delta) = f(\alpha \cdot (1 + \delta)), $$ +where $\delta = x_i/\alpha - 1$ is close to $0$ if $x_i$ converges to $\alpha$. + + Assuming $f$ has a derivative, the linear approximation gives: $$ -f(x_n) \approx f(\alpha) + f'(\alpha) \cdot (\alpha\delta) = f'(\alpha) \cdot \alpha \delta +f(x_n) \approx f(\alpha) + f'(\alpha) \cdot (\alpha\delta) = \alpha \cdot f'(\alpha) \cdot \delta $$ -So we should consider `f(x_n)` an *approximate zero* when it is on the scale of $f'(\alpha) \cdot \alpha \delta$. That $\alpha$ factor means we consider a *relative* tolerance for `f`. +So we should consider `f(x_i)` an *approximate zero* when it is on the scale of $\alpha \cdot f'(\alpha) \cdot \delta$. That $\alpha$ factor means we consider a *relative* tolerance, $\delta$, for $f(x_i)$ based on $\lvert x_i\rvert$. -> For checking if $f(x_n) \approx 0$ both a relative and absolute error should be used---the relative error involving the size of $x_n$. - -A good condition to check if `f(x_n)` is small is +As well though, for $\alpha$ values close to $0$ this relative tolerance might be an issue, and a small absolute tolerance can be needed. -`abs(f(x_n)) <= abs(x_n) * rtol + atol`, or `abs(f(x_n)) <= max(abs(x_n) * rtol, atol)` +A good condition to check if `f(x_i)` is small is + +* `abs(f(x_i)) <= abs(x_i) * rtol + atol`, or +* `abs(f(x_i)) <= max(abs(x_i) * rtol, atol)` where the relative tolerance, `rtol`, would absorb an estimate for $f'(\alpha)$. -Now, in Newton's method the update step is $f(x_n)/f'(x_n)$. Naturally when $f(x_n)$ is close to $0$, the update step is small and $\Delta$ will be close to $0$. *However*, should $f'(x_n)$ be large, then $\Delta$ can also be small and the algorithm will possibly stop, as $x_{n+1} \approx x_n$ – but not necessarily $x_{n+1} \approx \alpha$. So termination on $\Delta$ alone can be off. Checking if $f(x_{n+1})$ is an approximate zero is also useful to include in a stopping criteria. + +One thing to keep in mind is that the right-hand side of the rule `abs(f(x_i)) <= abs(x_i) * rtol + atol`, as a function of `x_i`, goes to `Inf` as `x_i` increases. So if `f` has `0` as an asymptote (like `e^(-x)`) for large enough `x_i`, the rule will be `true` and `x_i` could be counted as an approximate zero, despite it not being one. -One thing to keep in mind is that the right-hand side of the rule `abs(f(x_n)) <= abs(x_n) * rtol + atol`, as a function of `x_n`, goes to `Inf` as `x_n` increases. So if `f` has `0` as an asymptote (like `e^(-x)`) for large enough `x_n`, the rule will be `true` and `x_n` could be counted as an approximate zero, despite it not being one. +A modified criteria for convergence might look like: -So a modified criteria for convergence might look like: +* stop if $\Delta$ is small and `f` is an approximate zero with some tolerances - - * stop if $\Delta$ is small and `f` is an approximate zero with some tolerances - * stop if `f` is an approximate zero with some tolerances, but be mindful that this rule can identify mathematically erroneous answers. +* stop if `f` is an approximate zero with some tolerances, but be mindful that this rule can identify mathematically erroneous answers. It is not uncommon to assign `rtol` to have a value like `sqrt(eps())` to account for accumulated floating point errors and the factor of $f'(\alpha)$, though in the `Roots` package it is set smaller by default. @@ -397,12 +554,17 @@ It is not uncommon to assign `rtol` to have a value like `sqrt(eps())` to accoun ### Conditioning and stability -In Part III of @doi:10.1137/1.9781611977165 we find language of numerical analysis useful to formally describe the zero-finding problem. Key concepts are errors, conditioning, and stability. These give some theoretical justification for the tolerances above. +This next part is a technical, mathematical---not practical---motivation for why we might stop when $x_i \approx x_{i-1}$ or $f(x_i) \approx 0$. + +In Part III of @doi:10.1137/1.9781611977165 we find language of numerical analysis useful to formally describe the zero-finding problem. Key concepts are errors, conditioning, and stability, which can be used to give some theoretical justification for the tolerances above. Abstractly a *problem* is a mapping, $F$, from a domain $X$ of data to a range $Y$ of solutions. Both $X$ and $Y$ have a sense of distance given by a *norm*. A norm (denoted with $\lVert\cdot\rVert$) is a generalization of the absolute value and gives quantitative meaning to terms like small and large. +::: {.definition title="Well conditioned problem"} -> A *well-conditioned* problem is one with the property that all small perturbations of $x$ lead to only small changes in $F(x)$. +A *well-conditioned* problem is one with the property that all small perturbations of $x$ lead to only small changes in $F(x)$. + +::: This sense of "small" is measured through a *condition number*. @@ -412,25 +574,32 @@ The *forward error* is $\lVert\delta_F\rVert = \lVert F(x+\delta_x) - F(x)\rVert The *backward error* is $\lVert\delta_x\rVert$, the *relative backward error* is $\lVert\delta_x\rVert / \lVert x\rVert$. - The *absolute condition number* $\hat{\kappa}$ is worst case of this ratio $\lVert\delta_F\rVert/ \lVert\delta_x\rVert$ as the perturbation size shrinks to $0$. -The relative condition number $\kappa$ divides $\lVert\delta_F\rVert$ by $\lVert F(x)\rVert$ and $\lVert\delta_x\rVert$ by $\lVert x\rVert$ before taking the ratio. + The *absolute condition number*, $\hat{\kappa}$, is the worst case of the forward error divided by the backward error, or this ratio $\lVert\delta_F\rVert/ \lVert\delta_x\rVert$, as the perturbation size shrinks to $0$. + +The *relative condition number*, $\kappa$, divides $\lVert\delta_F\rVert$ by $\lVert F(x)\rVert$ and $\lVert\delta_x\rVert$ by $\lVert x\rVert$ before taking the ratio. A *problem* is a mathematical concept, an *algorithm* the computational version. Algorithms may differ for many reasons, such as floating point errors, tolerances, etc. We use notation $\tilde{F}$ to indicate the algorithm. -The absolute error in the algorithm is $\lVert\tilde{F}(x) - F(x)\rVert$, the relative error divides by $\lVert F(x)\rVert$. A good algorithm would have smaller relative errors. +The *absolute error in the algorithm* is $\lVert\tilde{F}(x) - F(x)\rVert$, the relative error divides by $\lVert F(x)\rVert$. A good algorithm would have smaller relative errors. An algorithm is called *stable* if $$ -\frac{\lVert\tilde{F}(x) - F(\tilde{x})\rVert}{\lVert F(\tilde{x})\rVert} +\frac{\lVert\tilde{F}(x) - F(\tilde{x})\rVert}{\lVert F(\tilde{x})\rVert}, $$ is *small* for *some* $\tilde{x}$ relatively near $x$, $\lVert\tilde{x}-x\rVert/\lVert x\rVert$. -> A *stable* algorithm gives nearly the right answer to nearly the right question. +::: {.definition tilte="Stable algorithm"} + +A *stable* algorithm gives nearly the right answer to nearly the right question. + +::: + +The right answer is $F(x)$, the nearly right answer is $F(\tilde{x})$, the nearly right question is $\tilde{F}(x)$. + -(The answer it gives is $\tilde{F}(x)$, the nearly right question: what is $F(\tilde{x})$?) A related concept is an algorithm $\tilde{F}$ for a problem $F$ is *backward stable* if for each $x \in X$, @@ -438,9 +607,15 @@ $$ \tilde{F}(x) = F(\tilde{x}) $$ -for some $\tilde{x}$ where $\lVert\tilde{x} - x\rVert/\lVert x\rVert$ is small. +for *some* $\tilde{x}$ where $\lVert\tilde{x} - x\rVert/\lVert x\rVert$ is small. -> "A backward stable algorithm gives exactly the right answer to nearly the right question." +::: {.definition tilte="Backward stable algorithm"} + +"A backward stable algorithm gives exactly the right answer to nearly the right question." + +::: + +The nearly right question is $\tilde{F}(x)$, the exactly right answer to this is $F(\tilde{x})$. The concepts are related by Trefethen and Bao's Theorem 15.1 which says for a backward stable algorithm the relative error $\lVert\tilde{F}(x) - F(x)\rVert/\lVert F(x)\rVert$ is small in a manner proportional to the relative condition number. @@ -449,7 +624,7 @@ Applying this to the zero-finding we follow @doi:10.1137/1.9781611975086. To be specific, the problem, $F$, is finding a zero of a function $f$ starting at an initial point $x_0$. The data is $(f, x_0)$, the solution is $r$ a zero of $f$. -Take the algorithm as Newton's method. Any implementation must incorporate tolerances, so this is a computational approximation to the problem. The data is the same, but technically we use $\tilde{f}$ for the function, as any computation is dependent on machine implementations. The output is $\tilde{r}$ an *approximate* zero. +For concreteness, take the algorithm as Newton's method. Any implementation must incorporate tolerances, so this is a computational approximation to the problem. The data is the same, but technically we use $\tilde{f}$ for the function, as any computation is dependent on machine implementations. The output is $\tilde{r}$ an *approximate* zero. Suppose for sake of argument that $\tilde{f}(x) = f(x) + \epsilon$, $f$ has a continuous derivative, and $r$ is a root of $f$ and $\tilde{r}$ is a root of $\tilde{f}$. Then by linearization: @@ -463,15 +638,22 @@ $$ $$ Rearranging gives $\lVert\delta/\epsilon\rVert \approx 1/\lVert f'(r)\rVert$. But the $|\delta|/|\epsilon|$ ratio is related to the condition number: -> The absolute condition number is $\hat{\kappa}_r = |f'(r)|^{-1}$. +::: {.definition title="Absolute condition number"} + +The absolute condition number is $\hat{\kappa}_r = |f'(r)|^{-1}$. + +::: The error formula in Newton's method measuring the distance between the actual root and an approximation includes the derivative in the denominator, so we see large condition numbers are tied into possibly larger errors. Now consider $g(x) = f(x) - f(\tilde{r})$. Call $f(\tilde{r})$ the residual. We have $g$ is near $f$ if the residual is small. The algorithm will solve $(g, x_0)$ with $\tilde{r}$, so with a small residual an exact solution to an approximate question will be found. Driscoll and Braun state -> The backward error in a root estimate is equal to the residual. +::: {.relationship title="Backward error and residual"} +The backward error in a root estimate is equal to the residual. + +::: Practically these two observations lead to @@ -630,8 +812,8 @@ choices = [ "The function oscillates too much to rely on the tangent line approximation far from the zero", "We can find an answer" ] -answ = 4 -radioq(choices, answ, keep_order=true) +answer = 4 +radioq(choices, answer, keep_order=true) ``` Does `find_zero` find a zero to this function starting from $0.175$? @@ -664,6 +846,6 @@ choices = [ "The function oscillates too much to rely on the tangent line approximations far from the zero", "We can find an answer" ] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` diff --git a/quarto/derivatives/newtons_method.qmd b/quarto/derivatives/newtons_method.qmd index c09812e..30d5869 100644 --- a/quarto/derivatives/newtons_method.qmd +++ b/quarto/derivatives/newtons_method.qmd @@ -17,14 +17,20 @@ using Roots --- +This section discusses two key algorithms for finding a zero of a real-valued function of a single variable, that is solving $f(x) = 0$. +The bisection method is one such algorithm and requires the knowledge that the zero is **between** two values, which are called a bracketing interval. The two main methods discussed here, Newton's method and the secant method, are more efficient---when they work---and usually require the knowledge of a starting point **near** the desired zero. + + +## The Babylonian method + +We begin with a special purpose algorithm to illustrate the key ideas. The Babylonian method is an algorithm to find an approximate value for $\sqrt{k}$. It was described by the first-century Greek mathematician Hero of [Alexandria](http://en.wikipedia.org/wiki/Babylonian_method). - The method starts with some initial guess, called $x_0$. This is usually some nearby value to the answer. The method then applies a formula to produce an improved guess. This is repeated until the improved guess is accurate enough or it is clear the algorithm fails to work. -For the Babylonian method, the next guess, $x_{i+1}$, is derived from the current guess, $x_i$. In mathematical notation, this is the updating step: +For the Babylonian method, the next guess, $x_{i+1}$, is derived from the current guess, $x_i$ by $$ @@ -33,158 +39,149 @@ $$ We use this algorithm to approximate the square root of $2$, a value known to the Babylonians. - -Start with $x$, then form $x/2 + 1/x$, from this again form $x/2 + 1/x$, repeat. - - -We represent this step using a function - +We start with $x = 2$. In this example, we use rational numbers to keep exact quantities: ```{julia} -babylon(x) = x/2 + 1/x +x₀ = 2//1 +x₁ = x₀/2 + 1/x₀ ``` -Let's look starting with $x = 2$ as a rational number: - +We have $x_0^2 = 4$, what about $x_1^2$? We use a floating point exponent to see the decimal value. ```{julia} -#| hold: true -x₁ = babylon(2//1) -x₁, x₁^2.0 +x₁^2.0 ``` -Our estimate improved from something which squared to $4$ down to something which squares to $2.25.$ A big improvement, but there is still more to come. Had we done one more step: - +A value much closer to $2$. We repeat with another step: ```{julia} -x₂ = (babylon ∘ babylon)(2//1) +x₂ = x₁/2 + 1/x₁ x₂, x₂^2.0 ``` -We now see accuracy until the third decimal point. - +We now see accuracy until the third decimal point. Repeating another time gives even more accuracy: ```{julia} -x₃ = (babylon ∘ babylon ∘ babylon)(2//1) +x₃ = x₂/2 + 1/x₂ x₃, x₃^2.0 ``` -This is now accurate to the sixth decimal point. That is about as far as we, or the Babylonians, would want to go by hand. Using rational numbers quickly grows out of hand. The next step shows the explosion. - +Over rational numbers, the value for the estimate gets more and more complicated, as a peak at the next value shows: ```{julia} -reduce((x,step) -> babylon(x), 1:4, init=2//1) +x₄ = x₃/2 + 1/x₃ +x₄, x₄^2.0 ``` -(In the above, we used `reduce` to repeat a function call $4$ times, as an alternative to the composition operation. In this section we show a few styles to do this repetition before introducing a packaged function.) - - -However, with the advent of floating point numbers, the method stays quite manageable: - +This is not the case over floating point numbers, where we see increasing convergence towards $\sqrt{2}$. ```{julia} -#| hold: true -xₙ = reduce((x, step) -> babylon(x), 1:6, init=2.0) -xₙ, xₙ^2 +float.([x₀, x₁, x₂, x₃, x₄]) .- sqrt(2) ``` -We can see that the algorithm - to the precision offered by floating point numbers - has resulted in an answer `1.414213562373095`. This answer is an *approximation* to the actual answer. Approximation is necessary, as $\sqrt{2}$ is an irrational number and so can never be exactly represented in floating point. That being said, we can see that the value of $f(x)$ is accurate to the last decimal place, so our approximation is very close and is achieved in a few steps. +We see this algorithm rapidly converges to $\sqrt{2}$. In fact, in two more steps it will get as close as machine precision will allow a floating point number to approximate an irrational number. The algorithm produces *approximations* to the actual answer which can be easily computed in a few steps to a desired tolerance and, if needed, repeated more often to near exactness. -## Newton's generalization +## Newton's method + +Is there some generalization to the Babylonian method that applies to non-linear functions? -Let $f(x) = x^3 - 2x -5$. The value of $2$ is almost a zero, but not quite, as $f(2) = -1$. We can check that there are no *rational* roots. Though there is a method to solve the cubic it may be difficult to compute and will not be as generally applicable as some algorithm like the Babylonian method to produce an approximate answer. - - -Is there some generalization to the Babylonian method? +Let $f(x) = x^3 - 2x -5$. The value of $2$ is almost a zero, but not quite, as $f(2) = -1$. We can check that there are no *rational* roots. Though there is a method to solve the cubic it may be difficult to compute and will not be as generally applicable as some iterative algorithm like the Babylonian method to produce an approximate answer to a non-linear problem. We know that the tangent line is a good approximation to the function at the point. Looking at this graph gives a hint as to an algorithm: +::: {#fig-plot-x-3-minus-2-x-minus-5} ```{julia} -#| hold: true #| echo: false -f(x) = x^3 - 2x - 5 -fp(x) = 3x^2 - 2 -c = 2 -p = plot(f, 1.75, 2.25, legend=false) -plot!(x->f(2) + fp(2)*(x-2)) -plot!(zero) -scatter!(p, [c], [f(c)], color=:orange, markersize=3) -p +let + gr() + f(x) = x^3 - 2x - 5 + fp(x) = 3x^2 - 2 + c = 2 + + plt = plot(;empty_style..., xlims=(1.75, 2.25)) + plot!(plt, [(1.75,0.0), (2.25,0.0)]; line=(1, :gray), arrow=true, side=:right) + + plot!(plt, f; line=(2, :black)) + plot!(plt, x->f(c) + fp(c)*(x-c); line=(1, :black)) + scatter!(plt, [(c, f(c))]; marker=(5, :orange)) + ticks = 1.8:0.1:2.2 + annotate!(plt, [(x, 0.0, text(latexstring(x), :top)) for x in ticks]) + annotate!([(c, f(c), text(L"(c, f(c))", :top, :left))]) + for x in ticks + plot!(plt, [(x,0.0), (x, 0.1)]; line=(1, :gray)) + end + plotly() + plt +end ``` -The tangent line and the function nearly agree near $2$. So much so, that the intersection point of the tangent line with the $x$ axis nearly hides the actual zero of $f(x)$ that is near $2.1$. -That is, it seems that the intersection of the tangent line and the $x$ axis should be an improved approximation for the zero of the function. +Plot of $f(x) = x^3 - 2x - 5$ and a tangent line at a point, $(c, f(c))$, near a $0$ +::: + +The tangent line and the function nearly agree near $2$. So much so, that the intersection point of the tangent line with the $x$ axis and the intersection of $f(x)$ with the $x$ axis are nearly the same value. + +The key observation is: the intersection of the tangent line and the $x$ axis should be an improved approximation for the zero of the function. -Let $x_0$ be $2$, and $x_1$ be the intersection point of the tangent line at $(x_0, f(x_0))$ with the $x$ axis. Then by the definition of the tangent line: +Let $x_0$ be the initial estimate for a zero, and $x_1$ be the intersection point of the tangent line at $(x_0, f(x_0))$ with the $x$ axis. Then by the definition of the tangent line: $$ -f'(x_0) = \frac{\Delta y }{\Delta x} = \frac{f(x_0)}{x_0 - x_1}. +f'(x_0) = \frac{\Delta y }{\Delta x} = \frac{f(x_1) - f(x_0)}{x_1 - x_0} = \frac{0 - f(x_0)}{x_1 - x_0}. $$ -This can be solved for $x_1$ to give $x_1 = x_0 - f(x_0)/f'(x_0)$. In general, if we had $x_i$ and used the intersection point of the tangent line to produce $x_{i+1}$ we would have Newton's method: +This can be solved for $x_1$ to give $x_1 = x_0 - f(x_0)/f'(x_0)$. In general, if our current approximation is $x_i$ and used the intersection point of the tangent line to produce $x_{i+1}$ we would have Newton's method: $$ x_{i+1} = x_i - \frac{f(x_i)}{f'(x_i)}. $$ -Using automatic derivatives, as brought in with the `CalculusWithJulia` package, we can implement this algorithm. - - -The algorithm above starts at $2$ and then becomes: +---- +Using automatic derivatives, as brought in with the `CalculusWithJulia` package, we can implement this algorithm step by step. Starting at $x_0=2$ we have: ```{julia} f(x) = x^3 - 2x - 5 -x0 = 2.0 -x1 = x0 - f(x0) / f'(x0) +x₀ = 2.0 +x₁ = x₀ - f(x₀) / f'(x₀) +x₁, f(x₁) ``` -We can see we are closer to a zero: +We can see we are closer to a zero. Repeating, we have: + +```{julia} +x₂ = x₁ - f(x₁)/ f'(x₁) +x₂, f(x₂) +``` + +And: ```{julia} -f(x0), f(x1) -``` - -Trying again, we have - - -```{julia} -x2 = x1 - f(x1)/ f'(x1) -x2, f(x2), f(x1) -``` - -And again: - - -```{julia} -x3 = x2 - f(x2)/ f'(x2) -x3, f(x3), f(x2) +x₃ = x₂ - f(x₂)/ f'(x₂) +x₃, f(x₃) ``` ```{julia} -x4 = x3 - f(x3)/ f'(x3) -x4, f(x4), f(x3) +x₄ = x₃ - f(x₃)/ f'(x₃) +x₄, f(x₄) ``` -We see now that $f(x_4)$ is within machine tolerance of $0$, so we call $x_4$ an *approximate zero* of $f(x)$. +We see now that $f(x_4)$ is within machine tolerance of $0$ and that if we were to try another iteration we would find $x_{i+1} \approx x_i$. We call $x_4$ an *approximate zero* of $f(x)$. -::: {.callout-note icon=false} -## Newton's method +::: {.definition title="Newton's method"} -Let $x_0$ be an initial guess for a zero of $f(x)$. Iteratively define $x_{i+1}$ in terms of the just generated $x_i$ by: +Let $x_0$ be an initial guess for a zero of $f(x)$. Iteratively define $x_{i+1}$ in terms of $x_i$ by: $$ -x_{i+1} = x_i - f(x_i) / f'(x_i). +x_{i+1} = x_i - \frac{f(x_i)}{f'(x_i)}. $$ Then for reasonable functions and reasonable initial guesses, the sequence of points converges to a zero of $f$. @@ -192,19 +189,20 @@ Then for reasonable functions and reasonable initial guesses, the sequence of po ::: -On the computer, we know that actual convergence will likely never occur, but accuracy to a certain tolerance can often be achieved. +On the computer, we know that actual convergence will likely never occur, but accuracy to a certain tolerance---either with $\lvert x_{i+1} - x_i\rvert$ or $\lvert f(x_i) \rvert$---can often be achieved. -In the example above, we kept track of the previous values. This is unnecessary if only the answer is sought. In that case, the update step could use the same variable. Here we use `reduce`: - +In the example above, we tediously kept track of each value to match the formula for the update step of Newton's method. However, the subscripting mathematically is used to specify assignment, as opposed to an equation, and that is exactly what the equals sign does in `Julia`, so we could have just done these steps: ```{julia} -#| hold: true -xₙ = reduce((x, step) -> x - f(x)/f'(x), 1:4, init=2) -xₙ, f(xₙ) +x = 2.0 +x = x - f(x) / f'(x) +x = x - f(x) / f'(x) +x = x - f(x) / f'(x) +x = x - f(x) / f'(x) ``` -In practice, the algorithm is implemented not by repeating the update step a fixed number of times, rather by repeating the step until either we converge or it is clear we won't converge. For good guesses and most functions, convergence happens quickly. +In practice, the algorithm is implemented not by repeating the update step a fixed number of times, rather by repeating the step until either we "converge" or it is clear we won't converge. For good guesses and most functions, convergence happens quickly. :::{.callout-note} @@ -226,10 +224,9 @@ Raphson (1690) proposed a simplification avoiding the computation of new polynom @fig-newtons-method demonstrates the method and the rapid convergence: - +::: {#fig-newtons-method} ```{julia} #| echo: false -nothing function newtons_method_graph(n, f, a, b, c; label=false) xstars = [c] @@ -254,159 +251,31 @@ function newtons_method_graph(n, f, a, b, c; label=false) plot!(plt, xs, ys, color=:orange) scatter!(plt, xstars, 0*xstars, color=:orange, markersize=5) if label - subs = collect("₁₂₃₄₅₆₇₈₉") - labs = ["x$(subs[i])" for i in eachindex(xstars)] - annotate!(collect(zip(xstars, 0*xstars, labs,[:bottom for _ in xstars]))) + annotate!(plt, [(xᵢ, 0, text(latexstring("x_$(i-1)"), :bottom,:left)) for (i, xᵢ) in enumerate(xstars)]) end plt end -``` -```{julia} -#| hold: true -#| echo: false -#| cache: true -#| label: fig-newtons-method -### {{{newtons_method_example}}} -gr() -caption = """ +let + gr() + caption = "" + n = 5 + + fn, a, b, c = x->log(x), .15, 2, .2 + + anim = @animate for i=1:n + newtons_method_graph(i-1, fn, a, b, c; label=true) + end + + imgfile = tempname() * ".gif" + gif(anim, imgfile, fps = 1) + plotly() + ImageFile(imgfile, caption) +end +``` Illustration of Newton's Method converging to a zero of a function. - -""" -n = 6 - -fn, a, b, c = x->log(x), .15, 2, .2 - -anim = @animate for i=1:n - newtons_method_graph(i-1, fn, a, b, c; label=true) -end - -imgfile = tempname() * ".gif" -gif(anim, imgfile, fps = 1) -plotly() -ImageFile(imgfile, caption) -``` - ---- - - -This interactive graphic (built using [JSXGraph](https://jsxgraph.uni-bayreuth.de/wp/index.html)) allows the adjustment of the point `x0`, initially at $0.85$. Five iterations of Newton's method are illustrated. Different positions of `x0` clearly converge, others will not. - - -```{=html} -
-``` - -```{ojs} -//| echo: false -//| output: false - -JXG = require("jsxgraph"); - -// newton's method - -b = JXG.JSXGraph.initBoard('jsxgraph', { - boundingbox: [-3,5,3,-5], axis:true -}); - - -f = function(x) {return x*x*x*x*x - x - 1}; -fp = function(x) { return 4*x*x*x*x - 1}; -x0 = 0.85; - -nm = function(x) { return x - f(x)/fp(x);}; - -l = b.create('point', [-1.5,0], {name:'', size:0}); -r = b.create('point', [1.5,0], {name:'', size:0}); -xaxis = b.create('line', [l,r]) - - -P0 = b.create('glider', [x0,0,xaxis], {name:'x0'}); -P0a = b.create('point', [function() {return P0.X();}, - function() {return f(P0.X());}], {name:''}); - -P1 = b.create('point', [function() {return nm(P0.X());}, - 0], {name:''}); -P1a = b.create('point', [function() {return P1.X();}, - function() {return f(P1.X());}], {name:''}); - -P2 = b.create('point', [function() {return nm(P1.X());}, - 0], {name:''}); -P2a = b.create('point', [function() {return P2.X();}, - function() {return f(P2.X());}], {name:''}); - -P3 = b.create('point', [function() {return nm(P2.X());}, - 0], {name:''}); -P3a = b.create('point', [function() {return P3.X();}, - function() {return f(P3.X());}], {name:''}); - -P4 = b.create('point', [function() {return nm(P3.X());}, - 0], {name:''}); -P4a = b.create('point', [function() {return P4.X();}, - function() {return f(P4.X());}], {name:''}); -P5 = b.create('point', [function() {return nm(P4.X());}, - 0], {name:'x5', strokeColor:'black'}); - - - - - -P0a.setAttribute({fixed:true}); -P1.setAttribute({fixed:true}); -P1a.setAttribute({fixed:true}); -P2.setAttribute({fixed:true}); -P2a.setAttribute({fixed:true}); -P3.setAttribute({fixed:true}); -P3a.setAttribute({fixed:true}); -P4.setAttribute({fixed:true}); -P4a.setAttribute({fixed:true}); -P5.setAttribute({fixed:true}); - -sc = '#000000'; -b.create('segment', [P0,P0a], {strokeColor:sc, strokeWidth:1}); -b.create('segment', [P0a, P1], {strokeColor:sc, strokeWidth:1}); -b.create('segment', [P1,P1a], {strokeColor:sc, strokeWidth:1}); -b.create('segment', [P1a, P2], {strokeColor:sc, strokeWidth:1}); -b.create('segment', [P2,P2a], {strokeColor:sc, strokeWidth:1}); -b.create('segment', [P2a, P3], {strokeColor:sc, strokeWidth:1}); -b.create('segment', [P3,P3a], {strokeColor:sc, strokeWidth:1}); -b.create('segment', [P3a, P4], {strokeColor:sc, strokeWidth:1}); -b.create('segment', [P4,P4a], {strokeColor:sc, strokeWidth:1}); -b.create('segment', [P4a, P5], {strokeColor:sc, strokeWidth:1}); - -b.create('functiongraph', [f, -1.5, 1.5]) - -``` - -##### Example: numeric not algebraic - - -For the function $f(x) = \cos(x) - x$, we see that SymPy can not solve symbolically for a zero: - - -```{julia} -#| error: true -@syms x::real -solve(cos(x) - x, x) -``` - -We can find a numeric solution, even though there is no closed-form answer. Here we try Newton's method: - - -```{julia} -#| hold: true -f(x) = cos(x) - x -x = .5 -x = x - f(x)/f'(x) # 0.7552224171056364 -x = x - f(x)/f'(x) # 0.7391416661498792 -x = x - f(x)/f'(x) # 0.7390851339208068 -x = x - f(x)/f'(x) # 0.7390851332151607 -x = x - f(x)/f'(x) -x, f(x) -``` - -To machine tolerance the answer is a zero, even though the exact answer is irrational and all finite floating point values can be represented as rational numbers. +::: ##### Example non-polynomial @@ -426,43 +295,64 @@ x = x - f(x) / fp(x) x, f(x) ``` - -##### Example +##### Example: numeric not algebraic -Use Newton's method to find the *largest* real solution to $e^x = x^6$. +For the function $f(x) = \cos(x) - x$ consider this SymPy code to symbolically solve for a zero: +```{julia} +#| error: true +#| eval: false +@syms x::real +solve(cos(x) ~ x, x) +``` -A plot shows us roughly where the value lies: +Were this run it would produce an error + +``` +NotImplementedError('multiple generators [x, cos(x)] + No algorithms are implemented to solve equation -x + cos(x)') +``` + +Non-linear equations may not have exact symbolic answers. However, +With Newton's method we can readily find a numeric solution, even though there is no closed-form answer. ```{julia} #| hold: true -f(x) = exp(x) -g(x) = x^6 -plot(f, 0, 25; label="f") -plot!(g; label="g") +f(x) = cos(x) - x +x = 0.5 +x = x - f(x)/f'(x) # 0.7552224171056364 +x = x - f(x)/f'(x) # 0.7391416661498792 +x = x - f(x)/f'(x) # 0.7390851339208068 +x = x - f(x)/f'(x) # 0.7390851332151607 +x = x - f(x)/f'(x) +x, f(x) ``` -Clearly by $20$ the two paths diverge. We know exponentials eventually grow faster than powers, and this is seen in the graph. +To machine tolerance the answer is a zero, even though the exact answer is irrational and all finite floating point values can be represented as rational numbers. -To use Newton's method to find the intersection point. Stop when the increment $f(x)/f'(x)$ is smaller than `1e-4`. We need to turn the solution to an equation into a value where a function is $0$. Just moving the terms to one side of the equals sign gives $e^x - x^6 = 0$, or the $x$ we seek is a solution to $h(x)=0$ with $h(x) = e^x - x^6$. +##### Example +Use Newton's method to find the *largest* real solution to $e^x = x^6$. A plot shows that that answer is *near* $x=20$, so we begin there. To use Newton's method to find an intersection point, we create a new function which is zero when the two functions are equal through subtraction. + +For this problem we use a loop to illustrate the progression of the algorithm: + ```{julia} #| hold: true #| term: true h(x) = exp(x) - x^6 x = 20 -for step in 1:10 +for step in 1:11 delta = h(x)/h'(x) x = x - delta @show step, x, delta end ``` -So it takes $8$ steps to get an increment that small and about `10` steps to get to full convergence. +By the ninth step, the increment `delta`---which tracks $\lvert x_{i+1} - x_i \rvert$ is negligible and the algorithm has stopped improving. The approximate zero found is `16.99888735229605`. ##### Example division as multiplication @@ -490,9 +380,10 @@ $$ x_{i+1} = x_i - (1/x_i - q)/(-1/x_i^2) = -qx^2_i + 2x_i. $$ -Now for $q$ in the interval $[1/2, 1]$ we want to get a *good* initial guess. Here is a claim: we can use $x_0=48/17 - 32/17 \cdot q$. Let's check graphically that this is a reasonable initial approximation to $1/q$: +Now for $q$ in the interval $[1/2, 1]$ we want to get a *good* initial guess. Here is a claim: we can use $x_0=48/17 - 32/17 \cdot q$. We check graphically in @fig-plot-1-over-q-and-reasonable-initial-approximation that this is a reasonable initial approximation to $1/q$. +::: {#fig-plot-1-over-q-and-reasonable-initial-approximation} ```{julia} #| hold: true @@ -500,6 +391,9 @@ plot(q -> 1/q, 1/2, 1, label="1/q") plot!(q -> 1/17 * (48 - 32q), label="linear approximation") ``` +The linear approximation shows good starting point for Newton's method +::: + It can be shown that we have for any $q$ in $[1/2, 1]$ with initial guess $x_0 = 48/17 - 32/17\cdot q$ that Newton's method will converge to $16$ digits in no more than this many steps: @@ -507,15 +401,14 @@ $$ \log_2(\frac{53 + 1}{\log_2(17)}). $$ +Computing, we see that four steps suffices. + ```{julia} a = log2((53 + 1)/log2(17)) ceil(Integer, a) ``` -That is $4$ steps suffices. - - -For $q = 0.80$, to find $1/q$ using the above we have +Now we try to find $1/q$ when $q=0.8 = 4/5$ without dividing by $q$. ```{julia} @@ -528,13 +421,12 @@ x = -q*x*x + 2*x x = -q*x*x + 2*x ``` -This method has basically $18$ multiplication and addition operations for one division, so it naively would seem slower, but timing this shows the method is competitive with a regular division. +If values for `48/17` and `32/17` are pre-computed, this method has basically $18$ multiplication and addition operations for one division, so it naively would seem slower, but timing this shows the method is competitive with a regular division. -## Wrapping in a function +## Automating Newton's method - -In the previous examples, we saw fast convergence, guaranteed converge in $4$ steps, and an example where $8$ steps were needed to get the requested level of approximation. Newton's method usually converges quickly, but may converge slowly, and may not converge at all. Automating the task to avoid repeatedly running the update step is a task best done by the computer. +In the previous examples, we saw fast convergence, guaranteed converge in $4$ steps, and an example where $9$ steps were needed to get convergence. Newton's method usually converges quickly, but may converge slowly, and may not converge at all. Automating the task to avoid repeatedly running the update step is a task best done by the computer. The `while` loop is a good way to repeat commands until some condition is met. With this, we present a simple function implementing Newton's method, we iterate until the update step gets really small (the `atol`) or the convergence takes more than $50$ steps. (There are other, better choices that could be used to determine when the algorithm should stop, these are just easy to understand.) @@ -558,7 +450,7 @@ end ##### Examples - * Find a zero of $\sin(x)$ starting at $x_0=3$: +* Find a zero of $\sin(x)$ starting at $x_0=3$: ```{julia} @@ -568,7 +460,7 @@ nm(sin, cos, 3) This is an approximation for $\pi$, that historically found use, as the convergence is fast. - * Find a solution to $x^5 = 5^x$ near $2$: +* Find a solution to $x^5 = 5^x$ near $2$: Writing a function to handle this, we have: @@ -586,13 +478,13 @@ alpha = nm(k, k', 2) alpha, k(alpha) ``` -### Functions in the Roots package +### `Roots.Newton()` -Typing in the `nm` function might be okay once, but would be tedious if it was needed each time. Besides, it isn't as robust to different inputs as possible. The `Roots` package provides a `Newton` method for `find_zero`. +Typing in the `nm` function might be okay once, but would be tedious if it was needed each time. Besides, it isn't as robust to different inputs as possible. The `Roots` package provides a `Newton` method for its `find_zero` function. -To use a different method with `find_zero`, the calling pattern is `find_zero(f, x, M)` where `f` represent the function(s), `x` the initial point(s), and `M` the method. Here we have: +To use a different method with `find_zero`, the calling pattern is `find_zero(f, x, M)` where `f` represent the function(s), `x` the initial point(s), and `M` the method. For `Newton` we have: ```{julia} @@ -608,16 +500,25 @@ f(x) = sin(x) find_zero((f, f'), 2, Roots.Newton()) ``` -The argument `verbose=true` will force a print out of a message summarizing the convergence and showing each step. +The `Newton` method isn't exported, so it is qualified via `Roots.Newton()`. +##### Example: solving $f(x) = c$ for non-zero $c$ + +Find a value for which `erf(x) = 0.75`. + +The `erf` function is increasing, so there is just one value and an exploratory graph shows the answer to be near $1$. + +We can apply Newton's method, but first we need to restate the problem in terms of some function equaling $0$. This can be done directly, but we do it in two steps here: + ```{julia} -#| hold: true -f(x) = exp(x) - x^4 -find_zero((f,f'), 8, Roots.Newton(); verbose=true) +f(x) = erf(x) +c = 0.75 +h(x) = f(x) - c +find_zero((h, h'), 1.0, Roots.Newton()) ``` -##### Example: intersection of two graphs +##### Example: intersection of two graphs, or solving $f(x) = g(x)$ Find the intersection point between $f(x) = \cos(x)$ and $g(x) = 5x$ near $0$. @@ -631,8 +532,8 @@ We have Newton's method to solve for zeros of $f(x)$, i.e. when $f(x) = 0$. Here f(x) = cos(x) g(x) = 5x h(x) = f(x) - g(x) -x0 = find_zero((h,h'), 0, Roots.Newton()) -x0, h(x0), f(x0), g(x0) +x0 = find_zero((h, h'), 0, Roots.Newton()) +x0, h(x0), f(x0) - g(x0) ``` --- @@ -649,7 +550,7 @@ xn = find_zero((f,fp), pi/4, Roots.Newton(); p=5) xn, f(xn, 5) ``` -To use automatic differentiation is not straightforward, as we must hold the `p` fixed. For this, we introduce a closure that fixes `p` and differentiates in the `x` variable (called `u` below): +To use automatic differentiation with a parameter is not straightforward, as we must hold the `p` fixed. For this, we introduce a closure that fixes `p` and differentiates in the `x` variable (called `u` below): ```{julia} @@ -659,48 +560,184 @@ fp(x,p) = (u -> f(u,p))'(x) xn = find_zero((f,fp), pi/4, Roots.Newton(); p=5) ``` -##### Example: Finding $c$ in Rolle's Theorem +##### Example: finding $c$ in Rolle's Theorem -The function $r(x) = \sqrt{1 - \cos(x^2)^2}$ has a zero at $0$ and one at $a$ near $1.77$. +The function $r(x) = \sqrt{1 - \cos(x^2)^2}$ has a zero at $0$ and one at $a$ near $1.77$, as can be seen in @fig-sqrt-1-cos-x-squared-squared. +::: {#fig-sqrt-1-cos-x-squared-squared} ```{julia} +#| echo: false r(x) = sqrt(1 - cos(x^2)^2) -plot(r, 0, 1.77) +tks = 0:0.5:1.5 +plot(r, 0, 1.77, xticks = (vcat(tks, 1.77), vcat(string.(tks), L"a"))) ``` +Plot of $r(x) = \sqrt{1 - \cos(x^2)^2}$ over $[0, 1.77]$ +::: + + As $f(x)$ is differentiable between $0$ and $a$, Rolle's theorem says there will be value where the derivative is $0$. Find that value. -This value will be a zero of the derivative. A graph shows it should be near $1.2$, so we use that as a starting value to get the answer: +This value will be a zero of the derivative. @fig-sqrt-1-cos-x-squared-squared shows it should be near $1.2$, so we use that as a starting value to get the answer: ```{julia} find_zero((r',r''), 1.2, Roots.Newton()) ``` +##### Example: seeing the trace + +The steps of Newton's method can be see by passing a `Roots.Tracks` object. We name it `tracks` in this example to take advantage of `Julia`'s handling of matching variables with keywords with the same name ([argument destructuring](https://docs.julialang.org/en/v1/manual/functions/#man-argument-destructuring)). + +Consider finding a zero of $f(x) = e^{x} - x^{\pi}$. There is one near $2$. + +We have two additional steps to see the trace, first we create a `tracks` object. + +```{julia} +f(x) = exp(x) - x^pi +x0 = 2 +tracks = Roots.Tracks() +find_zero((f, f'), x0, Roots.Newton(); tracks) +``` + +The we display the `tracks` object to see the steps taken by the algorithm along with some diagnostic details: + +```{julia} +tracks +``` + +## The secant method + +The secant method is an alternative to Newton's method which uses secant lines instead of tangent lines in the update step. Like Newton's method, the secant method is iterative. Unlike Newton's method---which uses just the previous value to identify the next value---the secant method uses the two previous values in its update step. + +::: {.relationship title="Secant method"} + +Let $x_0$ and $x_1$ be two different estimates for $c$, a zero of $f(x)$. The iterative algorithm with update step + +$$ +x_{i+1} = x_i - \frac{x_{i} - x_{i-1}}{f(x_{i}) - f(x_{i-1})} \cdot f(x_i) +$$ + +is called the secant method. +::: + +The multiplier of $f(x_i)$ is the reciprocal of the slope of the secant line from $(x_i, f(x_i))$ and $(x_{i-1}, f(x_{i-1}))$---in contrast to Newton's method which uses the reciprocal of the slope of the tangent line at $(x_i, f(x_i))$. + +The secant method can be preferred if either a function's evaluation or a function's derivative evaluation are difficult to find. + +##### Example + +Find a zero of $f(x) = \cos(x) - x$ using the secant method starting from $x_0, x_1 = 0, \pi/2$. + +```{julia} +f(x) = cos(x) - x +xi_1, xi = 0, pi/2 +xi, xi_1 = xi - (xi - xi_1) / (f(xi) - f(xi_1)) * f(xi), xi # x2 = 0.6110154703516573 +xi, xi_1 = xi - (xi - xi_1) / (f(xi) - f(xi_1)) * f(xi), xi # x3 = 0.7232695414357495 +xi, xi_1 = xi - (xi - xi_1) / (f(xi) - f(xi_1)) * f(xi), xi # x4 = 0.739567106974727 +xi, xi_1 = xi - (xi - xi_1) / (f(xi) - f(xi_1)) * f(xi), xi # x5 = 0.7390834365030763 +xi, xi_1 = xi - (xi - xi_1) / (f(xi) - f(xi_1)) * f(xi), xi # x6 = 0.739085133034638 +xi, xi_1 = xi - (xi - xi_1) / (f(xi) - f(xi_1)) * f(xi), xi # x7 = 0.7390851332151608 +``` + +This example takes six iterations to reach convergence to machine tolerance. Newton's method takes only five, as in general it converges more rapidly. However, the secant method takes just one function evaluation per step, or $6$ in total not counting the initial two values; Newton's method takes $2$ per step or $10$ in total, not counting the initial value. In general this holds: the secant method might take more steps, but will require fewer function calls. + + +::: {.callout-note} +## Origin of secant method + +[Papakonstantinou and Tapia](https://www.jstor.org/stable/10.4169/amer.math.monthly.120.06.500?origin=JSTOR-pdf) discuss the origin of the secant method tracing it back to the rule of double false position used in some manner since the 18th century BC which in modern language translates to one step of the secant method and when applied to a linear function gives a solution. + +::: + +### `Roots.Secant()` + +The `Roots` package has a `Secant` method for `find_zero` to carry out this method. We redo the last example, using a `Roots.tracks` object so we can see the algorithm, though in most cases this is not of interest. + + +```{julia} +f(x) = cos(x) - x +xs = (0, pi/2) +tracks = Roots.Tracks() +find_zero(f, xs, Secant(); tracks) +``` + +The tracks are identical up to floating point differences to those in the comments in the previous example. + +```{julia} +tracks +``` + + +#### Default method for a nearby initial guess + +The `find_zero` function has a default, `Roots.Order0()`, when a single *nearby* value is specified as a starting point. This method use a secant method (after using an approximate derivative to get the second step) up *until* convergence *or* the values $x_{i-1}$ and $x_i$ form a bracketing interval. If the latter happens, then a bracketing method is used to find convergence. (Bracketing methods have guaranteed convergence). + +For example, we might have: + +```{juiia} +empty!(tracks) # can empty tracks or create a new one +x0 = first(xs) # just a single value, not even a good initial guess here +find_zero(f, x0; tracks) # no method specified so defaults to Order0() +``` + +The trace shows the steps of the secant method until a bracket is identified and then the brackets up to convergence. + +```{julia} +tracks +``` + +The `Order0` method isn't quite as convergent as Newton's method, but is a bit more robust to some of that methods idiosyncrasies that are discussed later in this section. + ## Convergence rates -Newton's method is famously known to have "quadratic convergence." What does this mean? Let the error in the $i$th step be called $e_i = x_i - \alpha$. Then Newton's method satisfies a bound of the type: +Newton's method is famously known to have "quadratic convergence". What does this mean? + +When it works, Newton's method forms a sequence $x_0, x_1, x_2, \dots$, converging to a zero, $\alpha$ of some function $f(x)$. + +Define error in the $i$th step by: + +$$ +e_i = x_i - \alpha. +$$ + +We take the *order of convergence* to be the value $p$ for which + + +$$ +\lim_{n \rightarrow \infty} \frac{e_{n+1}}{e_n^p} = L > 0. +$$ + +We say, the sequence converges with order $p$. Quadratic convergence is when $p=2$. + + + +### Convergence of Newton's method + +We will see Newton's method satisfies a bound like this: $$ \lvert e_{i+1} \rvert \leq M_i \cdot e_i^2. $$ -If $M$ were just a constant and we suppose $e_0 = 10^{-1}$ then $e_1$ would be less than $M 10^{-2}$ and $e_2$ less than $M^2 10^{-4}$, $e_3$ less than $M^3 10^{-8}$ and $e_4$ less than $M^4 10^{-16}$ which for $M=1$ is basically the machine precision when values are near $1$. That is for some problems, with a good initial guess it will take around $4$ or so steps to converge. +In fact, we will see that under assumptions the value for $M$ will converge to $f''(\alpha)/(2f'(\alpha))$. + +If $M$ were just a constant in the above and we suppose a good initial guess, say with $e_0 = 10^{-1}$, then $e_1$ would be less than $M 10^{-2}$ and $e_2$ less than $M^2 10^{-4}$, $e_3$ less than $M^3 10^{-8}$ and $e_4$ less than $M^4 10^{-16}$ which for $M=1$ is basically the machine precision when values are near $1$. That is for some problems, with a good initial guess it will take around $4$ or so steps to converge. -To identify $M$, let $\alpha$ be the zero of $f$ to be approximated. Assume +To identify $M$, *assume* + +* The function $f$ has a continuous *second* derivative in a neighborhood of $\alpha$. + +* The value $f'(\alpha)$ is *non-zero* in the neighborhood of $\alpha$.^[This property says that this is a *simple* zero or the zero has *multiplicity* of $1$.] - * The function $f$ has a continuous second derivative in a neighborhood of $\alpha$. - * The value $f'(\alpha)$ is *non-zero* in the neighborhood of $\alpha$. - - -Then this linearization holds at each $x_i$ in the above neighborhood: +Then the Lagrange remainder form for linearization holds at each $x_i$ in the above neighborhood: $$ @@ -710,14 +747,14 @@ $$ The value $\xi$ is from the mean value theorem and is between $x$ and $x_i$. -Dividing by $f'(x_i)$ and setting $x=\alpha$ (as $f(\alpha)=0$) leaves +Setting $x=\alpha$ (as $f(\alpha)=0$) and dividing by $f'(x_i)$ leaves: $$ -0 = \frac{f(x_i)}{f'(x_i)} + (\alpha-x_i) + \frac{1}{2}\cdot \frac{f''(\xi)}{f'(x_i)} \cdot (\alpha-x_i)^2. +0 = \frac{f(\alpha)}{f'(x_i)} = \frac{f(x_i)}{f'(x_i)} + (\alpha-x_i) + \frac{1}{2}\cdot \frac{f''(\xi)}{f'(x_i)} \cdot (\alpha-x_i)^2. $$ -For this value, we have +We can write $e_{i+1}$ in terms of $x_i$ and the update step and simplify using the above relationship: $$ @@ -733,7 +770,7 @@ x_{i+1} - \alpha $$ -That is +That is, $M$ can be read off from this equality: $$ @@ -743,38 +780,42 @@ $$ This convergence to $\alpha$ will be quadratic *if*: - * The initial guess $x_0$ is not too far from $\alpha$, so $e_0$ is managed. - * The derivative at $\alpha$ is not too close to $0$, hence, by continuity $f'(x_i)$ is not too close to $0$. (As it appears in the denominator). That is, the function can't be too flat, which should make sense, as then the tangent line is nearly parallel to the $x$ axis and would intersect far away. - * The function $f$ has a continuous second derivative at $\alpha$. - * The second derivative is not too big (in absolute value) near $\alpha$. A large second derivative means the function is very concave, which means it is "turning" a lot. In this case, the function turns away from the tangent line quickly, so the tangent line's zero is not necessarily a good approximation to the actual zero, $\alpha$. +* The initial guess $x_0$ is *near* $\alpha$, so $e_0$ is managed. + +* The derivative at $\alpha$ is not too close to $0$, hence, by continuity $f'(x_i)$ is not too close to $0$. (As it appears in the denominator). That is, the function can't be too flat, which should make sense, as then the tangent line is nearly parallel to the $x$ axis and would intersect far away or the algorithm can get trapped by a local extrema. + +* The function $f$ has a continuous second derivative at $\alpha$. + +* The second derivative is not too big (in absolute value) near $\alpha$. A large second derivative means the function is very concave, which means it is "turning" a lot. In this case, the function turns away from the tangent line quickly, so the tangent line's zero is not necessarily a good approximation to the actual zero, $\alpha$. :::{.callout-note} -## Note -The basic tradeoff: methods like Newton's are faster than the bisection method in terms of function calls, but are not guaranteed to converge, as the bisection method is. +## Tradeoffs + +The bisection method has linear convergence, in that $\lvert e_{i+1} \rvert \approx (1/2) \lvert e_i \rvert$, but *guaranteed* to converge. + +Newton's method is quadratic, so *can* converge in a few steps---but convergence is not guaranteed. ::: + + + +### When Newton's method fails + What can go wrong when one of these isn't the case is illustrated next: -### Poor initial step - +#### Poor initial guess +::: {#fig-poor-initial-step-newtons-method} ```{julia} #| hold: true #| echo: false #| cache: true ### {{{newtons_method_poor_x0}}} gr() -caption = """ - -Illustration of Newton's Method converging to a zero of a function, -but slowly as the initial guess, is very poor, and not close to the -zero. The algorithm does converge in this illustration, but not quickly and not to the nearest root from -the initial guess. - -""" +caption = "" fn, a, b, c = x -> sin(x) - x/4, -15, 20, 2pi @@ -789,23 +830,22 @@ plotly() ImageFile(imgfile, caption) ``` + +Illustration of Newton's Method converging to a zero of a function, +but slowly as the initial guess, is very poor, and not close to the +zero. The algorithm does converge in this illustration, but not quickly and not to the nearest root from +the initial guess. + +::: + +::: {#fig-poor-initial-step-newtons-method-example-2} ```{julia} #| hold: true #| echo: false #| cache: true # {{{newtons_method_flat}}} gr() -caption = L""" - -Illustration of Newton's method failing to converge as for some $x_i$, -$f'(x_i)$ is too close to ``0``. In this instance after a few steps, the -algorithm just cycles around the local minimum near $0.66$. The values -of $x_i$ repeat in the pattern: $1.0002, 0.7503, -0.0833, 1.0002, -\dots$. This is also an illustration of a poor initial guess. If there -is a local minimum or maximum between the guess and the zero, such -cycles can occur. - -""" +caption = "" fn, a, b, c = x -> x^5 - x + 1, -1.5, 1.4, 0.0 @@ -819,8 +859,19 @@ plotly() ImageFile(imgfile, caption) ``` -### The second derivative is too big +Illustration of Newton's method failing to converge as for some $x_i$, +$f'(x_i)$ is too close to $0$. In this instance after a few steps, the +algorithm just cycles around the local minimum near $0.66$. The values +of $x_i$ repeat in the pattern: $1.0002, 0.7503, -0.0833, 1.0002, +\dots$. This is also an illustration of a poor initial guess. If there +is a local minimum or maximum between the guess and the zero, such +cycles can occur. +::: + +#### The second derivative is too big + +::: {#fig-second-derivative-too-big-newtons-method} ```{julia} #| hold: true @@ -829,14 +880,7 @@ ImageFile(imgfile, caption) # {{{newtons_method_cycle}}} gr() fn, a, b, c, = x -> abs(x)^(0.49), -2, 2, 1.0 -caption = L""" - -Illustration of Newton's Method not converging. Here the second -derivative is too big near the zero - it blows up near $0$ - and the -convergence does not occur. Rather the iterates increase in their -distance from the zero. - -""" +caption = "" n=10 anim = @animate for i=1:n @@ -850,31 +894,24 @@ plotly() ImageFile(imgfile, caption) ``` -### The tangent line at some xᵢ is flat +Illustration of Newton's Method not converging. Here the second +derivative is too big near the zero - it blows up near $0$ - and the +convergence does not occur. Rather the iterates increase in their +distance from the zero. + +::: +#### The tangent line at some xᵢ is flat + +::: {#fig-tangent-line-too-flat-at-x0} ```{julia} #| hold: true #| echo: false #| cache: true # {{{newtons_method_wilkinson}}} gr() -caption = L""" - -The function $f(x) = x^{20} - 1$ has two bad behaviours for Newton's -method: for $x < 1$ the derivative is nearly $0$ and for $x>1$ the -second derivative is very big. In this illustration, we have an -initial guess of $x_0=8/9$. As the tangent line is fairly flat, the -next approximation is far away, $x_1 = 1.313\dots$. As this guess -is much bigger than $1$, the ratio $f(x)/f'(x) \approx -x^{20}/(20x^{19}) = x/20$, so $x_i - f(x_i)/f'(x_i) \approx (19/20)x_i$ -yielding slow, linear convergence until $f''(x_i)$ is moderate. For -this function, starting at $x_0=8/9$ takes 11 steps, at $x_0=7/8$ -takes 13 steps, at $x_0=3/4$ takes ``55`` steps, and at $x_0=1/2$ it takes -$204$ steps. - -""" - +caption = "" fn,a,b,c = x -> x^20 - 1, .7, 1.4, 8/9 n = 10 @@ -888,10 +925,25 @@ plotly() ImageFile(imgfile, caption) ``` -###### Example +The function $f(x) = x^{20} - 1$ has two bad behaviours for Newton's +method: for $x < 1$ the derivative is nearly $0$ and for $x>1$ the +second derivative is very big. In this illustration, we have an +initial guess of $x_0=8/9$. As the tangent line is fairly flat, the +next approximation is far away, $x_1 = 1.313\dots$. As this guess +is much bigger than $1$, the ratio $f(x)/f'(x) \approx +x^{20}/(20x^{19}) = x/20$, so $x_i - f(x_i)/f'(x_i) \approx (19/20)x_i$ +yielding slow, linear convergence until $f''(x_i)$ is moderate. For +this function, starting at $x_0=8/9$ takes 11 steps, at $x_0=7/8$ +takes 13 steps, at $x_0=3/4$ takes ``55`` steps, and at $x_0=1/2$ it takes +$204$ steps. + +::: -Suppose $\alpha$ is a simple zero for $f(x)$. (The value $\alpha$ is a zero of multiplicity $k$ if $f(x) = (x-\alpha)^kg(x)$ where $g(\alpha)$ is not zero. A simple zero has multiplicity $1$. If $f'(\alpha) \neq 0$ and the second derivative exists, then a zero $\alpha$ will be simple.) Around $\alpha$, quadratic convergence should apply. However, consider the function $g(x) = f(x)^k$ for some integer $k \geq 2$. Then $\alpha$ is still a zero, but the derivative of $g$ at $\alpha$ is zero, so the tangent line is basically flat. This will slow the convergence up. We can see that the update step $g(x)/g'(x)$ becomes $(1/k) f(x)/f'(x)$, so an extra factor is introduced. +###### Example: roots with multiplicity more than one + + +The assumption that $f'(\alpha)$ is non zero says $\alpha$ is a simple zero for $f(x)$. Near enough around $\alpha$, quadratic convergence should apply. However, consider the function $g(x) = f(x)^k$ for some integer $k \geq 2$. Then $\alpha$ is still a zero, but the derivative of $g$ at $\alpha$ is zero, so the tangent line is basically flat. This will slow the convergence up. We can see that the update step $g(x)/g'(x)$ becomes $(1/k) f(x)/f'(x)$, so an extra factor is introduced. The calculation that produces the quadratic convergence now becomes: @@ -904,7 +956,36 @@ x_{i+1} - \alpha &= (x_i - \alpha) - \frac{1}{k}(x_i-\alpha - \frac{f''(\xi)}{2f \end{align*} $$ -As $k > 1$, the $(x_i - \alpha)$ term dominates, and we see the convergence is linear with $\lvert e_{i+1}\rvert \approx (k-1)/k \lvert e_i\rvert$. +As $k > 1$, the $(x_i - \alpha)$ term dominates, and we see the convergence is linear with $\lvert e_{i+1}\rvert \approx \left((k-1)/k\right) \lvert e_i\rvert$. + +### Convergence of the secant method + +As above, let $\epsilon_{n+1} = x_{n+1}-\alpha$ and *assume* $f'(\alpha) \neq 0$, or $\alpha$ is a *simple* zero of $f(x)$. + + +With a more involved derivation than that for Newton's method, a [calculation](https://math.okstate.edu/people/binegar/4513-F98/4513-l08.pdf) shows that + + +$$ +\begin{align*} +\epsilon_{n+1} +& \approx \frac{f''(\alpha)}{2f'(\alpha)} \epsilon_n \epsilon_{n-1}\\ +&= C \epsilon_n \epsilon_{n-1}. +\end{align*} +$$ + + +The constant `C` is similar to that for Newton's method, and reveals potential troubles for the secant method similar to those of Newton's method: a poor initial guess (the initial error is too big), the second derivative is too large, the first derivative too flat near the answer. + + +Assuming the error term has the form $\lvert \epsilon_{n+1}\rvert = A|\epsilon_n|^\phi$ and substituting into the above leads to the equation + + +$$ +\frac{A^{1+1/\phi}}{C} = |\epsilon_n|^{1 - \phi +1/\phi}. +$$ + +The left side being a constant suggests $\phi$ solves: $1 - \phi + 1/\phi = 0$ or $\phi^2 -\phi - 1 = 0$. The solution is the golden ratio, $(1 + \sqrt{5})/2 \approx 1.618\dots$. That is convergence is super linear, but not quadratic, as Newton's method is. ## Questions @@ -913,20 +994,27 @@ As $k > 1$, the $(x_i - \alpha)$ term dominates, and we see the convergence is l ###### Question -Look at this graph with $x_0$ marked with a point: - +@fig-graph-of-airyai-minus-3.3-to-0-x0-minus-2-point-8 shows a graph of some $f(x)$ with $x_0$ marked with a point: +::: {#fig-graph-of-airyai-minus-3.3-to-0-x0-minus-2-point-8} ```{julia} #| hold: true #| echo: false import SpecialFunctions: airyai -p = plot(airyai, -3.3, 0, legend=false); -plot!(p, zero, -3.3, 0); -scatter!(p, [-2.8], [0], color=:orange, markersize=5); -annotate!(p, [(-2.8, 0.2, "x₀")]) -p +let + gr() + p = plot(airyai, -3.3, 0; legend=false); + plot!(p, zero, -3.3, 0); + scatter!(p, [(-2.8, 0)], marker=(:orange, 5)); + annotate!(p, [(-2.8, 0.0, text(L"x_0", :top))]) + plotly() + p +end ``` +Plot of $f(x)$ with a zero and an initial starting point for Newton's method marked with $x_0$. +::: + If one step of Newton's method was used, what would be the value of $x_1$? @@ -941,19 +1029,23 @@ radioq(choices, answ, keep_order=true) ###### Question -Look at this graph of some increasing, concave up $f(x)$ with initial point $x_0$ marked. Let $\alpha$ be the zero. - +@fig-plot-some-increasing-concave-up-function show a graph of some increasing, concave up $f(x)$ with initial point $x_0$ marked. Let $\alpha$ be the zero. +::: {#fig-plot-some-increasing-concave-up-function} ```{julia} #| hold: true #| echo: false p = plot(x -> x^2 - 2, .75, 2.2, legend=false); plot!(p, zero, color=:green); -scatter!(p, [1],[0], color=:orange, markersize=5); -annotate!(p, [(1,.25, "x₀"), (sqrt(2), .2, "α")]); +scatter!(p, [(1,0)], color=:orange, markersize=5); +annotate!(p, [(1,0, text(L"x_0",:top, :left)), + (sqrt(2), 0, text(L"\alpha", :top))]); p ``` +Graph of an increasing, concave up function. +::: + What can be said about $x_1$? @@ -972,19 +1064,27 @@ radioq(choices, answ) --- -Look at this graph of some increasing, concave up $f(x)$ with initial point $x_0$ marked. Let $\alpha$ be the zero. - +@fig-plot-some-increasing-concave-up-function-start-on-right is a graph of some increasing, concave up $f(x)$ with initial point $x_0$ marked. Let $\alpha$ be the zero. +::: {#fig-plot-some-increasing-concave-up-function-start-on-right} ```{julia} #| hold: true #| echo: false -p = plot(x -> x^2 - 2, .75, 2.2, legend=false); -plot!(p, zero, .75, 2.2, color=:green); -scatter!(p, [2],[0], color=:orange, markersize=5); -annotate!(p, [(2,.25, "x₀"), (sqrt(2), .2, "α")]); -p +let + gr() + plt = plot(x -> x^2 - 2, .75, 2.2; empty_style...) + plot!(plt, zero; line=(:green,), arrow=true, side=:right) + scatter!(plt, [2],[0], marker = (:orange, 5)) + annotate!(plt, [ + (2, 0, text(L"x_0",:top, :left)), + (sqrt(2), 0, text(L"\alpha", :top))]) + plotly() + plt +end ``` +Graph of an increasing, concave up function. +::: What can be said about $x_1$? @@ -1066,13 +1166,56 @@ xstar = Roots.newton(f, fp, 8); numericq(xstar, 1e-1) ``` +###### Question + +Let $f(x) = \exp(x) - x^4$. As mentioned, there are 3 zeros for this function. What does the *secant* method converge to when started with $x_0 = 0, x_1 = 3$? + + +```{julia} +#| hold: true +#| echo: false +f(x) = exp(x) - x^4; +xs = (0, 3) +xstar = find_zero(f, xs, Secant()) +numericq(xstar, 1e-1) +``` + +###### Question + +Let $f(x) = \exp(x) - x^4$. As mentioned, there are 3 zeros for this function. What does the *secant* method converge to when started with $x_0 = 3, x_1 = 6$? + + +```{julia} +#| hold: true +#| echo: false +f(x) = exp(x) - x^4; +xs = (3, 6) +xstar = find_zero(f, xs, Secant()) +numericq(xstar, 1e-1) +``` + +###### Question + +Let $f(x) = \exp(x) - x^4$. As mentioned, there are 3 zeros for this function. What does the *secant* method converge to when started with $x_0 = 6, x_1 = 9$? + + +```{julia} +#| hold: true +#| echo: false +f(x) = exp(x) - x^4; +xs = (6, 9) +xstar = find_zero(f, xs, Secant()) +numericq(xstar, 1e-1) +``` + + ###### Question Let $f(x) = \sin(x) - \cos(4\cdot x)$. -Starting at $\pi/8$, solve for the root returned by Newton's method +Starting at $\pi/8$, solve for the root returned by Newton's method. ```{julia} @@ -1085,6 +1228,26 @@ val = Roots.newton(f, fp, pi/(2k1)); numericq(val) ``` + +###### Question + + +Let $f(x) = \sin(x) - \cos(4\cdot x)$. + + +Starting at $x_0 = 0, x_1 = 1$, solve for the root returned by secant method. + + +```{julia} +#| hold: true +#| echo: false +k1=4 +f(x) = sin(x) - cos(k1*x) +xs = (0, 1) +val = find_zero(f, xs, Secant()) +numericq(val) +``` + ###### Question @@ -1112,7 +1275,6 @@ f(x) = x^5 + x - 1 val = Roots.newton(f,f', -1) numericq(val) ``` - ###### Question @@ -1129,14 +1291,19 @@ nothing For the following graph, graphically consider the algorithm for a few different starting points. - +::: {#fig-newton-method-consider-x0} ```{julia} #| hold: true #| echo: false # placeholder until CWJ bumps up a version? -plot(x -> x^5 - x - 1, -1, 2) +plot(x -> x^5 - x - 1, -1, 1.5; legend=false, line=(:black,)) +plot!(zero; line=(:black,)) ``` +Plot of $f(x)$. Consider Newton's method for different initial values. +::: + + If $x_0$ is $1$ what occurs? @@ -1359,14 +1526,18 @@ Will Newton's method find the zero at $0.0$ starting at $1$? yesnoq("no") ``` -Considering this plot: - +Consider the graph in $fig-newton-baffler-minus-1-point-1-to-1-point-1. +::: {#fig-newton-baffler-minus-1-point-1-to-1-point-1} ```{julia} #| hold: true -plot(newton_baffler, -1.1, 1.1) +plot(newton_baffler, -1.1, 1.1; label="newton baffler") +plot!(zero; label="zero") ``` +Plot of Newton baffler function +::: + Starting with $x_0=1$, you can see why Newton's method will fail. Why? @@ -1388,7 +1559,7 @@ This function does not have a small first derivative; or a large second derivati ###### Question -Let $f(x) = \sin(x) - x/4$. Starting at $x_0 = 2\pi$ Newton's method will converge to a value, but it will take many steps. Using the argument `verbose=true` for `find_zero`, how many steps does it take: +Let $f(x) = \sin(x) - x/4$. Starting at $x_0 = 2\pi$ Newton's method will converge to a value, but it will take many steps. Using a `tracks` argument to find how many steps it takes. ```{julia} @@ -1424,7 +1595,7 @@ yesnoq("no") ###### Question -Quadratic convergence of Newton's method only applies to *simple* roots. For example, we can see (using the `verbose=true` argument to the `Roots` package's `newton` method), that it only takes $4$ steps to find a zero to $f(x) = \cos(x) - x$ starting at $x_0 = 1$. But it takes many more steps to find the same zero for $f(x) = (\cos(x) - x)^2$. +Quadratic convergence of Newton's method only applies to *simple* roots. For example, we can see (using a `tracks` argument), that it only takes $4$ steps to find a zero to $f(x) = \cos(x) - x$ starting at $x_0 = 1$. But it takes many more steps to find the same zero for $f(x) = (\cos(x) - x)^2$. How many? @@ -1440,9 +1611,9 @@ numericq(val, 2) ###### Question: Implicit equations -The equation $x^2 + x\cdot y + y^2 = 1$ is a rotated ellipse. - +The equation $x^2 + x\cdot y + y^2 = 1$ is a rotated ellipse and is graphed in @fig-implicit-plot-of-rotated-ellipse. +::: {#fig-implicit-plot-of-rotated-ellipse} ```{julia} #| hold: true #| echo: false @@ -1451,6 +1622,9 @@ f(x,y) = x^2 + x * y + y^2 - 1 implicit_plot(f, xlims=(-2,2), ylims=(-2,2), legend=false) ``` +Plot of a rotated ellipse +::: + Can we find which point on its graph has the largest $y$ value? @@ -1465,7 +1639,7 @@ function findy(x) end ``` -For a *fixed* x, this solves for $y$ in the equation: $F(y) = x^2 + x \cdot y + y^2 - 1 = 0$. It should be that $(x,y)$ is a solution: +For a *fixed* $x$, this solves for $y$ in the equation: $F(y) = x^2 + x \cdot y + y^2 - 1 = 0$. It should be that $(x,y)$ is a solution: ```{julia} @@ -1584,3 +1758,6 @@ yesnoq(false) ``` All methods work quickly with this well-behaved problem. In general the convergence rates are slightly different for each, with the Steffensen method matching Newton's method and the difference quotient method being slower in general. All can be more sensitive to the initial guess. + + +##### Question diff --git a/quarto/derivatives/numeric_derivatives.qmd b/quarto/derivatives/numeric_derivatives.qmd index b144cb7..349d158 100644 --- a/quarto/derivatives/numeric_derivatives.qmd +++ b/quarto/derivatives/numeric_derivatives.qmd @@ -1,4 +1,4 @@ -# Numeric derivatives +# Computing derivatives in Julia {{< include ../_common_code.qmd >}} @@ -19,10 +19,10 @@ using Roots --- -`SymPy` returns symbolic derivatives. Up to choices of simplification, these answers match those that would be derived by hand. This is useful when comparing with known answers and for seeing the structure of the answer. However, there are times we just want to work with the answer numerically. For that we have other options within `Julia`. We discuss approximate derivatives and automatic derivatives. The latter will find wide usage in these notes. +`SymPy` returns symbolic derivatives. Up to choices of simplification, these answers match those that would be derived by hand. This is useful when comparing with known answers and for seeing the structure of the answer. However, there are times we just want to work with the answer numerically. For that we have other options within `Julia`. We discuss approximate derivatives and automatic derivatives in this section. The latter will find wide usage in these notes. -### Approximate derivatives +## Approximate derivatives By approximating the limit of the secant line with a value for a small, but positive, $h$, we get an approximation to the derivative. That is @@ -32,24 +32,25 @@ $$ f'(x) \approx \frac{f(x+h) - f(x)}{h}. $$ -This is the forward-difference approximation. The central difference approximation looks both ways: +This is the *forward-difference approximation*. The *central difference approximation* looks to both sides of $x$: $$ f'(x) \approx \frac{f(x+h) - f(x-h)}{2h}. $$ -Though in general they are different, they are both approximations. The central difference is usually more accurate for the same size $h$. However, both are susceptible to round-off errors. The numerator is a subtraction of like-size numbers - a perfect opportunity to lose precision. +Though in general they are different, both are easy and fast to compute, useful approximations to the derivative. The central difference is usually more accurate for the same size $h$. However, both are susceptible to round-off errors. The numerator is a subtraction of like-size numbers---a perfect opportunity to lose precision. -As such there is a balancing act: +Due to numeric issues there is a balancing act: + +* if $h$ is too big the approximation to the limit is not good. + +* if $h$ is too small the round-off errors are problematic, - * if $h$ is too small the round-off errors are problematic, - * if $h$ is too big the approximation to the limit is not good. - -For the forward difference $h$ values around $10^{-8}$ are typically good, for the central difference, values around $10^{-6}$ are typically good. +For the forward difference $h$ values around $10^{-8}$ are typically good, for the central difference, values around $10^{-6}$ are typically good, but these ranges aren't always the case. ##### Example @@ -78,7 +79,7 @@ abs(factual - fapprox) The error is about $1$ part in $100$ million. -The central difference is better here: +The central difference is better here, even with a bigger $h$:^[The [FiniteDifferences](https://github.com/JuliaDiff/FiniteDifferences.jl) and [FiniteDiff](https://github.com/JuliaDiff/FiniteDiff.jl) packages provide performant interfaces for differentiation based on finite differences.] ```{julia} @@ -88,49 +89,53 @@ cdapprox = (f(c+h) - f(c-h)) / (2h) abs(factual - cdapprox) ``` ---- - - -The [FiniteDifferences](https://github.com/JuliaDiff/FiniteDifferences.jl) and [FiniteDiff](https://github.com/JuliaDiff/FiniteDiff.jl) packages provide performant interfaces for differentiation based on finite differences. - ### Automatic derivatives +Roughly speaking, symbolic derivatives are exact, but can be slow to compute, whereas approximate derivatives are not exact, but fast to compute. However, the widely used automatic derivatives are both exact and fast to compute. -There are some other ways to compute derivatives numerically that give much more accuracy at the expense of slightly increased computing time. Automatic differentiation is the general name for a few different approaches. These approaches promise less complexity - in some cases - than symbolic derivatives and more accuracy than approximate derivatives; the accuracy is on the order of machine precision. +Automatic differentiation (AD) is the general name for a few different approaches. We utilize forward mode automatic differentiation. The `ForwardDiff` package provides one of [several](https://juliadiff.org/) ways for `Julia` to compute automatic derivatives. `ForwardDiff` is well suited for functions encountered in these notes, which depend on at most a few variables and output no more than a few values at once. - -The `ForwardDiff` package provides one of [several](https://juliadiff.org/) ways for `Julia` to compute automatic derivatives. `ForwardDiff` is well suited for functions encountered in these notes, which depend on at most a few variables and output no more than a few values at once. - - -The `ForwardDiff` package was loaded in this section; in general its features are available when the `CalculusWithJulia` package is loaded, as that package provides a more convenient interface. The `derivative` function is not exported by `ForwardDiff`, so its usage requires qualification. To illustrate, to find the derivative of $f(x)$ at a *point* we have this syntax: +The `ForwardDiff` package was loaded in this section; in general its features are available when the `CalculusWithJulia` package is loaded, as that package provides a more convenient interface. The `derivative` function is not exported by `ForwardDiff`, so its usage requires qualification. To illustrate, to find the derivative of $f(x)$ at a *point* we have this syntax for `ForwardDiff`: ```{julia} ForwardDiff.derivative(f, c) # derivative is qualified by a module name ``` -The `CalculusWithJulia` package defines an operator `D` which goes from finding a derivative at a point with `ForwardDiff.derivative` to defining a function which evaluates the derivative at each point. It is defined along the lines of `D(f) = x -> ForwardDiff.derivative(f,x)` in parallel to how the derivative operation for a function is defined mathematically from the definition for its value at a point. +The `CalculusWithJulia` package defines a method for `'` (a postfix operator in `Julia`) so that when applied to a `Function` object a function for computing values of the derivative, as above, is returned.^[The `CalculusWithJulia` package defines a method for `Base.adjoint(f::Function)`. In `Julia` speak this is a form of *type piracy*, as the package modifies a method (`adjoint`) for a type (`Function`) neither of which belongs to the package. As well, the meaning given does not conform with the expected generic meaning of the operation elsewhere in the `Julia` ecosystem. Admittedly this is bad form, but a useful deviation from good practice for pedagogical reasons.] + + +To be clear, this usage returns a function that computes the derivative of `f`: + +```{julia} +f' +``` + +And this usage *calls* the derivative of `f` at the value stored by `c`: + +```{julia} +f'(c) +``` + +This is the same mathematical notation that is commonly used. Here we see the error in estimating $f'(1)$: ```{julia} -fauto = D(f)(c) # D(f) is a function, D(f)(c) is the function called on c -abs(factual - fauto) +abs(factual - f'(c)) ``` -In this case, it is exact. +In this case, the automatic derivative is exact. -The `D` operator is defined for most all functions in `Julia`, though, like the `diff` operator in `SymPy` there are some for which it won't work. - ##### Example -For $f(x) = \sqrt{1 + \sin(\cos(x))}$ compare the difference between the forward derivative with $h=1e-8$ and that computed by `D` at $x=\pi/4$. +For $f(x) = \sqrt{1 + \sin(\cos(x))}$ compare the difference between the forward derivative with $h=1e-8$ and that computed by automatic differentiation at $x=\pi/4$. The forward derivative is found with: @@ -142,11 +147,11 @@ c, h = pi/4, 1e-8 fwd = (f(c+h) - f(c))/h ``` -That given by `D` is: +That given by automatic differentiation is: ```{julia} -ds_value = D(f)(c) +ds_value = f'(c) ds_value, fwd, ds_value - fwd ``` @@ -158,38 +163,25 @@ fp = diff(f(x), x) ``` ```{julia} -actual = convert(Float64, fp(PI/4)) +actual = float(fp(PI/4)) actual - ds_value, actual - fwd ``` -#### Convenient notation - - -`Julia` allows the possibility of extending functions to different types. Out of the box, the `'` notation is not employed for functions, but is used for matrices. It is used in postfix position, as with `A'`. We can define it to do the same thing as `D` for functions and then, we can evaluate derivatives with the familiar `f'(x)`. This is done in `CalculusWithJulia` along the lines of `Base.adjoint(f::Function) = D(f)`. - - -Then, we have, for example: - - -```{julia} -#| hold: true -f(x) = sin(x) -f'(pi), f''(pi) -``` +As expected, the automatic derivative is nearly exact and accurate up to possibly accumulated floating point differences; the forward difference is just pretty close. ##### Example -Suppose our task is to find a zero of the second derivative of $k(x) = e^{-x^2/2}$ in $[0, 10]$, a known bracket. The `D` function takes a second argument to indicate the order of the derivative (e.g., `D(f,2)`), but we use the more familiar notation: +Suppose our task is to find a zero of the second derivative of $k(x) = e^{-x^2/2}$ in $[0, 10]$, a known bracket. The second derivative is found by `k''` below, with the derivative operation being applied twice behind the scenes. With this, we have: ```{julia} #| hold: true k(x) = exp(-x^2/2) -find_zero(k'', 0..10) +find_zero(k'', (0, 10)) ``` -We pass in the function object, `k''`, and not the evaluated function. +As with plotting and other uses of functions as arguments, we pass in the function object, `k''`, and not the function evaluated at a point. ## Recap on derivatives in Julia @@ -197,9 +189,11 @@ We pass in the function object, `k''`, and not the evaluated function. A quick summary of the $3$ different ways for finding derivatives in `Julia` presented in these notes: - * Symbolic derivatives are found using `diff` from `SymPy` - * Automatic derivatives are found using the notation `f'` which utilizes `ForwardDiff.derivative` - * approximate derivatives at a point, `c`, for a given `h` are found with `(f(c+h)-f(c))/h`. +* Symbolic derivatives are found using `diff` from `SymPy` + +* Automatic derivatives are found using the notation `f'` which utilizes `ForwardDiff.derivative` + +* approximate derivatives at a point, `c`, for a given `h` are found with `(f(c+h)-f(c))/h`. For example, here all three are computed and compared: @@ -207,21 +201,17 @@ For example, here all three are computed and compared: ```{julia} #| hold: true -f(x) = exp(-x)*sin(x) +f(x) = exp(-x) * sin(x) c = pi h = 1e-8 fp = diff(f(x),x) -fp, fp(c), f'(c), (f(c+h) - f(c))/h +Dict(:approximate=>(f(c+h) - f(c))/h, :automatic=>f'(c), + :symbolic=>fp, :symbolic_evaluated => fp(x=>c)) ``` -:::{.callout-note} -## Note -The use of `'` to find derivatives provided by `CalculusWithJulia` is convenient, and used extensively in these notes, but it needs to be noted that it does **not conform** with the generic meaning of `'` within `Julia`'s wider package ecosystem and may cause issue with linear algebra operations; the symbol is meant for the adjoint of a matrix. - -::: ## Questions @@ -241,7 +231,7 @@ val = (f(c+h) - f(c))/h numericq(val) ``` -Using `D` or `f'` find the value using automatic differentiation +Use `f'` find the value using automatic differentiation ```{julia} @@ -253,44 +243,11 @@ val = f'(c) numericq(val) ``` -###### Question - - -Mathematically, as the value of `h` in the forward difference gets smaller the forward difference approximation gets better. On the computer, this is thwarted by floating point representation issues (in particular the error in subtracting two like-sized numbers in forming $f(x+h)-f(x)$.) - - -For `1e-16` what is the error (in absolute value) in finding the forward difference approximation for the derivative of $\sin(x)$ at $x=0$? - - -```{julia} -#| hold: true -#| echo: false -f(x) = sin(x) -h = 1e-16 -c = 0 -approx = (f(c+h)-f(c))/h -val = abs(cos(c) - approx) -numericq(val) -``` - -Repeat for $x=\pi/4$: - - -```{julia} -#| hold: true -#| echo: false -f(x) = sin(x) -h = 1e-16 -c = pi/4 -approx = (f(c+h)-f(c))/h -val = abs(cos(c) - approx) -numericq(val) -``` ###### Question -Let $f(x) = x^x$. Using `D`, find $f'(3)$. +Let $f(x) = x^x$. Using automatic differentiation, find $f'(3)$. ```{julia} @@ -304,7 +261,7 @@ numericq(val) ###### Question -Let $f(x) = \lvert 1 - \sqrt{1 + x}\rvert$. Using `D`, find $f'(3)$. +Let $f(x) = \lvert 1 - \sqrt{1 + x}\rvert$. Using automatic differentation, find $f'(3)$. ```{julia} @@ -318,7 +275,7 @@ numericq(val) ###### Question -Let $f(x) = e^{\sin(x)}$. Using `D`, find $f'(3)$. +Let $f(x) = e^{\sin(x)}$. Using automatic differentation, find $f'(3)$. ```{julia} @@ -371,3 +328,83 @@ fp_(h) = 3*32h^2 - 62 c = 2 numericq(fp_(2)) ``` + + +###### Question + + +Mathematically, as the value of `h` in the forward difference gets smaller the forward difference approximation gets better. On the computer, this is thwarted by floating point representation issues (in particular the error in subtracting two like-sized numbers in forming $f(x+h)-f(x)$.) + + +For `1e-16` what is the error (in absolute value) in finding the forward difference approximation for the derivative of $\sin(x)$ at $x=0$? + + +```{julia} +#| hold: true +#| echo: false +f(x) = sin(x) +h = 1e-16 +c = 0 +approx = (f(c+h)-f(c))/h +val = abs(cos(c) - approx) +numericq(val) +``` + +Repeat for $x=\pi/4$: + + +```{julia} +#| hold: true +#| echo: false +f(x) = sin(x) +h = 1e-16 +c = pi/4 +approx = (f(c+h)-f(c))/h +val = abs(cos(c) - approx) +numericq(val) +``` + + +##### Question + +Let $f(x) = e^{-x^2/2}$. At $c=2$ we can compare the exact answer for the derivative to the forward difference approximation for different values of `h`. For example: + +```{julia} +f(x) = exp(-x^2/2) +c = 2 +@syms x +exact = float(diff(f(x), x)(x=>2)) +``` + +Whereas, + +```{julia} +hs = [1/10^i for i in 0:16] +fdiffs = [(f(c+h) - f(c))/h for h in hs] +error = fdiffs .- exact +[hs error] +``` + +Which value of `h` provided the smallest error? Write the exponent as `i` in `1/10^i`. + +```{julia} +#| echo: false +numericq(8) +``` + + +We repeat with the central difference approximation. + +```{julia} +hs = [1/10^i for i in 0:16] +cdiffs = [(f(c+h) - f(c-h))/(2h) for h in hs] +error = cdiffs .- exact +[hs error] +``` + +Which value of `h` provided the smallest error? Write the exponent as `i` in `1/10^i`. + +```{julia} +#| echo: false +numericq(6) +``` diff --git a/quarto/derivatives/optimization.qmd b/quarto/derivatives/optimization.qmd index df306c0..a74ff6d 100644 --- a/quarto/derivatives/optimization.qmd +++ b/quarto/derivatives/optimization.qmd @@ -30,8 +30,11 @@ For example, The main tool is the extreme value theorem of Bolzano and Fermat's theorem about critical points, which combined say: +::: {.relationship title="Bolazano plus Fermat"} -> If the function $f(x)$ is continuous on $[a,b]$ and differentiable on $(a,b)$, then the extrema exist and must occur at either an end point or a critical point. +If the function $f(x)$ is continuous on $[a,b]$ then the extrema exist and must occur at either an end point or a critical point. + +::: @@ -52,14 +55,14 @@ With the computer we can take some shortcuts, as we will be able to graph our fu The simplest way to investigate the maximum or minimum value of a function over a closed interval is to just graph it and look. -We began with the question of which rectangles of perimeter $20$ have the largest area? The figure shows a few different rectangles with this perimeter and their respective areas. - +We began with the question of which rectangles of perimeter $20$ have the largest area? @fig-fixed-perimeter-different-shapes-give-different-areas shows a few different rectangles with this perimeter and their respective areas. +::: {#fig-fixed-perimeter-different-shapes-give-different-areas} ```{julia} #| hold: true #| echo: false #| cache: true -### {{{perimeter_area_graphic}}} +#| fig-alt: Some possible rectangles that satisfy the constraint on the perimeter and their area. gr() function perimeter_area_graphic_graph(n) @@ -74,11 +77,7 @@ function perimeter_area_graphic_graph(n) plt end -caption = """ - -Some possible rectangles that satisfy the constraint on the perimeter and their area. - -""" +caption = "" n = 5 anim = @animate for i=1:n perimeter_area_graphic_graph(i-1) @@ -90,6 +89,9 @@ plotly() ImageFile(imgfile, caption) ``` +Some possible rectangles that satisfy the constraint on the perimeter and their area. +::: + The basic mathematical approach is to find a function of a single variable to maximize or minimize. In this case we have two variables describing a rectangle: a base $b$ and height $h$. Our formulas are the area of a rectangle: @@ -110,14 +112,17 @@ From this last one, we see that $b$ can be no bigger than $10$ and no smaller th Maximize $A(b) = b \cdot (10 - b)$ over the interval $[0,10]$. -This is exactly the form needed to apply our theorem about the existence of extrema (a continuous function on a closed interval). Rather than solve analytically by taking a derivative, we simply graph to find the value: - +This is exactly the form needed to apply our theorem about the existence of extrema (a continuous function on a closed interval). Rather than solve analytically by taking a derivative, in @fig-plot-Area-over-0-10 we simply graph to find the value. +::: {#fig-plot-Area-over-0-10} ```{julia} Area(b) = b * (10 - b) plot(Area, 0, 10) ``` +Plot of `Area` over possible values for `b`; $[0,10]$ +::: + You should see the maximum occurs at $b=5$ by symmetry, so $h=5$ as well, and the maximum area is then $25$. This gives the satisfying answer that among all rectangles of fixed perimeter, that with the largest area is a square. As well, this indicates a common result: there is often some underlying symmetry in the answer. @@ -177,9 +182,10 @@ Here is a similar, though more complicated, example where the analytic approach Let a "[Norman](https://en.wikipedia.org/wiki/Norman_architecture)" window consist of a rectangular window of top length $x$ and side length $y$ and a half circle on top. The goal is to maximize the area for a fixed value of the perimeter. Again, assume this perimeter is $20$ units. -This figure shows two such windows, one with base length given by $x=3$, the other with base length given by $x=4$. The one with base length $4$ seems to have much bigger area, what value of $x$ will lead to the largest area? +This figure shows two such windows, one with base length given by $x=3,$ the other with base length given by $x=4$. The one with base length $4$ seems to have much bigger area, what value of $x$ will lead to the largest area? +::: {#fig-norman-window-figure} ```{julia} #| hold: true #| echo: false @@ -198,6 +204,9 @@ plot!(p, x2/2 .+ x2/2*cos.(ts), y2 .+ x2/2*sin.(ts), linetype=:polygon, p ``` +Figure of two possible Norman windows each with the same, fixed perimeter +::: + For this problem, we have two equations. @@ -242,30 +251,37 @@ Of course both $x$ and $y$ are non-negative. The latter forces $x$ to be no more This leaves us the calculus problem of finding an absolute maximum of a continuous function over the closed interval $[0, 20/(1+\pi/2)]$. Our theorem tells us this maximum must occur, we now proceed to find it. -We begin by simply graphing and estimating the values of the maximum and where it occurs. - +We begin by simply graphing `A` in @fig-plot-area-over-0-20-over-1-plus-pi-over-2 and estimating the values of the maximum and where it occurs. +::: {#fig-plot-area-over-0-20-over-1-plus-pi-over-2} ```{julia} +#| echo: false plot(A, 0, 20/(1+pi/2)) ``` +Plot of area function `A` over the range of possible values for `x`, $[0, 20/(1 + \pi/2)]$ +::: + The naked eye sees that maximum value is somewhere around $27$ and occurs at $x\approx 5.6$. Clearly from the graph, we know the maximum value happens at the critical point and there is only one such critical point. -As reading the maximum from the graph is more difficult than reading a $0$ of a function, we plot the derivative using our approximate derivative. - +As reading the maximum from the graph is more difficult than reading a $0$ of a function, In @fig-plot-of-derivative-of-A-over-5-point-5-to-5-point-7 we plot the derivative using our approximate derivative, confirming that the critical point is around $5.6$. +::: {#fig-plot-of-derivative-of-A-over-5-point-5-to-5-point-7} ```{julia} +#| echo: false plot(A', 5.5, 5.7) ``` -We confirm that the critical point is around $5.6$. +Plot of `A'` over $[5.5, 5.7]$ +::: + #### Using `find_zero` to locate critical points. -Rather than zoom in graphically, we now use a root-finding algorithm, to find a more precise value of the zero of $A'$. We know that the maximum will occur at a critical point, a zero of the derivative. The `find_zero` function from the `Roots` package provides a non-linear root-finding algorithm based on the bisection method. The only thing to keep track of is that solving $f'(x) = 0$ means we use the derivative and not the original function. +Rather than zoom in graphically, we now use a root-finding algorithm, to find a more precise value of the zero of $A'$. $A$ is differentiable so the maximum will occur at a critical point that is a zero of the derivative. The only thing to keep track of is that solving $f'(x) = 0$ means we use the derivative and not the original function. In a few sections we will see how to use `find_zero` to find a zero *near* an initial guess, but for now we will use `find_zero` to find a zero *between* two values which form a bracketing interval. @@ -309,9 +325,9 @@ We could also do the above problem symbolically with the aid of `SymPy`. Here ar A₀ = w₀ * h₀ + pi * (w₀/2)^2 / 2 Perim = 2*h₀ + w₀ + pi * w₀/2 -h₁ = solve(Perim - 20, h₀)[1] +h₁ = first(solve(Perim - 20, h₀)) A₁ = A₀(h₀ => h₁) -w₁ = solve(diff(A₁,w₀) ~ 0, w₀)[1] +w₁ = first(solve(diff(A₁,w₀) ~ 0, w₀)) ``` We know that `w₀` is the maximum in this example from our previous work. We shall see soon, that just knowing that the second derivative is negative at `w₀` would suffice to know this. Here we check that condition: @@ -349,17 +365,38 @@ The figure shows a trapezoid inscribed in a circle. By adjusting the point ``P_3 nothing ``` +::: {#fig-trapezoid-inscribed-in-half-circle} ```{julia} #| hold: true #| echo: false -function trapezoid(r) - plot(x -> sqrt(1 - x^2), -1, 1, legend=false, aspect_ratio=:equal) - plot!([-1,1,r,-r,-1], [0,0,sqrt(1-r^2), sqrt(1-r^2), 0], lw=3, color=:red) +let + gr() + function trapezoid(r) + plot(; empty_style..., aspect_ratio=:equal) + plot!([(-1.1, 0), (1.1, 0)]; arrow=true, side=:right, line=(1, :gray)) + plot!([(0, -0.1), (0, 1.1)]; arrow=true, side=:right, line=(1, :gray)) + plot!(x -> sqrt(1 - x^2), -1, 1; line=(2, :black)) + L, R, X, Y = (-1,0), (1,0), (r, sqrt(1-r^2)), (-r, sqrt(1-r^2)) + plot!([L, R, X, Y, L], line=(3, :red)) + scatter!([X], marker=(3,)) + annotate!([ + (X..., text(L"(x,y)", :bottom, :left)), + (0, 0, text(L"(0,0)", :top, :left)), + (-1, 0, text(L"-r", :top)), + ( 1, 0, text(L"r", :top)) + ]) + end + plt = trapezoid(0.75) + plotly() + plt end -trapezoid(0.75) ``` -A trapezoid is *inscribed* in the upper-half circle of radius $r$. The trapezoid is found be connecting the points $(x,y)$ (in the first quadrant) with $(r, 0)$, $(-r,0)$, and $(-x, y)$. Find the maximum area. (The above figure has $x=0.75$ and $r=1$.) +A trapezoid inscribed in a upper-half circle with key point $(x,y)$ laying in quadrant $I$ +::: + + +A trapezoid is *inscribed* in the upper-half circle of radius $r$. The trapezoid is found be connecting the points $(x,y)$ (in the first quadrant) with $(r, 0)$, $(-r,0)$, and $(-x, y)$. Find the maximum area. (@fig-trapezoid-inscribed-in-half-circle shows $x=0.75$ and $r=1$.) Here the constraint is simply $r^2 = x^2 + y^2$ with $x$ and $y$ being non-negative. The area is then found through the average of the two lengths times the height. Using `height` for `y`, we have: @@ -367,17 +404,18 @@ Here the constraint is simply $r^2 = x^2 + y^2$ with $x$ and $y$ being non-negat ```{julia} @syms x::positive r::positive -hₜ = sqrt(r^2 - x^2) -aₜ = (2x + 2r)/2 * hₜ -possible_sols = solve(diff(aₜ, x) ~ 0, x) # possibly many solutions +height = sqrt(r^2 - x^2) +a = (2x + 2r)/2 * height +possible_sols = solve(diff(a, x) ~ 0, x) # possibly many solutions x0 = first(possible_sols) # only solution is also found from first or [1] indexing ``` The other values of interest can be found through substitution. For example: + ```{julia} -hₜ(x => x0) +height(x => x0) ``` ## Trigonometry problems @@ -389,9 +427,9 @@ Many maximization and minimization problems involve triangles, which in turn use A ladder is to be moved through a two-dimensional hallway which has a bend and gets narrower after the bend. The hallway is $8$ feet wide then $5$ feet wide. What is the longest such ladder that can be navigated around the corner? -The figure shows a ladder of length $l_1 + l_2$ that got stuck - it was too long. - +@fig-ladder-length-l1-l2 shows a ladder of length $l_1 + l_2$ that got stuck - it was too long. +::: {#fig-ladder-length-l1-l2} ```{julia} #| hold: true #| echo: false @@ -401,7 +439,7 @@ let xticks = [0,5, 15], yticks = [0,8, 12], line=(:blue, 2), - legend=false) + legend=false) plot!(p, [5, 5, 15], [15, 8, 8]; line=(:blue,2)) plot!(p, [0,14.53402874075368], [12.1954981558864, 0], linewidth=3) plot!(p, [0,5], [8,8], color=:orange) @@ -420,6 +458,10 @@ plotly() nothing ``` +Figure illustrating top view of a ladder of some length being moved around a corner in two dimensions. The ladder in the figure would be too big, as it is stuck in this position. +::: + + We approach this problem in reverse. It is easy to see when a ladder is too long. It gets stuck at some angle $\theta$. So for each $\theta$ we find that ladder length that is just too long. Then we find the minimum length of all these ladders that are too long. If a ladder is this length or more it will get stuck for some angle. However, if it is less than this length it will not get stuck. So to maximize a ladder length, we minimize a different function. Neat. @@ -445,13 +487,16 @@ Our goal is to minimize this function for all angles between $0$ and $90$ degree This is not a continuous function on a closed interval - it is undefined at the endpoints. That being said, a quick plot will convince us that the minimum occurs at a critical point and there is only one critical point in $(0, \pi/2)$. - +::: {#fig-plot-l-over-0-pi-over-2} ```{julia} -delta = 0.2 -plot(l, delta, pi/2 - delta) +#| echo: false +plot(rangeclamp(l, 40), 0, pi/2; legend=false) ``` -The graph shows the minimum occurs between $0.50$ and $1.00$ radians, a bracket for the derivative. Here we find $x$ and the minimum value: +Plot of `l`, the maximum length ladder stuck at angle $\theta$, over $(0, \pi/2)$ +::: + +@fig-plot-l-over-0-pi-over-2 shows the minimum occurs between $0.50$ and $1.00$ radians, a bracket for the derivative. Here we find $x$ and the minimum value: ```{julia} @@ -470,7 +515,7 @@ Ethan Hunt, a top secret spy, has a mission to chase a bad guy. Here is what we * Ethan likes to run. He can run at $10$ miles per hour. - * He can drive a car - usually some concept car by BMW - at $30$ miles per hour, but only on the road. + * He can drive a car---usually some concept car by BMW---at $30$ miles per hour, but only on the road. For his mission, he needs to go $10$ miles west and $5$ miles north. He can do this by: @@ -497,30 +542,28 @@ $$ With the endpoints given by $T(0) = \sqrt{10^2 + 5^2}/10$ and $T(10) = (10 + 5)/30$. - -Let's plot $T(x)$ over the interval $(0,10)$ and look: +@fig-ethan-hunts-time-over-0-10 shows a plot of $T(x)$ over the interval $(0,10)$. + +::: {#fig-ethan-hunts-time-over-0-10} ```{julia} +#| echo: false T(x) = x/30 + sqrt(5^2 + (10-x)^2)/10 -``` - -```{julia} plot(T, 0, 10) ``` -The minimum happens way out near 8. We zoom in a bit: +Plot of $T$ over $[0,10]$. +::: + +The lone minimum happens way out near 8. + + +We now use `find_zero` to refine our guess at the critical point using $[0, 10]$, as there is only one obvious solution: ```{julia} -plot(T, 7, 9) -``` - -It appears to be around $8.3$. We now use `find_zero` to refine our guess at the critical point using $[7,9]$: - - -```{julia} -α = find_zero(T', (7, 9)) +α = find_zero(T', (0, 10)) ``` Okay, got it. Around $8.23$. So is our minimum time. @@ -540,9 +583,9 @@ sqrt(10^2 + 5^2)/10, T(α), (10+5)/30 Ahh, we see that $T(x)$ is not continuous on $[0, 10]$, as it jumps at $x=10$ down to an even smaller amount of $1/2$. It may not look as impressive as a miles-long sprint, but Mr. Hunt is advised by Benji to drive the whole way. -### Rate times time ... the origin story - +### Rate times time---the origin story +::: {#fig-lhospital-image-43} ```{julia} #| hold: true #| echo: false @@ -558,7 +601,7 @@ region on the side of $C$, they cover distance $a$ in time $c$, and that on the other, on the side of $F$, distance $b$ in the same time $c$. We ask through which point $E$ on the line $AEB$ they should pass, so as to take the least possible time to get from $C$ to $F$? (From -http://www.ams.org/samplings/feature-column/fc-2016-05.) +[AMS.org](http://www.ams.org/samplings/feature-column/fc-2016-05).) """ @@ -566,15 +609,17 @@ http://www.ams.org/samplings/feature-column/fc-2016-05.) nothing ``` -![Image number $43$ from l'Hospital's calculus book (the first). A +![](./figures/fcarc-may2016-fig43-250.png){fig-alt="Image number 43 from l'Hospital's calculus book"} + +Image number $43$ from l'Hospital's calculus book (the first). A traveler leaving location $C$ to go to location $F$ must cross two regions separated by the straight line $AEB$. We suppose that in the region on the side of $C$, he covers distance $a$ in time $c$, and that on the other, on the side of $F$, distance $b$ in the same time $c$. We ask through which point $E$ on the line $AEB$ he should pass, so as to take the least possible time to get from $C$ to $F$? (From -http://www.ams.org/samplings/feature-column/fc-2016-05.)](./figures/fcarc-may2016-fig43-250.png) - +[AMS.org](http://www.ams.org/samplings/feature-column/fc-2016-05).) +::: The last example is a modern day illustration of a problem of calculus dating back to l'Hospital. His parameterization is a bit different. Let's change his by taking two points $(0, a)$ and $(L,-b)$, with $a,b,L$ positive values. Above the $x$ axis travel happens at rate $r_0$, and below, travel happens at rate $r_1$, again, both positive. What value $x$ in $[0,L]$ will minimize the total travel time? @@ -598,7 +643,7 @@ The answer will occur at a critical point or an endpoint, either $x=0$ or $x=L$. The structure of `dt` is too complicated for simply calling `solve` to find the critical points. Instead we help `SymPy` out a bit. We are solving an equation of the form $a/b + c/d = 0$. These solutions will also be solutions of $(a/b)^2 - (c/d)^2=0$ or even $a^2d^2 - c^2b^2 = 0$. This follows as solutions to $u+v=0$, also solve $(u+v)\cdot(u-v)=0$, or $u^2 - v^2=0$. Setting $u=a/b$ and $v=c/d$ completes the comparison. -We can get these terms - $a$, $b$, $c$, and $d$ - as follows: +We can get these terms---$a$, $b$, $c$, and $d$---as follows: ```{julia} @@ -620,7 +665,7 @@ p = sympy.Poly(ex, x) # a0 + a1⋅x + a2⋅x^2 + a3⋅x^3 + a4⋅x^4 p.coeffs() ``` -Fourth degree polynomials can be solved. The critical points of the original equation will be among the $4$ solutions given. However, the result is complicated. The [article](http://www.ams.org/samplings/feature-column/fc-2016-05) – from which the figure came – states that "In today's textbooks the problem, usually involving a river, involves walking along one bank and then swimming across; this corresponds to setting $g=0$ in l'Hospital's example, and leads to a quadratic equation." Let's see that case, which we can get in our notation by taking $b=0$: +Fourth degree polynomials can be solved. The critical points of the original equation will be among the $4$ solutions given. However, the result is complicated. The [article](http://www.ams.org/samplings/feature-column/fc-2016-05)---from which the figure came---states that "In today's textbooks the problem, usually involving a river, involves walking along one bank and then swimming across; this corresponds to setting $g=0$ in l'Hospital's example, and leads to a quadratic equation." Let's see that case, which we can get in our notation by taking $b=0$: ```{julia} @@ -707,22 +752,30 @@ Here the extreme value theorem doesn't technically apply, as we don't have a clo In general, for an optimization problem of a continuous function on the interval $(a,b)$ if the right limit at $a$ and left limit at $b$ can be ruled out as candidates, the optimal value must occur at a critical point.) -So to approach this problem we first graph it over a wide interval. - +So to approach this problem we first graph it over a wide interval in @fig-plot-of-x-exp-minus-x-squared-over-wide-interval. +::: {#fig-plot-of-x-exp-minus-x-squared-over-wide-interval} ```{julia} +#| echo: false f(x) = x * exp(-x^2) plot(f, 0, 100) ``` +Plot of $f(x) = x \cdot e^{-x^2}$ over $[0, 100]$, a wide interval +::: + Clearly the action is nearer to $1$ than $100$. We try graphing the derivative near that area: - +::: {#fig-plot-of-x-exp-minus-x-squared-over-interval-of-interest} ```{julia} +#| echo: false plot(f', 0, 5) ``` +Plot of the *derivative$ of $f(x) = x \cdot e^{-x^2}$ over $[0, 5]$, an interval where the values of interest lie +::: -This shows the value of interest near $0.7$ for a critical point. We use `find_zero` with $[0,1]$ as a bracket + +@fig-plot-of-x-exp-minus-x-squared-over-interval-of-interest shows the value of interest near $0.7$ for a critical point. We use `find_zero` with $[0,1]$ as a bracket ```{julia} @@ -780,13 +833,17 @@ SA(r) = SA(canheight(r), r) This is minimized subject to the constraint that $r \geq 0$. A quick glance shows that as $r$ gets close to $0$, the can must get infinitely tall to contain that fixed volume, and would have infinite surface area as the $1/r^2$ in the first term implies. On the other hand, as $r$ goes to infinity, the height must go to $0$ to make a really flat can. Again, we would have infinite surface area, as the $r^2$ term at the end indicates. With this observation, we can rule out the endpoints as possible minima, so any minima must occur at a critical point. -We start by making a graph, making an educated guess that the answer is somewhere near a real life answer, or around $3$-$5$ cms in radius: - +We start by making the graph in @fig-plot-of-surface-area-of-can-over-2-10 using an educated guess that the answer is somewhere near a real life answer, or around $3$-$5$ cms in radius. +::: {#fig-plot-of-surface-area-of-can-over-2-10} ```{julia} +#| echo: false plot(SA, 2, 10) ``` +Plot of surface area of a can over $[2, 10]$ +::: + The minimum looks to be around $4$cm and is clearly between $2$cm and $6$cm. We can use `find_zero` to zero in on the value of the critical point: @@ -848,19 +905,27 @@ numericq(val) A rancher with $10$ meters of fence wishes to make a pen adjacent to an existing fence. The pen will be a rectangle with one edge using the existing fence. Say that has length $x$, then $10 = 2y + x$, with $y$ the other dimension of the pen. What is the maximum area that can be made? - +::: {#fig-plot-of-stupid-rancher-figure-for-fencing} ```{julia} #| hold: true #| echo: false - p = plot(; legend=false, aspect_ratio=:equal, axis=nothing, border=:none) +let + gr() + p = plot(; legend=false, aspect_ratio=:equal, axis=nothing, border=:none) -plot!([0,10, 10, 0, 0], [0,0,10,10,0]; linewidth=3) -plot!(p, [10,14,14,10], [2, 2, 8,8]; linewidth = 1) + plot!([0,10, 10, 0, 0], [0,0,10,10,0]; linewidth=3) + plot!(p, [10,14,14,10], [2, 2, 8,8]; linewidth = 1) -annotate!(p, [(14-0.1, 5, text("x", :right)), (12,2, text("y",:bottom))]) -p + annotate!(p, [(14-0.1, 5, text(L"x", :right)), (12,2, text(L"y",:bottom))]) + plotly() + p +end ``` +Plot of rancher's adjacent pen +::: + + ```{julia} #| hold: true #| echo: false @@ -917,8 +982,8 @@ choices = [ "It is also 20", "``17.888``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1054,22 +1119,31 @@ val = y(x0) numericq(val) # 0 ``` -###### Question (Thanks https://www.math.ucdavis.edu/~kouba) +###### Question -A movie screen projects on a wall 20 feet high beginning 10 feet above the floor. This figure shows $\theta$ for $x=30$: - +A movie screen projects on a wall 20 feet high beginning 10 feet above the floor. @fig-plot-movie-screen-projector shows $\theta$ for $x=30$: +::: {#fig-plot-movie-screen-projector} ```{julia} #| hold: true #| echo: false -p = plot([0, 30,30], [0,0,10], xlim=(0, 32), color=:blue, legend=false) -plot!(p, [30, 30], [10, 30], color=:blue, linewidth=4) -plot!(p, [0, 30,30,0], [0,10,30,0], color=:orange) -annotate!(p, [(x,y,l) for (x,y,l) in zip([15, 5, 31, 31], [1.5, 3.5, 5, 20], ["x=30", "θ", "10", "20"])]) +let + gr() + p = plot([0, 30,30], [0,0,10], xlim=(0, 32), color=:blue, legend=false) + plot!(p, [30, 30], [10, 30], color=:blue, linewidth=4) + plot!(p, [0, 30,30,0], [0,10,30,0], color=:orange) + annotate!(p, [(x,y,l) for (x,y,l) in zip([15, 5, 31, 31], [1.5, 3.5, 5, 20], [L"x=30", L"\theta", L"10", L"20"])]) + + plotly() + p +end ``` -What value of the largest angle $\theta$ that $x$ gives? (In degrees.) +Plot of a $20$-foot movie screen from $30$ feet away and the angle subtended +::: + +What value of the largest angle $\theta$ in degrees that $x$ gives? (Thanks https://www.math.ucdavis.edu/~kouba).) ```{julia} @@ -1107,8 +1181,8 @@ Now if $Likhood(t) = \exp(-3t) \cdot \exp(-2t) \cdot \exp(-4t), \quad 0 \leq t \ choices=["It does work and the answer is x = 2.27...", L" $Likhood(t)$ is not continuous on $0$ to $10$", L" $Likhood(t)$ takes its maximum at a boundary point - not a critical point"]; -answ = 3; -radioq(choices, answ) +answer = 3; +radioq(choices, answer) ``` ##### Question @@ -1139,8 +1213,8 @@ choices=[ "The median, or middle number, of the values", L"The square roots of the values squared, $(x_1^2 + \cdots + x_n^2)^2$" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1231,23 +1305,31 @@ numericq(hstar) ###### Question -Find the value $x > 0$ which minimizes the distance from the graph of $f(x) = \log_e(x) - x$ to the origin $(0,0)$. - +Find the value $x > 0$ which minimizes the distance from the graph of $f(x) = \ln(x) - x$ to the origin $(0,0)$. +::: {#fig-minimize-distance-from-logx-minus-x} ```{julia} #| hold: true #| echo: false -f(x) = log(x) - x -p = plot(f, 0.2, 2, ylim=(-2,0.25), legend=false, linewidth=3) -plot!(p, [0,0], [-2, 0.25], color=:blue) -plot!(p, [0,2],[0,0], color=:blue) -xs = [0,1]; ys = [0, f(1)] -scatter!(p, xs,ys, color=:orange) -plot!(p, xs, ys, color=:orange, linewidth=3) -annotate!(p, [(.75, f(.5)/2, "d = $(round(sqrt(.5^2 + f(.5)^2), digits=2))")]) -p +let + gr() + f(x) = log(x) - x + p = plot(f, 0.2, 2, ylim=(-2,0.25), legend=false, linewidth=3) + plot!(p, [0,0], [-2, 0.25], color=:blue) + plot!(p, [0,2],[0,0], color=:blue) + xs = [0,1/2]; ys = [0, f(1/2)] + O, P = (0,0), (1/2, f(1/2)) + scatter!(p, [O, P], color=:orange) + plot!(p, [O, P], color=:orange, linewidth=3) + annotate!(p, [(.28, f(0.5)/2, text(L"d = 1.29\cdots", :left))]) + plotly() + p +end ``` +Plot of $f(x) = \ln(x) - x$ and the distance, $d$, of the point $(1/2, f(1/2)$ to the origin $(0,0)$ +::: + ```{julia} #| hold: true #| echo: false @@ -1274,10 +1356,14 @@ Image number $40$ from l'Hospital's calculus book (the first calculus book). Amo nothing ``` -![Image number $40$ from l'Hospital's calculus book (the first calculus book). Among all the cones that can be inscribed in a sphere, determine which one has the largest lateral area. (From [AMS](http://www.ams.org/samplings/feature-column/fc-2016-05)).](./figures/fcarc-may2016-fig40-300.png) +::: {#fig-lhospital-image-40-optimization} +![](./figures/fcarc-may2016-fig40-300.png){fig-alt="L'Hospital image 40"} + +Image number $40$ from l'Hospital's calculus book (the first calculus book). Among all the cones that can be inscribed in a sphere, determine which one has the largest lateral area. (From [AMS](http://www.ams.org/samplings/feature-column/fc-2016-05)). +::: -The figure above poses a problem about cones in spheres, which can be reduced to a two-dimensional problem. Take a sphere of radius $r=1$, and imagine a secant line of length $l$ connecting $(-r, 0)$ to another point $(x,y)$ with $y>0$. Rotating that line around the $x$ axis produces a cone and its lateral surface is given by $SA=\pi \cdot y \cdot l$. Write $SA$ as a function of $x$ and solve. +@fig-lhospital-image-40-optimization poses a problem about cones in spheres, which can be reduced to a two-dimensional problem. Take a sphere of radius $r=1$, and imagine a secant line of length $l$ connecting $(-r, 0)$ to another point $(x,y)$ with $y>0$. Rotating that line around the $x$ axis produces a cone and its lateral surface is given by $SA=\pi \cdot y \cdot l$. Write $SA$ as a function of $x$ and solve. The largest lateral surface area is: @@ -1306,8 +1392,8 @@ choices = ["exactly four times", L"exactly $\pi$ times", L"about $2.6$ times as big", "about the same"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` ###### Question @@ -1354,8 +1440,8 @@ choices = [ "``e/a``", "``a \\cdot e``" ] -answ=2 -radioq(choices, answ) +answer=2 +radioq(choices, answer) ``` ###### Question @@ -1363,26 +1449,36 @@ radioq(choices, answ) The ladder problem has a trigonometry-free solution. We show one attributed to [Asma](http://www.mathematische-basteleien.de/ladder.htm). - +::: {#fig-ladder-problem-look-ma-no-trigonometry} ```{julia} #| hold: true #| echo: false -plt = plot(; axis=nothing, border=:none, legend=false, aspect_ratio=:equal) -a,b = 1, 2 -p = 1/2 -x = a/p -plot!(plt, [0, b*(1+p), 0, 0], [0, 0, a+x, 0]) -plot!(plt, [b,b,0,0],[0,a,a,0]) -annotate!(plt, [ - (b/2,0, text("b",:top)), - (0,a/2, text("a",:right)), - (0,a+x/2, text("x",:right)), - (b+b*p/2,0, text("bp",:top)) -]) -plt +let + gr() + plt = plot(; axis=nothing, border=:none, legend=false, aspect_ratio=:equal) + a,b = 1, 2 + p = 1/2 + x = a/p + O, P, Q = (0,0), (b*(1+p), 0), (0, a+x) + plot!(plt, [Q,O,P]; line=(1, :blue)) + plot!(plt, [Q,P]; line=(3, :green)) + plot!(plt, [b,b,0,0],[0,a,a,0]) + annotate!(plt, [ + (b/2,0, text(L"b",:top)), + (0,a/2, text(L"a",:right)), + (0,a+x/2, text(L"x",:right)), + (b+b*p/2,0, text(L"b\cdot p",:top)), + ((b + b*p)/2, (a+x)/2, text(L"c", :bottom, :left)) + ]) + plotly() + plt +end ``` -Introducing a variable $p$, we get, following the above figure, the ladder of length $c$ touching the wall at $b+bp$ and $a + x$. +Figure to solve a ladder problem with Pythagorean's theorem +::: + +Introducing a variable $p$, we get, following @fig-ladder-problem-look-ma-no-trigonometry, the ladder of length $c$ touching the wall at $b+bp$ and $a + x$. Using similar triangles, we have: diff --git a/quarto/derivatives/related_rates.qmd b/quarto/derivatives/related_rates.qmd index 8997205..296bc92 100644 --- a/quarto/derivatives/related_rates.qmd +++ b/quarto/derivatives/related_rates.qmd @@ -18,7 +18,7 @@ using SymPy --- -Related rates problems involve two (or more) unknown quantities that are related through an equation. As the two variables depend on each other, also so do their rates - change with respect to some variable which is often time. Exactly how remains to be discovered. Hence the name "related rates." +Related rates problems involve two (or more) unknown quantities that are related through an equation. As the two variables depend on each other, also so do their rates---change with respect to some variable which is often time. Exactly how remains to be discovered. Hence the name "related rates". #### Examples @@ -30,50 +30,51 @@ The following is a typical "book" problem: > A *vintage* screen saver displays the outline of a $3$ cm by $2$ cm rectangle and then expands the rectangle in such a way that the $2$ cm side is expanding at the rate of $4$ cm/sec and the proportions of the rectangle never change. How fast is the area of the rectangle increasing when its dimensions are $12$ cm by $8$ cm? [Source.](http://oregonstate.edu/instruct/mth251/cq/Stage9/Practice/ratesProblems.html) - +::: {#fig-growing-rects-graph} ```{julia} #| hold: true #| echo: false #| cache: true ### {{{growing_rects}}} ## Secant line approaches tangent line... -gr() -function growing_rects_graph(n) - w = (t) -> 2 + 4t - h = (t) -> 3/2 * w(t) - t = n - 1 +let + gr() + function growing_rects_graph(n) + w = (t) -> 2 + 4t + h = (t) -> 3/2 * w(t) + t = n - 1 - w_2 = w(t)/2 - h_2 = h(t)/2 + w_2 = w(t)/2 + h_2 = h(t)/2 - w_n = w(5)/2 - h_n = h(5)/2 + w_n = w(5)/2 + h_n = h(5)/2 - plt = plot(w_2 * [-1, -1, 1, 1, -1], h_2 * [-1, 1, 1, -1, -1], xlim=(-17,17), ylim=(-17,17), - legend=false, size=fig_size) - annotate!(plt, [(-1.5, 1, "Area = $(round(Int, 4*w_2*h_2))")]) - plt + plt = plot(w_2 * [-1, -1, 1, 1, -1], h_2 * [-1, 1, 1, -1, -1], xlim=(-17,17), ylim=(-17,17), + legend=false, size=fig_size) + area = round(Int, 4*w_2*h_2) + annotate!(plt, [(-1.5, 1, text(latexstring(L"Area = \$$area\$", :center)))]) + plt + end + caption = "" + n=6 + anim = @animate for i=1:n + growing_rects_graph(i) + end + imgfile = tempname() * ".gif" + gif(anim, imgfile, fps = 1) + + plotly() + ImageFile(imgfile, caption) end -caption = L""" - -As $t$ increases, the size of the rectangle grows. The ratio of width to height is fixed. If we know the rate of change in time for the width ($dw/dt$) and the height ($dh/dt$) can we tell the rate of change of *area* with respect to time ($dA/dt$)? - -""" -n=6 - -anim = @animate for i=1:n - growing_rects_graph(i) -end - -imgfile = tempname() * ".gif" -gif(anim, imgfile, fps = 1) -plotly() -ImageFile(imgfile, caption) ``` -Here we know $A = w \cdot h$ and we know some things about how $w$ and $h$ are related *and* about the rate of how both $w$ and $h$ grow in time $t$. That means that we could express this growth in terms of some functions $w(t)$ and $h(t)$, then we can figure out that the area - as a function of $t$ - will be expressed as: +As $t$ increases, the size of the rectangle grows. The ratio of width to height is fixed. If we know the rate of change in time for the width ($dw/dt$) and the height ($dh/dt$) can we tell the rate of change of *area* with respect to time ($dA/dt$)? +::: + +Here we know $A = w \cdot h$ and we know some things about how $w$ and $h$ are related *and* about the rate of how both $w$ and $h$ grow in time $t$. That means that we could express this growth in terms of some functions $w(t)$ and $h(t)$, then we can figure out that the area---as a function of $t$---will be expressed as: $$ @@ -94,7 +95,7 @@ $$ \frac{dA}{dt} = \frac{dw}{dt} h + w \frac{dh}{dt}. $$ -This relationship is true for all $t$, but the problem discusses a certain value of $t$ - when $w(t)=8$ and $h(t) = 12$. At this same value of $t$, we have $w'(t) = 4$ and so $h'(t) = 6$. Substituting these 4 values into the 4 unknowns in the formula for $A'(t)$ gives: +This relationship is true for all $t$, but the problem discusses a certain value of $t$---when $w(t)=8$ and $h(t) = 12$. At this same value of $t$, we have $w'(t) = 4$ and so $h'(t) = 6$. Substituting these 4 values into the 4 unknowns in the formula for $A'(t)$ gives: $$ @@ -198,6 +199,7 @@ A ladder, with length $l$, is leaning against a wall. We parameterize this probl If the ladder starts to slip away at the base, but remains in contact with the wall, express the rate of change of $h$ with respect to $t$ in terms of $db/dt$. +::: {#fig-ladder-along-wall-slipping-away} ```{julia} #| echo: false let @@ -231,16 +233,13 @@ let (b/2, h/2, text(L"L", rotation = -atand(h,b), :bottom)) ]) + plotly() current() end ``` -```{julia} -#| echo: false -plotly() -nothing -``` - +A ladder placed on a wall is slipping away from its base +::: We have from implicitly differentiating in $t$ the equation $l^2 = h^2 + b^2$, noting that $l$ is a constant, that: @@ -298,9 +297,13 @@ imgfile = "figures/long-shadow-noir.png" ImageFile(:derivatives, imgfile, caption) ``` -![A man and woman walk towards the light](./figures/long-shadow-noir.png) +::: {#fig-film-noir} +![](./figures/long-shadow-noir.png){fig-alt="A man and woman walk towards the light"} -Shadows are a staple of film noir. In the photo, suppose a man and a woman walk towards a street light. As they approach the light the length of their shadow changes. +A man and woman walk towards the light +::: + +Shadows are a staple of film noir. In @fig-film-noir, suppose a man and a woman walk towards a street light. As they approach the light the length of their shadow changes. Suppose, we focus on the $5$ foot tall woman. Her shadow comes from a streetlight $12$ feet high. She is walking at $3$ feet per second towards the light. What is the rate of change of her shadow? @@ -340,6 +343,7 @@ Solving for $l'$ gives an answer in terms of $x'$ the rate the woman is walking. ##### Example +::: {#fig-sun-setting-casts-a-shadow} ```{julia} #| hold: true #| echo: false @@ -351,6 +355,10 @@ plot!(p, [25,25],[25,0], linewidth=5, color=:black) plot!(p, [25,50], [0,0], linewidth=2, color=:black) ``` +As the sun sets shadows change length +::: + + The sun is setting at the rate of $1/20$ radian/min, and appears to be dropping perpendicular to the horizon, as depicted in the figure. How fast is the shadow of a $25$ meter wall lengthening at the moment when the shadow is $25$ meters long? @@ -361,7 +369,7 @@ $$ \tan(\theta) = \frac{25}{x} $$ -of $x\tan(\theta) = 25$. +or $x\tan(\theta) = 25$. As $t$ evolves, we know $d\theta/dt$ but what is $dx/dt$? Using implicit differentiation yields: @@ -442,6 +450,7 @@ $$ ##### Example +::: {#fig-baseball-been-berry-good-graph} ```{julia} #| hold: true @@ -469,15 +478,13 @@ function baseball_been_berry_good_graph(n) plt = plot(xs, ys, legend=false, size=fig_size, xlim=(0,150), ylim=(0,15)) plot!(plt, [x(t), 100], [y(t), 0.0], color=:orange) - annotate!(plt, [(55, 4,"θ = $(round(Int, degrees)) degrees"), - (x(t), y(t), "($(round(Int, x(t))), $(round(Int, y(t))))")]) + degs = round(Int, degrees) + xts, yts = round(Int, x(t)), round(Int, y(t)) + annotate!(plt, [(55, 4, text(latexstring("\$\\theta = $degs\$ degrees"))), + (x(t), y(t), text(latexstring("($xts, $yts)")))]) end -caption = L""" - -The flight of the ball as being tracked by a stationary outfielder. This ball will go over the head of the player. What can the player tell from the quantity $d\theta/dt$? - -""" +caption = "" n = 8 @@ -492,6 +499,9 @@ plotly() ImageFile(imgfile, caption) ``` +The flight of the ball as being tracked by a stationary outfielder. This ball will go over the head of the player. What can the player tell from the quantity $d\theta/dt$? +::: + A baseball player stands $100$ meters from home base. A batter hits the ball directly at the player so that the distance from home plate is $x(t)$ and the height is $y(t)$. @@ -559,16 +569,20 @@ dtheta₁ = dtheta(cos(theta(t))^2 => (100 -x(t))^2/(y(t)^2 + (100-x(t))^2)) Plotting reveals some interesting things. For $v_{0y} < 10$ we have graphs that look like: +::: {#fig-dtheta-small-v} ```{julia} plot(dtheta₁, 0, v0/5) ``` +Plot of `dtheta` for smaller velocity +::: + The ball will drop in front of the player, and the change in $d\theta/dt$ is monotonic. But let's rerun the code with $v_{0y} > 10$: - +::: {#fig-dtheta-larger-v} ```{julia} #| hold: true v0 = 15 @@ -581,6 +595,10 @@ dtheta₁ = subs(dtheta, cos(theta(t))^2, (100 - x(t))^2/(y(t)^2 + (100 - x(t))^ plot(dtheta₁, 0, v0/5) ``` +Plot of `dtheta` for larger velocity +::: + + In the second case we have a different shape. The graph is not monotonic, and before the peak there is an inflection point. Without thinking too hard, we can see that the greatest change in the angle is when it is just above the head ($t=2$ has $x(t)=100$). @@ -610,7 +628,7 @@ $$ \frac{dV}{dh} = \frac{\pi}{3} ( R(h)^2 + h \cdot 2 R \frac{dR}{dh}). $$ -We see that it depends on $R$ and the change in $R$ with respect to $h$. However, we visualize $h$ - the height - so it is better to re-express. Clearly, $dR/dh = \tan\theta$ and using $R(h) = h \tan(\theta)$ we get: +We see that it depends on $R$ and the change in $R$ with respect to $h$. However, we visualize $h$---the height---so it is better to re-express. Clearly, $dR/dh = \tan\theta$ and using $R(h) = h \tan(\theta)$ we get: $$ @@ -626,7 +644,7 @@ How do the quantities vary in time? For an incompressible fluid, by balancing the volume leaving with how it leaves we will have $dh/dt$ is the ratio of the cross-sectional area at bottom over that at the height of the fluid $(\pi \cdot (h_0\tan(\theta))^2) / (\pi \cdot ((h\tan\theta))^2)$ times the outward velocity of the fluid. -That is $dh/dt = (h_0/h)^2 \cdot v$. Which makes sense - larger openings ($h_0$) mean more fluid lost per unit time so the height change follows, higher levels ($h$) means the change in height is slower, as the cross-sections have more volume. +That is $dh/dt = (h_0/h)^2 \cdot v$. Which makes sense---larger openings ($h_0$) mean more fluid lost per unit time so the height change follows, higher levels ($h$) means the change in height is slower, as the cross-sections have more volume. By [Torricelli's](http://en.wikipedia.org/wiki/Torricelli's_law) law, the out velocity follows the law $v = \sqrt{2g(h-h_0)}$. This gives: @@ -672,8 +690,8 @@ choices = [ "The rate of change of price will increase", "The rate of change of price will be positive and will depend on the rate of change of excess demand." ] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` (Theoretically, when demand exceeds supply, prices increase.) @@ -692,8 +710,8 @@ choices = [ "If the rate of change of unemployment is negative, the rate of change of wages will be negative.", "If the rate of change of unemployment is negative, the rate of change of wages will be positive." ] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` (Colloquially, "the rate of change of unemployment is negative" means the unemployment rate is going down, so there are fewer workers available to fill new jobs.) @@ -713,8 +731,8 @@ L"The rate of change of pressure is always increasing by $c$", "If volume is constant, the rate of change of pressure is proportional to the temperature", "If volume is constant, the rate of change of pressure is proportional to the rate of change of temperature", "If pressure is held constant, the rate of change of pressure is proportional to the rate of change of temperature"] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -800,8 +818,8 @@ choices = [ "``f(x) = x``", "``f(x) = x^2``" ] -answ = 4 -radioq(choices, answ, keep_order=true) +answer = 4 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -833,8 +851,8 @@ choices = [ "``y = 1 - \\log(x)``", "``y = x(2x - 1/x)``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` If $dx/dt = -1$, what is $dy/dt$? @@ -849,6 +867,6 @@ choices = [ "``dy/dt = -2x - 1/x``", "``dy/dt = 1``" ] -answ=1 -radioq(choices, answ) +answer=1 +radioq(choices, answer) ``` diff --git a/quarto/derivatives/symbolic_derivatives.qmd b/quarto/derivatives/symbolic_derivatives.qmd index 3388032..c0659a0 100644 --- a/quarto/derivatives/symbolic_derivatives.qmd +++ b/quarto/derivatives/symbolic_derivatives.qmd @@ -15,9 +15,11 @@ using TermInterface The ability to breakdown an expression into operations and their arguments is necessary when trying to apply the differentiation rules. Such rules are applied from the outside in. Identifying the proper "outside" function is usually most of the battle when finding derivatives. +This section takes a detour into this point. **This section can be skipped; the material is not used in the sequel**. -In the following example, we provide a sketch of a framework to differentiate expressions by a chosen symbol to illustrate how the outer function drives the task of differentiation. +--- +In the following example, we provide a sketch of a framework to differentiate expressions by a chosen symbol to illustrate how the outer function drives the task of differentiation; how the steps of differentiation can be automated; and how `Julia`'s multiple dispatch can be leveraged to handle the many different cases to be considered. The `Symbolics` package provides native symbolic manipulation abilities for `Julia`, similar to `SymPy`, though without the dependence on `Python`. The `TermInterface` package, used by `Symbolics`, provides a generic interface for expression manipulation for this package that *also* is implemented for `Julia`'s expressions and symbols. @@ -66,9 +68,9 @@ function D(ex, var=:x) end ``` -(The use of `Val` is an idiom of `Julia` allowing dispatch on certain values such as function names and numbers.) +The use of `Val` is an idiom of `Julia` allowing dispatch on certain values such as function names and numbers. -Now to develop methods for `D` for different "outside" functions and arities. +Now to develop methods for `D` for different "outside" functions and arities. We being by implementing sum, product, and quotient rules. Addition can be unary (`:(+x)` is a valid quoting, even if it might simplify to the symbol `:x` when evaluated), *binary*, or *nary*. Here we implement the *sum rule*: @@ -88,10 +90,10 @@ function D(::Val{:+}, ::Any, args, var) end ``` -The `args` are always held in a container, so the unary method must pull out the first one. The binary case should read as: apply `D` to each of the two arguments, and then create a quoted expression containing the sum of the results. The dollar signs interpolate into the quoting. (The "primes" are unicode notation achieved through `\prime[tab]` and not operations.) The *nary* method (which catches *any* arity besides `1` and `2`) does something similar, only using splatting to produce the sum. +The `args` are always held in a container, so the unary method must pull out the first one. The binary case should read as: apply `D` to each of the two arguments, and then create a new expression containing the sum of the results. The dollar signs interpolate into the quoting. (The "primes" are unicode notation achieved through `\prime[tab]` and not operations.) The *nary* method (which catches *any* arity besides `1` and `2`) does something similar, only using splatting to produce the sum. -Subtraction must also be implemented in a similar manner, but not for the *nary* case, as subtraction is not associative: +Subtraction is implemented in a similar manner, but there is no *nary* case; subtraction is not associative: ```{julia} @@ -99,6 +101,7 @@ function D(::Val{:-}, ::Val{1}, args, var) a′ = D(first(args), var) :(-$a′) end + function D(::Val{:-}, ::Val{2}, args, var) a′, b′ = D.(args, var) :($a′ - $b′) @@ -126,17 +129,17 @@ function D(op::Val{:*}, ::Any, args, var) end ``` -The *nary* case above just peels off the first factor and then uses the binary product rule. +The *nary* case above just peels off the first factor and then recursively uses the product rule to find `b′`. -Division is only a binary operation, so here we have the *quotient rule*: +Division is only a binary operation: ```{julia} function D(::Val{:/}, ::Val{2}, args, var) u,v = args u′, v′ = D(u, var), D(v, var) - :( ($u′*$v - $u*$v′)/$v^2 ) + :(($u′*$v - $u*$v′)/$v^2) end ``` diff --git a/quarto/derivatives/taylor_series_polynomials.qmd b/quarto/derivatives/taylor_series_polynomials.qmd index 10f51de..1690491 100644 --- a/quarto/derivatives/taylor_series_polynomials.qmd +++ b/quarto/derivatives/taylor_series_polynomials.qmd @@ -30,9 +30,16 @@ Quadratic functions are still fairly easy to work with. Is it possible to find t More generally, for a given $n$, what would be the best polynomial of degree $n$ to approximate $f(x)$ at $c$? -We will see in this section how the Taylor polynomial answers these questions, and is the appropriate generalization of the tangent line approximation. +We will see in this section how the Taylor polynomial is the appropriate generalization of the tangent line approximation and is given by: +$$ +\begin{align*} +T_n(x) &= f(c) + f'(c)(x-c) + \frac{f''(c)}{2!}(x-c)^2 + \\ +&\qquad\frac{f'''(c)}{3!}(x-c)^3 + \cdots + \frac{f^{(n)}(c)}{n!}(x-c)^n. +\end{align*} +$$ +::: {#fig-taylor-polynomial-animation} ```{julia} #| hold: true #| echo: false @@ -66,19 +73,35 @@ imgfile = tempname() * ".gif" gif(anim, imgfile, fps = 1) -caption = L""" - -Illustration of the Taylor polynomial of degree $k$, $T_k(x)$, at $c=0$ and its graph overlaid on that of the function $1 - \cos(x)$. - -""" +caption = "" plotly() ImageFile(imgfile, caption) ``` +Illustration of the Taylor polynomial of degree $k$, $T_k(x)$, at $c=0$ and its graph overlaid on that of the function $1 - \cos(x)$. +::: + ## The secant line and the tangent line -Heads up: we approach this general problem **much** more indirectly than is needed by introducing notations that are attributed to Newton and proceed from there. By leveraging `SymPy` we avoid tedious computations and *hopefully* gain some insight. +The form of the Taylor polynomial can readily be seen, as it is the polynomial of degree $n$ or less that has $n$ derivatives matching those of $f$ at $x=c$. This `SymPy` code shows equivalence for $n=5$: + +```{julia} +@syms a[0:5] x c +T₅ = sum(a[i+1] * (x-c)^i for i in 0:5) +``` + +That is a polynomial centered at $x=c$. + +These are the derivatives that would match up with $f^{(k)}(c)$ which shows $f^{(k)}(c) = k! \cdot a_k$ (or $a_k = f^{(k)}(c)/k!$) for $k$ in $0, 1, \dots, 5$: + +```{julia} +[diff(T₅, x, k)(x => c) for k in 0:5] +``` + +--- + +However, in the following, we approach this general problem **much** more indirectly than is needed by introducing notations that are attributed to Newton and proceed from there. Again, by leveraging `SymPy` we avoid tedious computations and *hopefully* gain some insight. Suppose $f(x)$ is a function which is defined in a neighborhood of $c$ and has as many continuous derivatives as we care to take at $c$. @@ -87,7 +110,7 @@ Suppose $f(x)$ is a function which is defined in a neighborhood of $c$ and has a We have two related formulas: - * The *secant line* connecting $(c, f(c))$ and $(c+h, f(c+h))$ for a value of $h>0$ is given in point-slope form by +* The *secant line* connecting $(c, f(c))$ and $(c+h, f(c+h))$ for a value of $h>0$ is given in point-slope form by $$ @@ -97,14 +120,14 @@ $$ The slope is the familiar approximation to the derivative: $(f(c+h)-f(c))/h$. - * The *tangent line* to the graph of $f(x)$ at $x=c$ is described by the function +* The *tangent line* to the graph of $f(x)$ at $x=c$ is described by the function $$ tl(x) = f(c) + f'(c) \cdot(x - c). $$ -The key is the term multiplying $(x-c)$---for the secant line this is an approximation to the related term for the tangent line. That is, the secant line approximates the tangent line, which is the linear function that best approximates the function at the point $(c, f(c))$. +The key is the slope term multiplying $(x-c)$---for the secant line this is an approximation to the related term for the tangent line. That is, the secant line approximates the tangent line, which is the linear function that best approximates the function at the point $(c, f(c))$. This is quantified by the *mean value theorem* which states under our assumptions on $f(x)$ that there exists some $\xi$ between $x$ and $c$ for which: @@ -135,20 +158,26 @@ $$ That is, $f(x) = f(c) + f'(c)(x-c) + f''(\xi)/2\cdot(x-c)^2$, or $f(x)-tl(x)$ is as described.) -The secant line also has an interpretation that will generalize - it is the smallest order polynomial that goes through, or *interpolates*, the points $(c,f(c))$ and $(c+h, f(c+h))$. This is obvious from the construction - as this is how the slope is derived - but from the formula itself requires showing $tl(c) = f(c)$ and $tl(c+h) = f(c+h)$. The former is straightforward, as $(c-c) = 0$, so clearly $tl(c) = f(c)$. The latter requires a bit of algebra. +The secant line also has an interpretation that will generalize---it is the smallest order polynomial that goes through, or *interpolates*, the points $(c,f(c))$ and $(c+h, f(c+h))$. This is obvious from the construction---as this is how the slope is derived---but from the formula itself requires showing $sl(c) = f(c)$ and $sl(c+h) = f(c+h)$. The former is straightforward, as $(c-c) = 0$, so clearly $sl(c) = f(c)$. The latter requires a bit of algebra. We have: +::: {.relationship title="Linear approximation"} -> The best *linear* approximation at a point $c$ is related to the *linear* polynomial interpolating the points $c$ and $c+h$ as $h$ goes to $0$. +The best *linear* approximation at a point $c$ is related to the *linear* polynomial interpolating the points $c$ and $c+h$ as $h$ goes to $0$. + +::: -This is the relationship we seek to generalize through our round about approach below: +This is the relationship we seek to generalize through our round-about approach below: +::: {.relationship title="Best approximation by a polynomial of a given degree or less"} -> The best approximation at a point $c$ by a polynomial of degree $n$ or less is related to the polynomial interpolating through the points $c, c+h, \dots, c+nh$ as $h$ goes to $0$. +The best approximation at a point $c$ by a polynomial of degree $n$ or less is related to the polynomial interpolating through the points $c, c+h, \dots, c+nh$ as $h$ goes to $0$. + +::: @@ -158,19 +187,25 @@ As in the linear case, there is flexibility in the exact points chosen for the i --- -Now, we take a small detour to define some notation. Instead of writing our two points as $c$ and $c+h,$ we use $x_0$ and $x_1$. For any set of points $x_0, x_1, \dots, x_n$, recursively define the Newton **divided differences** of $f$ inductively, as follows: +Now, we take a small detour to define some notation. Instead of writing our two points as $c$ and $c+h,$ we use $x_0$ and $x_1$. For any set of points $x_0, x_1, \dots, x_n$, recursively define + + +::: {.definition title="Newton divided differences"} + +The divided differences of $f$ are defined inductively, as follows: $$ \begin{align*} -f[x_0] &= f(x_0) \\ -f[x_0, x_1] &= \frac{f[x_1] - f[x_0]}{x_1 - x_0}\\ -\cdots &\\ +f[x_0] &= f(x_0) \\ +f[x_0, x_1] &= \frac{f[x_1] - f[x_0]}{x_1 - x_0}\\ +f[x_0, x_1, x_2] &= \frac{f[x_1, x_2] - f[x_0, x_1]}{x_2 - x_0}\\ +\vdots\qquad & \qquad\qquad \vdots\\ f[x_0, x_1, x_2, \dots, x_n] &= \frac{f[x_1, \dots, x_n] - f[x_0, x_1, x_2, \dots, x_{n-1}]}{x_n - x_0}. \end{align*} $$ -We see the first two values look familiar, and to generate more we just take certain ratios akin to those formed when finding a secant line. - +We see the first two values look familiar, and to generate more we recursively take certain ratios akin to those formed when finding a secant line. +::: With this notation the secant line can be re-expressed as: @@ -179,7 +214,11 @@ $$ sl(x) = f[c] + f[c, c+h] \cdot (x-c). $$ -If we think of $f[c, c+h]$ as an approximate *first* derivative, we have an even stronger parallel between a secant line $x=c$ and the tangent line at $x=c$: $tl(x) = f(c) + f'(c)\cdot (x-c)$. +If we think of $f[c, c+h]$ as an approximate *first* derivative, we have an even stronger parallel between a secant line $x=c$ and the tangent line at $x=c$: + +$$ +tl(x) = f(c) + f'(c)\cdot (x-c). +$$ We use `SymPy` to investigate. First we create a *recursive* function to compute the divided differences: @@ -225,8 +264,7 @@ simplify(2ex₂) This relationship between higher-order divided differences and higher-order derivatives generalizes. This is expressed in this [theorem](http://tinyurl.com/zjogv83): -:::{.callout-note} -## Mean value theorem for Divided differences +:::{.theorem title="Mean value theorem for divided differences"} Suppose $m=x_0 < x_1 < x_2 < \dots < x_n=M$ are distinct points. If $f$ has $n$ continuous derivatives then there exists a value $\xi$, where $m < \xi < M$, satisfying: @@ -242,10 +280,13 @@ This immediately applies to the above, where we parameterized by $h$: $x_0=c, x_ A proof based on Rolle's theorem appears in the appendix. -## Quadratic approximations; interpolating polynomials +## Quadratic approximations -Why the fuss? The answer comes from a result of Newton on *interpolating* polynomials. Consider a function $f$ and $n+1$ points $x_0$, $x_1, \dots, x_n$. Then an interpolating polynomial is a polynomial of least degree that goes through each point $(x_i, f(x_i))$. The [Newton form](https://en.wikipedia.org/wiki/Newton_polynomial) of such a polynomial can be written as: +Why the fuss? The answer comes from a result of Newton on *interpolating* polynomials. Consider a function $f$ and $n+1$ points $x_0$, $x_1, \dots, x_n$. Then an interpolating polynomial is a polynomial of least degree that goes through each point $(x_i, f(x_i))$. + + +The [Newton form](https://en.wikipedia.org/wiki/Newton_polynomial) of such a polynomial can be written as: $$ @@ -259,20 +300,7 @@ $$ The case $n=0$ gives the value $f[x_0] = f(c)$, which can be interpreted as the slope-$0$ line that goes through the point $(c,f(c))$. -We are familiar with the case $n=1$, with $x_0=c$ and $x_1=c+h$, this becomes our secant-line formula: - - -$$ -f[c] + f[c, c+h](x-c). -$$ - -As mentioned, we can verify directly that it interpolates the points $(c,f(c))$ and $(c+h, f(c+h))$. Here we let `SymPy` do the algebra: - - -```{julia} -p₁ = u[c] + u[c, c+h] * (x-c) -p₁(x => c) - u(c), p₁(x => c+h) - u(c+h) -``` +We are familiar with the case $n=1$, with $x_0=c$ and $x_1=c+h$, as this becomes our secant-line formula/ Now for something new. Take the $n=2$ case with $x_0 = c$, $x_1 = c + h$, and $x_2 = c+2h$. Then the interpolating polynomial is: @@ -285,7 +313,7 @@ We add the next term to our previous polynomial and simplify ```{julia} -p₂ = p₁ + u[c, c+h, c+2h] * (x-c) * (x-(c+h)) +p₂ = u[c] + u[c, c+h] * (x-c) + u[c, c+h, c+2h] * (x-c) * (x-(c+h)) simplify(p₂) ``` @@ -296,7 +324,7 @@ We can check that this interpolates the three points. Notice that at $x_0=c$ and p₂(x => c+2h) - u(c+2h) ``` -Hmm, doesn't seem correct - that was supposed to be $0$. The issue isn't the math, it is that SymPy needs to be encouraged to simplify: +Hmm, doesn't seem correct---that was supposed to be $0$. The issue isn't the math, it is that SymPy needs to be encouraged to simplify: ```{julia} @@ -320,9 +348,9 @@ $$ This is clearly related to the tangent line approximation of $f(x)$ at $x=c$, but carrying an extra quadratic term. -Here we visualize the approximations with the function $f(x) = \cos(x)$ at $c=0$. - +In @fig-f-linear-quadratic-approximations we visualize the approximations with the function $f(x) = \cos(x)$ at $c=0$. +::: {#fig-f-linear-quadratic-approximations} ```{julia} #| hold: true f(x) = cos(x) @@ -340,6 +368,9 @@ plot!(p, x->f(c) + fp*(x-c) + (1/2)*fpp*(x-c)^2, a, b; color=:green, alpha=0.25, p ``` +Plot of function, a linear approximation, and a quadratic approximation at $c=0$ +::: + This graph illustrates that the extra quadratic term can track the curvature of the function, whereas the tangent line itself can't. So, we have a polynomial which is a "better" approximation, is it the best approximation? @@ -347,16 +378,19 @@ The Cauchy mean value theorem, as in the case of the tangent line, will guarante $$ -f(x) - \left(f(c) + f'(c) \cdot(x-c) + \frac{1}{2}\cdot f''(c) \cdot (x-c)^2 \right) = +\begin{align*} +f(x) &- \left(f(c) + f'(c) \cdot(x-c) + \frac{1}{2}\cdot f''(c) \cdot (x-c)^2 \right) \\ +&= \frac{1}{3!}f'''(\xi) \cdot (x-c)^3. +\end{align*} $$ In this sense, the above quadratic polynomial, called the Taylor Polynomial of degree 2, is the best *quadratic* approximation to $f$, as the difference goes to $0$ at a rate of $(x-c)^3$. -The graphs of the secant line and approximating parabola for $h=1/4$ are similar: - +@fig-f-linear-quadratic-approximations-divided-differences shows the similar graphs of the secant line and the interpolating parabola for $h=1/4$. +::: {#fig-f-linear-quadratic-approximations-divided-differences} ```{julia} #| hold: true f(x) = cos(x) @@ -376,7 +410,10 @@ plot!(p, x -> f0 + fd*(x-x0) + fdd * (x-x0)*(x-x1), a,b, color=:green, alpha=0.2 p ``` -Though similar, the graphs are **not** identical, as the interpolating polynomials aren't the best approximations. For example, in the tangent-line graph the parabola only intersects the cosine graph at $x=0$, whereas for the secant-line graph - by definition - the parabola intersects the graph at least $2$ times and the interpolating polynomial $3$ times (at $x_0$, $x_1$, and $x_2$). +Plot of function, a linear approximation using divided differences, and a quadratic approximation using divided differences at $c=0$ +::: + +Though similar, the graphs in @fig-f-linear-quadratic-approximations-divided-differences are **not** identical to those in @fig-f-linear-quadratic-approximations, as the interpolating polynomials aren't the best approximations. For example, in the tangent-line graph the parabola only intersects the cosine graph at $x=0$, whereas for the secant-line graph---by definition---the parabola intersects the graph at least $2$ times and the interpolating polynomial $3$ times (at $x_0$, $x_1$, and $x_2$). ##### Example @@ -389,9 +426,9 @@ $$ f(0) + f'(0) \cdot t + \frac{1}{2} \cdot f''(0) \cdot t^2 = 0 + 1t - \frac{t^2}{2} $$ -A graph shows the difference: - +@fig-plot-log-1-plus-t-and-linear-quadratic-approximations shows the difference. +::: {#fig-plot-log-1-plus-t-and-linear-quadratic-approximations} ```{julia} #| hold: true f(t) = log(1 + t) @@ -401,6 +438,9 @@ plot!(t -> t, a, b) plot!(t -> t - t^2/2, a, b) ``` +Plot of $f(t) = \ln(1+t)$ and a linear and quadratic approximation at $0$. +::: + Though we can see that the tangent line is a good approximation, the quadratic polynomial tracks the logarithm better farther from $c=0$. @@ -417,8 +457,10 @@ By ignoring friction, the total energy is conserved giving: $$ -K = \frac{1}{2}m v^2 + mgR \cdot (1 - \cos(\theta)) = -\frac{1}{2} m R^2 (\frac{d\theta}{dt})^2 + mgR \cdot (1 - \cos(\theta)). +\begin{align*} +K &= \frac{1}{2}m v^2 + mgR \cdot (1 - \cos(\theta)) \\ +&= \frac{1}{2} m R^2 (\frac{d\theta}{dt})^2 + mgR \cdot (1 - \cos(\theta)). +\end{align*} $$ The value of $1-\cos(\theta)$ inhibits further work which would be possible were there an easier formula there. In fact, we could try the excellent approximation $1 - \theta^2/2$ from the quadratic approximation. Then we have: @@ -490,27 +532,27 @@ f[x_0] &+ f[x_0,x_1] \cdot (x - x_0) + f[x_0, x_1, x_2] \cdot (x - x_0)\cdot(x-x $$ -and taking $x_i = c + i\cdot h$, for a given $n$, we have in the limit as $h > 0$ goes to zero that coefficients of this polynomial converge: +and taking $x_i = c + i\cdot h$, for a given $n$, in the limit as $h0$ goes to zero the coefficients of this polynomial converge. -:::{.callout-note} -## Taylor polynomial of degree $n$ +:::{.definition title="Taylor polynomial of degree n"} Suppose $f(x)$ has $n+1$ derivatives (continuous on $c$ and $x$), then $$ T_n(x) = f(c) + f'(c)\cdot(x-c) + \frac{f''(c)}{2!}(x-c)^2 + \cdots + \frac{f^{(n)}(c)}{n!} (x-c)^n, $$ -will be the best approximation of degree $n$ or less to $f$, near $c$. - -The error will be given - again by an application of the Cauchy mean value theorem: +will be the best polynomial approximation of degree $n$ or less to $f$, near $c$. +The error will be given by: $$ \frac{1}{(n+1)!} \cdot f^{(n+1)}(\xi) \cdot (x-c)^n $$ -for some $\xi$ between $c$ and $x$. +for some $\xi$ between $c$ and $x$. (Again by an application of the Cauchy mean value theorem.) + +When $c=0$, a Taylor polynomial is also called a Maclaurin polynomial. ::: @@ -553,37 +595,24 @@ The output of `series` includes a big "Oh" term, which identifies the scale of t :::{.callout-note} ## Note -A Taylor polynomial of degree $n$ consists of $n+1$ terms and an error term. The "Taylor series" (below) is an *infinite* collection of terms, the first $n+1$ matching the Taylor polynomial of degree $n$. The fact that series are *infinite* means care must be taken when even talking about their existence, unlike a Taylor polynomial, which is just a polynomial and exists as long as a sufficient number of derivatives are available. +A Taylor polynomial of degree $n$ consists of $n+1$ terms and has an error term. The "Taylor series" (below) is an *infinite* collection of terms, the first $n+1$ matching the Taylor polynomial of degree $n$. The fact that series are *infinite* means care must be taken when even talking about their existence, unlike a Taylor polynomial, which is just a polynomial and exists as long as a sufficient number of derivatives are available. ::: -We define a function to compute Taylor polynomials from a function. The following returns a function, not a symbolic object, using `D`, from `CalculusWithJulia`, which is based on `ForwardDiff.derivative`, to find higher-order derivatives: - +##### Example: visualizing the approximations ```{julia} +#| echo: false +using CalculusWithJulia: D function taylor_poly(f, c=0, n=2) x -> f(c) + sum(D(f, i)(c) * (x-c)^i / factorial(i) for i in 1:n) end +nothing ``` -With a function, we can compare values. For example, here we see the difference between the Taylor polynomial and the answer for a small value of $x$: - - -```{julia} -#| hold: true -a = .1 -f(x) = log(1+x) -Tn = taylor_poly(f, 0, 5) -Tn(a) - f(a) -``` - -### Plotting - - -Let's now visualize a function and the two approximations - the Taylor polynomial and the interpolating polynomial. We use this function to generate the interpolating polynomial as a function: - ```{julia} +#| echo: false function newton_form(f, xs) x -> begin tot = divided_differences(f, xs[1]) @@ -593,12 +622,15 @@ function newton_form(f, xs) tot end end +nothing ``` -To see a plot, we have +@fig-plot-newton-form-taylor-poly} shows a plot of $f(x) = \sin(x)$ it's Taylor at $c=0$, $T_4$, and the fourth degree interpolating polynomial at $c, c+h, c+2h, c+3h, c+4h$, with $h=1/4$. Both polynomials track the function well near $0$, but the Taylor polynomial does a better job approximating $f(x)$ farther from $0$. +::: {#fig-plot-newton-form-taylor-poly} ```{julia} +#| echo: false f(x) = sin(x) c, h, n = 0, 1/4, 4 int_poly = newton_form(f, [c + i*h for i in 0:n]) @@ -609,32 +641,27 @@ plot!(int_poly; color=:green, label="interpolating") plot!(tp; color=:red, label="Taylor") ``` -To get a better sense, we plot the residual differences here: +Plot of function, its Taylor polynomial, $T-4$ and a fourth-degree interpolating polynomial +::: + +In @fig-plot-1-minus-cos-x-and-3-taylor-polys we make a plot of the Taylor polynomial for different sizes of $n$ for the function $f(x) = 1 - \cos(x)$: + +::: {#fig-plot-1-minus-cos-x-and-3-taylor-polys} ```{julia} -d1(x) = f(x) - int_poly(x) -d2(x) = f(x) - tp(x) -plot(d1, a, b; linecolor=:blue, label="interpolating") -plot!(d2; linecolor=:green, label="Taylor") -``` - -The graph should be $0$ at each of the points in `xs`, which we can verify in the graph above. Plotting over a wider region shows a common phenomenon that these polynomials approximate the function near the values, but quickly deviate away: - - -In this graph we make a plot of the Taylor polynomial for different sizes of $n$ for the function $f(x) = 1 - \cos(x)$: - - -```{julia} -#| hold: true +#| echo: false f(x) = 1 - cos(x) a, b = -pi, pi -plot(f, a, b, linewidth=5, label="f") -plot!(taylor_poly(f, 0, 2), label="T₂") -plot!(taylor_poly(f, 0, 4), label="T₄") -plot!(taylor_poly(f, 0, 6), label="T₆") +plot(f, a, b, linewidth=5, label=L"1 - \cos(x)") +plot!(taylor_poly(f, 0, 2), label=L"T_2") +plot!(taylor_poly(f, 0, 4), label=L"T_4") +plot!(taylor_poly(f, 0, 6), label=L"T_6") ``` +Plot of $f(x) = 1 - \cos(x)$ and Taylor polynomials of degree $2$, $4$, and $6$ +::: + Though all are good approximations near $c=0$, as more terms are included, the Taylor polynomial becomes a better approximation over a wider range of values. @@ -655,7 +682,11 @@ Suppose $R$ is the radius of the earth and $h$ the height above the earth assumi $$ -P = \frac{2\pi}{\sqrt{G\cdot M}} \cdot (h+R)^{3/2} = \frac{2\pi}{\sqrt{G\cdot M}} \cdot R^{3/2} \cdot (1 + h/R)^{3/2} = P_0 \cdot (1 + h/R)^{3/2}, +\begin{align*} +P &= \frac{2\pi}{\sqrt{G\cdot M}} \cdot (h+R)^{3/2} \\ +&= \frac{2\pi}{\sqrt{G\cdot M}} \cdot R^{3/2} \cdot (1 + h/R)^{3/2} \\ +&= P_0 \cdot (1 + h/R)^{3/2}, +\end{align*} $$ where $P_0$ collects terms that involve the constants. @@ -678,7 +709,7 @@ $$ Typically, if $h$ is much smaller than $R$ the first term is enough giving a formula like $P \approx P_0 \cdot(1 + \frac{3h}{2R})$. -A satellite phone utilizes low orbit satellites to relay phone communications. The [Iridium](http://www.kddi.com/english/business/cloud-network-voice/satellite/iridium/mobile/) system uses satellites with an elevation $h=780km$. The radius of the earth is $3,959 miles$, the mass of the earth is $5.972 × 10^{24} kg$, and the gravitational [constant](https://en.wikipedia.org/wiki/Gravitational_constant), $G$ is $6.67408 \cdot 10^{-11}$ $m^3/(kg \cdot s^2)$. +A satellite phone utilizes low orbit satellites to relay phone communications. The [Iridium](http://www.kddi.com/english/business/cloud-network-voice/satellite/iridium/mobile/) system uses satellites with an elevation $h=780km$. The radius of the earth is $3,959$ miles, the mass of the earth is $5.972 × 10^{24} kg$, and the gravitational [constant](https://en.wikipedia.org/wiki/Gravitational_constant), $G$ is $6.67408 \cdot 10^{-11}$ $m^3/(kg \cdot s^2)$. Compare the approximate value with $1$ term to the exact value. @@ -732,7 +763,7 @@ Prealₛ, P1ₛ, P5ₛ We see the Taylor polynomial underestimates badly in this case. A reminder that these approximations are locally good, but may not be good on all scales. Here $h \approx 3R$. We can see from this graph of $(1+x)^{3/2}$ and its $5$th degree Taylor polynomial $T_5$ that it is a bad approximation when $x > 2$. - +::: {#fig-plot-function-5th-degree-poly} ```{julia} #| echo: false f1(x) = (1+x)^(3/2) @@ -741,6 +772,8 @@ plot(f1, -1, 3, linewidth=4, legend=false) plot!(p2, -1, 3) ``` +Plot of $f(x)$ and a fifth-degree Taylor polynomial +::: --- @@ -761,7 +794,7 @@ Preal = P0 * (1 + HR)^(3/2) # in seconds Preal, uconvert(hr, Preal) # ≈ 11.65 hours ``` -We see `Preal` has the right units - the units of mass and distance cancel leaving a measure of time - but it is hard to sense how long this is. Converting to hours, helps us see the satellite orbits about twice per day. +We see `Preal` has the right units---the units of mass and distance cancel leaving a measure of time---but it is hard to sense how long this is. Converting to hours, helps us see the satellite orbits about twice per day. ##### Example: computing $\log(x)$ @@ -773,7 +806,7 @@ Where exactly does the value assigned to $\log(5)$ come from? The value needs to But how? One can see details of a possible way [here](https://github.com/musm/Amal.jl/blob/master/src/log.jl). -First, there is usually a reduction stage. In this phase, the problem is transformed in a manner to one involving only a fixed interval of values. For this, function values of $k$ and $m$ are found so that $x = 2^k \cdot (1+m)$ *and* $\sqrt{2}/2 < 1+m < \sqrt{2}$. If these are found, then $\log(x)$ can be computed with $k \cdot \log(2) + \log(1+m)$. The first value - a multiplication - can easily be computed using pre-computed value of $\log(2)$, the second then *reduces* the problem to an interval. +First, there is usually a reduction stage. In this phase, the problem is transformed in a manner to one involving only a fixed interval of values. For this, function values of $k$ and $m$ are found so that $x = 2^k \cdot (1+m)$ *and* $\sqrt{2}/2 < 1+m < \sqrt{2}$. If these are found, then $\log(x)$ can be computed with $k \cdot \log(2) + \log(1+m)$. The first value---a multiplication---can easily be computed using a pre-computed value of $\log(2)$, the second then *reduces* the problem to an interval. Now, for this problem a further trick is utilized, writing $s= m/(2+m)$ so that $\log(1+m)=\log(1+s)-\log(1-s)$ for some small range of $s$ values. These combined make it possible to compute $\log(x)$ for any real $x$. @@ -786,7 +819,7 @@ To compute $\log(1\pm s)$, we can find a Taylor polynomial. Let's go out to degr @syms s aₗ = series(log(1 + s), s, 0, 19) bₗ = series(log(1 - s), s, 0, 19) -a_b = (aₗ - bₗ).removeO() # remove"Oh" not remove"zero" +a_b = (aₗ - bₗ).removeO() # remove "Oh" not remove"zero" ``` This is re-expressed as $2s + s \cdot p$ with $p$ given by: @@ -809,7 +842,7 @@ How big can the error be between this *approximations* and $\log(1+m)$? The expr Max = (x/(2+x))(x => sqrt(2) - 1) ``` -The error term is like $2/19 \cdot \xi^{19}$ which is largest at this value of $M$. Large is relative - it is really small: +The error term is like $2/19 \cdot \xi^{19}$ which is largest at this value of $M$. Large is relative---it is really small: ```{julia} @@ -838,7 +871,7 @@ The two values differ by less than $10^{-16}$, as advertised. Re-assembling then Δ = k * log(2) + (m - s*(m-pₗ)) - log(5) ``` -The actual code is different, as the Taylor polynomial isn't used. The Taylor polynomial is a great approximation near a point, but there might be better polynomial approximations for all values in an interval. In this case there is, and that polynomial is used in the production setting. This makes things a bit more efficient, but the basic idea remains - for a prescribed accuracy, a polynomial approximation can be found over a given interval, which can be cleverly utilized to solve for all applicable values. +The actual code is different, as the Taylor polynomial isn't used. The Taylor polynomial is a great approximation near a point, but there might be better polynomial approximations for all values in an interval. In this case there is, and that polynomial is used in the production setting. This makes things a bit more efficient, but the basic idea remains---for a prescribed accuracy, a polynomial approximation can be found over a given interval, which can be cleverly utilized to solve for all applicable values. ##### Example: higher order derivatives of the inverse function @@ -872,11 +905,11 @@ That is: $$ -b_n \left(\sum_{j=1}^n a_j (\Delta_x)^j \right)^n = +b_n \left(\sum_{j=1}^n a_j (\Delta_x)^j \right)^n \approx (x_0 + \Delta_x) - \left( x_0 + \sum_{i=1}^{n-1} b_i \left(\sum_{j=1}^n a_j (\Delta_x)^j \right)^i \right) $$ -Solving for $b_n = g^{(n)}(y_0) / n!$ gives the formal expression: +Solving for $b_n = g^{(n)}(y_0) / n!$ gives a formal expression for $n$th derivative of the inverse function for $f(x)$, denoted $g(x)$: $$ @@ -937,7 +970,7 @@ The `solve` function is used to identify $g^{(n)}$ represented in terms of lower ## Taylor series -Recall a *power series* has the form $\sum_{n=0}^\infty a_n (x-c)^n$. Power series have a radius of convergence ($|x - c| < r$) for which the series converges and diverges when $|x-c| > r$. +Recall a *power series* has the form $\sum_{n=0}^\infty a_n (x-c)^n$. Power series have a radius of convergence $r$ for which the series converges if $|x - c| < r$ and diverges when $|x-c| > r$. The Taylor polynomial formula can be extended to a formal power series with through @@ -993,8 +1026,8 @@ choices = [ "``\\sum_{k=0}^{4} (-1)^k/(2k+1)! \\cdot x^{2k+1}``", "``\\sum_{k=0}^{10} x^n/n!``" ] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` ###### Question @@ -1012,8 +1045,8 @@ choices = [ "``\\sum_{k=0}^{4} (-1)^k/(2k+1)! \\cdot x^{2k+1}``", "``\\sum_{k=0}^{10} x^n/n!``" ] -answ = 4 -radioq(choices, answ) +answer = 4 +radioq(choices, answer) ``` ###### Question @@ -1031,8 +1064,8 @@ choices = [ "``\\sum_{k=0}^{4} (-1)^k/(2k+1)! \\cdot x^{2k+1}``", "``\\sum_{k=0}^{10} x^n/n!``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1050,8 +1083,8 @@ choices = [ "``1/5!``", "``2/15``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1071,8 +1104,8 @@ choices = [ "``x^2``", "``x^2 \\cdot (x - x^3/3! + x^5/5!)``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1094,8 +1127,8 @@ If this is true, then formally evaluating at $x=0$ gives $f(0) = a$, so $a$ is d choices = ["``f''''(0) = e``", "``f''''(0) = 4 \\cdot 3 \\cdot 2 e = 4! e``", "``f''''(0) = 0``"] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question @@ -1123,8 +1156,8 @@ yesnoq(true) #| hold: true #| echo: false choices =["It is increasing", "It is decreasing", "It both increases and decreases"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` * The maximum value of $e^x$ over $[-1,1]$ occurs at @@ -1134,8 +1167,8 @@ radioq(choices, answ) #| hold: true #| echo: false choices=["A critical point", "An end point"] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` * Which theorem tells you that for a *continuous* function over *closed* interval, a maximum value will exist? @@ -1148,8 +1181,8 @@ choices = [ "The intermediate value theorem", "The mean value theorem", "The extreme value theorem"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` * What is the *largest* possible value of the error: @@ -1161,8 +1194,8 @@ radioq(choices, answ) choices = [ "``1/6!\\cdot e^1 \\cdot 1^6``", "``1^6 \\cdot 1 \\cdot 1^6``"] -answ = 1 -radioq(choices,answ) +answer = 1 +radioq(choices,answer) ``` ###### Question @@ -1182,8 +1215,8 @@ L"The function $e^x$ is increasing, so takes on its largest value at the endpoin L"The function has a critical point at $x=1/2$", L"The function is monotonic in $k$, so achieves its maximum at $k+1$" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` Assuming the above is right, find the smallest value $k$ guaranteeing an error no more than $10^{-16}$. @@ -1276,9 +1309,9 @@ Now, assuming the Newton form is correct, a [proof](http://tinyurl.com/zjogv83) By Rolle's theorem, between any two such zeros $x_i, x_{i+1}$, $0 \leq i < n$ there must be a zero of the derivative of $h(x)$, say $\xi^1_i$. So $h'(x)$ has zeros $\xi^1_0 < \xi^1_1 < \dots < \xi^1_{n-1}$. -We visualize this with $f(x) = \sin(x)$ and $x_i = i$ for $i=0, 1, 2, 3$, The $x_i$ values are indicated with circles, the $\xi^1_i$ values indicated with squares: - +We visualize this with $f(x) = \sin(x)$ and $x_i = i$ for $i=0, 1, 2, 3$ in @fig-plot-of-between-any-two-zeros-is-a-zero-of-fprime, The $x_i$ values are indicated with circles, the $\xi^1_i$ values indicated with squares. +::: {#fig-plot-of-between-any-two-zeros-is-a-zero-of-fprime} ```{julia} #| hold: true #| echo: false @@ -1293,4 +1326,7 @@ scatter!(xs, h1.(xs), markersize=5) scatter!(cps, h1.(cps), markersize=5, marker=:square) ``` +Between any adjacent pair of zeros, there lies at least one relative extrema +::: + Again by Rolle's theorem, between any pair of adjacent zeros $\xi^1_i, \xi^1_{i+1}$ there must be a zero $\xi^2_i$ of $h''(x)$. So there are $n-1$ zeros of $h''(x)$. Continuing, we see that there will be $n+1-3$ zeros of $h^{(3)}(x)$, $n+1-4$ zeros of $h^{4}(x)$, $\dots$, $n+1-(n-1)$ zeros of $h^{n-1}(x)$, and finally $n+1-n$ ($1$) zeros of $h^{(n)}(x)$. Call this last zero $\xi$. It satisfies $x_0 \leq \xi \leq x_n$. Further, $0 = h^{(n)}(\xi) = f^{(n)}(\xi) - g^{(n)}(\xi)$. But $g$ is a degree $n$ polynomial, so the $n$th derivative is the coefficient of $x^n$ times $n!$. In this case we have $0 = f^{(n)}(\xi) - f[x_0, \dots, x_n] n!$. Rearranging yields the result. diff --git a/quarto/index.qmd b/quarto/index.qmd index 581cae5..b922e3d 100644 --- a/quarto/index.qmd +++ b/quarto/index.qmd @@ -83,15 +83,7 @@ A *very* special thanks goes out to `@fangliu-tju` for their careful and most-ap ## Running Julia -`Julia` is installed quite easily with the `juliaup` utility. There are some brief installation notes in the overview of `Julia` commands. To run `Julia` through the web (though in a resource-constrained manner), these links resolve to `binder.org` instances: - - -* [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jverzani/CalculusWithJuliaBinder.jl/main?labpath=blank-notebook.ipynb) (Image without SymPy) - - -* [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jverzani/CalculusWithJuliaBinder.jl/sympy?labpath=blank-notebook.ipynb) (Image with SymPy, longer to load) - - +For individual users `Julia` is installed quite easily with the `juliaup` utility or by other means. There are some brief installation notes in the overview of `Julia` commands. ---- diff --git a/quarto/integrals/arc_length.qmd b/quarto/integrals/arc_length.qmd index 0753932..41a73e0 100644 --- a/quarto/integrals/arc_length.qmd +++ b/quarto/integrals/arc_length.qmd @@ -8,8 +8,7 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia -using Plots -plotly() +using Plots; plotly() using SymPy using QuadGK using Roots @@ -17,7 +16,7 @@ using Roots --- - +::: {#fig-kids-jumprope} ```{julia} #| hold: true #| echo: false @@ -31,10 +30,12 @@ A kids' jump rope by Lifeline is comprised of little plastic segments of uniform nothing ``` -![A kids' jump rope by Lifeline is comprised of little plastic segments of uniform length around a cord. The length of the rope can be computed by adding up the lengths of each segment, regardless of how the rope is arranged. -](./figures/jump-rope.png) +![](./figures/jump-rope.png) -The length of the jump rope in the picture can be computed by either looking at the packaging it came in, or measuring the length of each plastic segment and multiplying by the number of segments. The former is easier, the latter provides the intuition as to how we can find the length of curves in the $x-y$ plane. The idea is old, [Archimedes](http://www.maa.org/external_archive/joma/Volume7/Aktumen/Polygon.html) used fixed length segments of polygons to approximate $\pi$ using the circumference of circle producing the bounds $3~\frac{1}{7} > \pi > 3~\frac{10}{71}$. +A kids' jump rope by Lifeline is comprised of little plastic segments of uniform length around a cord. The length of the rope can be computed by adding up the lengths of each segment, regardless of how the rope is arranged. +::: + +The length of the jump rope in @fig-kids-jumprope can be computed by either looking at the packaging it came in, or measuring the length of each plastic segment and multiplying by the number of segments. The former is easier, the latter provides the intuition as to how we can find the length of curves in the $x-y$ plane. The idea is old, [Archimedes](http://www.maa.org/external_archive/joma/Volume7/Aktumen/Polygon.html) used fixed length segments of polygons to approximate $\pi$ using the circumference of circle producing the bounds $3~\frac{1}{7} > \pi > 3~\frac{10}{71}$. A more modern application is the algorithm used by GPS devices to record a path taken. However, rather than record times for a fixed distance traveled, the GPS device records position ($(x,y)$) or longitude and latitude at fixed units of time - similar to how parametric functions are used. The device can then compute distance traveled and speed using some familiar formulas. @@ -43,13 +44,10 @@ A more modern application is the algorithm used by GPS devices to record a path ## Arc length formula -Recall the distance formula gives the distance between two points: $\sqrt{(x_1 - x_0)^2 + (y_1 - y_0)^2}$. - Consider now two functions $g(t)$ and $f(t)$ and the parameterized graph between $a$ and $b$ given by the points $(g(t), f(t))$ for $a \leq t \leq b$. Assume that both $g$ and $f$ are differentiable on $(a,b)$ and continuous on $[a,b]$ and furthermore that $\sqrt{g'(t)^2 + f'(t)^2}$ is Riemann integrable. -::: {.callout-note icon=false} -## The arc length of a curve +::: {.definition title="The arc length of a parameterized curve"} For $f$ and $g$ as described, the arc length of the parameterized curve is given by @@ -63,13 +61,12 @@ For the special case of the graph of a function $f(x)$ between $a$ and $b$ the f :::{.callout-note} ## Note -The form of the integral may seem daunting with the square root and the derivatives. A more general writing would create a vector out of the two functions: $\phi(t) = \langle g(t), f(t) \rangle$. It is natural to then let $\phi'(t) = \langle g'(t), f'(t) \rangle$. With this, the integrand is just the norm - or length - of the derivative, or $L=\int \| \phi'(t) \| dt$. This is similar to the distance traveled being the integral of the speed, or the absolute value of the derivative of position. +The form of the integral may seem daunting with the square root and the derivatives. A more general writing would create a vector out of the two functions: $\phi(t) = \langle g(t), f(t) \rangle$. It is natural to then let $\phi'(t) = \langle g'(t), f'(t) \rangle$. With this, the integrand is just the norm---or length---of the derivative, or $L=\int \| \phi'(t) \| dt$. This is similar to the distance traveled being the integral of the speed, or the absolute value of the derivative of position. ::: -To see why, any partition of the interval $[a,b]$ by $a = t_0 < t_1 < \cdots < t_n =b$ gives rise to $n+1$ points in the plane given by $(g(t_i), f(t_i))$. - +::: {#fig-arc-length-as-riemann-sum-animation} ```{julia} #| hold: false #| echo: false @@ -113,21 +110,27 @@ end imgfile = tempname() * ".gif" gif(anim, imgfile, fps = 1) -caption = L""" - -The arc length of the parametric curve can be approximated using straight line segments connecting points. This gives rise to an integral expression defining the length in terms of the functions $f$ and $g$. - -""" +caption = "" plotly() ImageFile(imgfile, caption) ``` -The distance between points $(g(t_i), f(t_i))$ and $(g(t_{i-1}), f(t_{i-1}))$ is just +The arc length of the parametric curve can be approximated using straight line segments connecting points. This gives rise to an integral expression defining the length in terms of the functions $f$ and $g$. + +::: + +Recall the distance formula: + +$$ +\text{distance between points} = \sqrt{(x_1 - x_0)^2 + (y_1 - y_0)^2}. +$$ + +We use this to see why the arc-length formula has its form. Let $a = t_0 < t_1 < \cdots < t_n =b$ be any partition of the interval $[a,b]$. This partition gives rise to $n+1$ points in the plane given by $(g(t_i), f(t_i))$. The distance between points $(g(t_i), f(t_i))$ and $(g(t_{i-1}), f(t_{i-1}))$ is given by the distance formula: $$ -d_i = \sqrt{(g(t_i)-g(t_{i-1}))^2 + (f(t_i)-f(t_{i-1}))^2} +d_i = \sqrt{(g(t_i)-g(t_{i-1}))^2 + (f(t_i)-f(t_{i-1}))^2}. $$ The total approximate distance of the curve would be $L_n = d_1 + d_2 + \cdots + d_n$. This is exactly how we would compute the length of the jump rope or the distance traveled from GPS recordings. @@ -157,7 +160,7 @@ $$ The values $\xi_i$ and $\psi_i$ are guaranteed by the mean value theorem and must be in $[t_{i-1}, t_i]$. -With this, if $\sqrt{f'(t)^2 + g'(t)^2}$ is integrable, as assumed, then as the size of the partition goes to zero, the sum of the $d_i$, $L_n$, must converge to the integral: +With this, if $\sqrt{f'(t)^2 + g'(t)^2}$ is integrable, as assumed, then as the size of the partition goes to zero (as suggested in @fig-arc-length-as-riemann-sum-animation) the sum of the $d_i$, $L_n$, must converge to the integral: $$ @@ -190,14 +193,20 @@ F = integrate(sqrt(1 + (2x)^2), x) F(1) - F(0) ``` -That number has some context, as can be seen from the graph, which gives simple lower and upper bounds of $\sqrt{1^2 + 1^2} = 1.414...$ and $1 + 1 = 2$. - +That number has some context, as can be seen from the graph in @fig-plot-x-squared-bounds-are-easy-for-arclength, which gives simple lower and upper bounds of $\sqrt{1^2 + 1^2} = 1.414...$ and $1 + 1 = 2$. +::: {#fig-plot-x-squared-bounds-are-easy-for-arclength} ```{julia} +#| echo: false f(x) = x^2 -plot(f, 0, 1) +plot(f, 0, 1; legend=false, line=(2, :black)) +plot!([(0,0), (1,1)]; line=(1, :black, :dot)) +plot!([(0,0), (1,0), (1,1)]; line=(1, :black, :dash)) ``` +The graph of $f(x) = x^2$ over $[0,1]$ has length *between* $\sqrt{2}$ (dotted line) and $2 = 1 + 1$ (dashed line segments) +::: + :::{.callout-note} ## Note The integrand $\sqrt{1 + f'(x)^2}$ may seem odd at first, but it can be interpreted as the length of the hypotenuse of a right triangle with "run" of $1$ and "rise" of $f'(x)$. This triangle is easily formed using the tangent line to the graph of $f(x)$. By multiplying by $dx$, the integral is "summing" up the lengths of infinitesimal pieces of the tangent line approximation. @@ -211,8 +220,12 @@ Let $f(t) = R\cos(t)$ and $g(t) = R\sin(t)$. Then the parametric curve over $[0, $$ -L = \int_0^{2\pi} \sqrt{(R\cos(t))^2 + (-R\sin(t))^2} dt = R\int_0^{2\pi} \sqrt{\cos(t)^2 + \sin(t)^2} dt = -R\int_0^{2\pi} dt = 2\pi R. +\begin{align*} +L &= \int_0^{2\pi} \sqrt{(R\cos(t))^2 + (-R\sin(t))^2} dt\\ +&= R\int_0^{2\pi} \sqrt{\cos(t)^2 + \sin(t)^2} dt \\ +&= R\int_0^{2\pi} dt \\ +&= 2\pi R. +\end{align*} $$ ##### Example @@ -239,32 +252,35 @@ Which isn't so satisfying. From a quick graph, we see the answer should be no mo ```{julia} -N(ex) +float(ex) ``` ##### Example -A [catenary shape](http://en.wikipedia.org/wiki/Catenary) is the shape a hanging chain will take as it is suspended between two posts. It appears elsewhere, for example, power wires will also have this shape as they are suspended between towers. A formula for a catenary can be written in terms of the hyperbolic cosine, `cosh` in `julia` or exponentials. +A [catenary shape](http://en.wikipedia.org/wiki/Catenary) is the shape a hanging chain will take as it is suspended between two posts. It appears elsewhere, for example, power wires will also have this shape as they are suspended between towers. A formula for a catenary can be written in terms of the hyperbolic cosine, `cosh` in `Julia` or exponentials. $$ y = a \cosh(x/a) = a \cdot \frac{e^{x/a} + e^{-x/a}}{2}. $$ -Suppose we have the following chain hung between $x=-1$ and $x=1$ with $a = 2$: - +Suppose we have a chain hung between $x=-1$ and $x=1$ with $a = 2$, as in @fig-catenary-with-a-equal-2-over-minus-1-1. +::: {#fig-catenary-with-a-equal-2-over-minus-1-1} ```{julia} chain(x; a=2) = a * cosh(x/a) plot(chain, -1, 1) ``` +Plot of catenary $a\cosh(x/a)$ with $a=2$ over $[-1,1]$ +::: + How long is the chain? Looking at the graph we can guess an answer is between $2$ and $2.5$, say, but it isn't much work to get an approximate numeric answer. Recall, the accompanying `CalculusWithJulia` package defines `f'` to find the derivative using the `ForwardDiff` package. ```{julia} -quadgk(x -> sqrt(1 + chain'(x)^2), -1, 1)[1] +first(quadgk(x -> sqrt(1 + chain'(x)^2), -1, 1)) ``` We used a numeric approach, but this can be solved by hand and the answer is surprising. @@ -273,24 +289,27 @@ We used a numeric approach, but this can be solved by hand and the answer is sur ##### Example -This picture of Jasper John's [Near the Lagoon](http://www.artic.edu/aic/collections/artwork/184095) was taken at The Art Institute Chicago. - +@fig-jasper-johns-catenary-chicago shows Jasper Johns' [Near the Lagoon](http://www.artic.edu/aic/collections/artwork/184095) from The Art Institute of Chicago. +::: {#fig-jasper-johns-catenary-chicago} ```{julia} #| hold: true #| echo: false imgfile = "figures/johns-catenary.jpg" -caption = "One of Jasper Johns' Catenary series. Art Institute of Chicago." +caption = "One of Jasper Johns' Catenary series. The Art Institute of Chicago." # ImageFile(:integrals, imgfile, caption) nothing ``` -![One of Jasper Johns' Catenary series. Art Institute of Chicago.](./figures/johns-catenary.jpg) +![](./figures/johns-catenary.jpg) + +One of Jasper Johns' Catenary series (The Art Institute of Chicago) +::: The museum notes have -> For his Catenary series (1997–2003), of which Near the Lagoon is the largest and last work, Johns formed catenaries—a term used to describe the curve assumed by a cord suspended freely from two points—by tacking ordinary household string to the canvas or its supports. +> For his Catenary series (1997–2003), of which Near the Lagoon is the largest and last work, Johns formed catenaries—a term used to describe the curve assumed by a cord suspended freely from two points—by tacking ordinary household string to the canvas or its supports. @@ -324,7 +343,7 @@ Rounding, we take $a=13$. With these parameters ($a=13$, $b = 131$), we compute a = 13 b = 118 + a f(x) = cat(x; a=13, b=118+13) -quadgk(x -> sqrt(1 + f'(x)^2), -78/2, 78/2)[1] +first(quadgk(x -> sqrt(1 + f'(x)^2), -78/2, 78/2)) ``` ##### Example @@ -339,8 +358,9 @@ The Verrazzano-Narrows [Bridge](https://www.brownstoner.com/brooklyn-life/verraz Suppose the drop of the main cables is $147$ meters over this span. Then the cable itself can be modeled as a parabola with - * The $x$-intercepts $a = 1298/2$ and $-a$ and - * vertex $(0,b)$ with $b=-147$. +* The $x$-intercepts $a = 1298/2$ and $-a$ and + +* vertex $(0,b)$ with $b=-147$. The parabola that fits these three points is @@ -362,6 +382,7 @@ val, _ = quadgk(x -> sqrt(1 + f'(x)^2), -a, a) val ``` +::: {#fig-verrazzano-bridge-unloaded-during-construction} ```{julia} #| hold: true #| echo: false @@ -373,8 +394,12 @@ The Verrazzano-Narrows Bridge during construction. The unloaded suspension cable nothing ``` -![The Verrazzano-Narrows Bridge during construction. The unloaded suspension cables form a catenary.](./figures/verrazzano-unloaded.jpg) +![](./figures/verrazzano-unloaded.jpg) +The Verrazzano-Narrows Bridge during construction. The unloaded suspension cables form a catenary. +::: + +::: {#fig-verrazzano-bridge-loaded-after-construction} ```{julia} #| hold: true #| echo: false @@ -386,13 +411,15 @@ A rendering of the Verrazzano-Narrows Bridge after construction (cf. [nycgovpark nothing ``` -![A rendering of the Verrazzano-Narrows Bridge after construction (cf. [nycgovparks.org](https://www.nycgovparks.org/highlights/verrazano-bridge)). The uniformly loaded suspension cables would form a parabola, presumably a fact the artist of this rendering knew. (The spelling in the link is not the official spelling, which carries two zs.) -](./figures/verrazzano-loaded.jpg) +![](./figures/verrazzano-loaded.jpg) + +A rendering of the Verrazzano-Narrows Bridge after construction (cf. [nycgovparks.org](https://www.nycgovparks.org/highlights/verrazano-bridge)). The uniformly loaded suspension cables would form a parabola, presumably a fact the artist of this rendering knew. (The spelling in the link is not the official spelling, which carries two zs.) +::: ##### Example -The [nephroid](http://www-history.mcs.st-and.ac.uk/Curves/Nephroid.html) is a curve that can be described parametrically by +The [nephroid](https://mathshistory.st-andrews.ac.uk/Curves/Nephroid/) is a curve that can be described parametrically by $$ @@ -405,14 +432,17 @@ $$ Taking $a=1$ we have this graph: - +::: {#fig-nephroid-with-a-equal-1} ```{julia} a = 1 -𝒈(t) = a*(3cos(t) - cos(3t)) -𝒇(t) = a*(3sin(t) - sin(3t)) -plot(𝒈, 𝒇, 0, 2pi) +g(t) = a * (3cos(t) - cos(3t)) +f(t) = a * (3sin(t) - sin(3t)) +plot(g, f, 0, 2pi) ``` +Parametric plot of nephroid +::: + Find the length of the perimeter of the closed figure formed by the graph. @@ -420,7 +450,7 @@ We have $\sqrt{g'(t)^2 + f'(t)^2} = \sqrt{18 - 18\cos(2t)}$. An antiderivative i ```{julia} -quadgk(t -> sqrt(𝒈'(t)^2 + 𝒇'(t)^2), 0, 2pi)[1] +quadgk(t -> sqrt(g'(t)^2 + f'(t)^2), 0, 2pi)[1] ``` The answer seems like a floating point approximation of $24$, which suggests that this integral is tractable. Pursuing this, the integrand simplifies: @@ -446,14 +476,18 @@ By graphing, we see that integrating over $[0,2\pi]$ gives twice the answer to i $$ -L = \int_0^{2\pi} \sqrt{g'(t)^2 + f'(t)^2}dt = \int_0^{2\pi} 3\sqrt{2}\sqrt{2\sin(t)^2} = -3 \cdot 2 \cdot 2 \int_0^\pi \sin(t) dt = 3 \cdot 2 \cdot 2 \cdot 2 = 24. +\begin{align*} +L &= \int_0^{2\pi} \sqrt{g'(t)^2 + f'(t)^2}dt = \int_0^{2\pi} 3\sqrt{2}\sqrt{2\sin(t)^2}\\ +&= 3 \cdot 2 \cdot 2 \int_0^\pi \sin(t) dt = 3 \cdot 2 \cdot 2 \cdot 2 \\ +&= 24. +\end{align*} $$ ##### Example The following link shows how the perimeter of a complex figure relates to the perimeter of a circle: +::: {#fig-tweet-of-pineapple-shape-being-formed} ```{julia} #| echo: false tweet = """ @@ -463,10 +497,13 @@ tweet = """ HTMLoutput(tweet) ``` +Relationship of perimeter of circle to a complex shape +::: + ##### Example -A teacher of small children assigns his students the task of computing the length of a jump rope by counting the number of $1$-inch segments it is made of. He knows that if a student is accurate, no matter how fast or slow they count the answer will be the same. (That is, unless the student starts counting in the wrong direction by mistake). The teacher knows this, as he is certain that the length of curve is independent of its parameterization, as it is a property intrinsic to the curve. +A teacher of small children assigns his students the task of computing the length of a jump rope by counting the number of $1$-inch segments it is made of. He knows that if a student is accurate, no matter how fast or slow they count the answer will be the same. (That is, unless the student starts counting in the wrong direction by mistake). The teacher knows this, as they are certain that the length of curve is independent of its parameterization, as it is a property intrinsic to the curve. Mathematically, suppose a curve is described parametrically by $(g(t), f(t))$ for $a \leq t \leq b$. A new parameterization is provided by $\gamma(t)$. Suppose $\gamma$ is strictly increasing, so that an inverse function exists. (This assumption is implicitly made by the teacher, as it implies the student won't start counting in the wrong direction.) Then the same curve is described by composition through $(g(\gamma(u)), f(\gamma(u)))$, $\gamma^{-1}(a) \leq u \leq \gamma^{-1}(b)$. That the arc length is the same follows from substitution: @@ -495,7 +532,7 @@ $$ \gamma(u) = \int_0^u \sqrt{g'(t)^2 + f'(t)^2} dt. $$ -Supposing $\sqrt{g'(t)^2 + f'(t)^2}$ is continuous and positive, This transformation is increasing, as its derivative by the Fundamental Theorem of Calculus is $\gamma'(u) = \sqrt{g'(u)^2 + f'(u)^2}$, which by assumption is positive. (It is certainly non-negative.) So there exists an inverse function. That it exists is one thing, computing all of this is a different matter, of course. +Supposing $\sqrt{g'(t)^2 + f'(t)^2}$ is continuous and positive, This transformation is increasing, as its derivative---by the Fundamental Theorem of Calculus---is $\gamma'(u) = \sqrt{g'(u)^2 + f'(u)^2}$, which by assumption is positive. (It is certainly non-negative.) So there exists an inverse function. That it exists is one thing, computing all of this is a different matter, of course. For a simple example, we have $g(t) = R\cos(t)$ and $f(t)=R\sin(t)$ parameterizing the circle of radius $R$. The arc length between $0$ and $t$ is simply $\gamma(t) = Rt$, which we can easily see from the formula. The inverse of this function is $\gamma^{-1}(u) = u/R$, so we get the parameterization $(g(Rt), f(Rt))$ for $0/R \leq t \leq 2\pi/R$. @@ -517,38 +554,40 @@ Where $C = 2c + c^2$ is a constant. But, despite it not looking too daunting, th ```{julia} -𝒂, 𝒃 = 1, 2 -𝒔(u) = quadgk(t -> sqrt(𝒂^2 * sin(t)^2 + 𝒃^2 * cos(t)^2), 0, u)[1] +a, b = 1, 2 +s(u) = first(quadgk(t -> sqrt(a^2 * sin(t)^2 + b^2 * cos(t)^2), 0, u)) ``` -This has a graph, which does not look familiar, but we can see is monotonically increasing, so will have an inverse function: +This has a graph shown in @fig-plot-arc-length-parameterization which does not look familiar, but we can see is monotonically increasing, so will have an inverse function whose domain will be $[0, s(2\pi)]$. + +::: {#fig-plot-arc-length-parameterization} +```{julia} +plot(s, 0, 2pi) +``` + +Plot of $s$ over $[0, 2\pi]$. This monotonic function has some inverse function. +::: + +The inverse function can be found by solving numerically. We use the bracketing version of `find_zero` for this: ```{julia} -plot(𝒔, 0, 2pi) +sinv(u) = find_zero(x -> s(x) - u, (0, s(2pi))) ``` -The range is $[0, s(2\pi)]$. - - -The inverse function can be found by solving, we use the bracketing version of `find_zero` for this: - - -```{julia} -sinv(u) = find_zero(x -> 𝒔(x) - u, (0, 𝒔(2pi))) -``` - -Here we see visually that the new parameterization yields the same curve: - +@fig-plot-g-f-parameterized-by-arc-length shows that the new parameterization yields the same elliptical curve. +::: {#fig-plot-g-f-parameterized-by-arc-length} ```{julia} #| hold: true -g(t) = 𝒂 * cos(t) -f(t) = 𝒃 * sin(t) - -plot(t -> g(𝒔(t)), t -> f(𝒔(t)), 0, sinv(2*pi)) +g(t) = a * cos(t) +f(t) = b * sin(t) +plot(g∘s, f∘s, 0, sinv(2*pi)) ``` +Plot of $(g(s(t)), f(s(t)))$ shows the parameterization doesn't effect the shape of the resulting curve +::: + #### Example: An implication of concavity @@ -563,69 +602,79 @@ Following (faithfully) [Kantorwitz and Neumann](https://www.researchgate.net/pub ```{julia} #| hold: true #| echo: false -function trajectory(x; g = 9.8, v0 = 50, theta = 45*pi/180, k = 1/8) +let + gr() + function trajectory(x; g = 9.8, v0 = 50, theta = 45*pi/180, k = 1/8) + a = v0 * cos(theta) + (g/(k*a) + tan(theta))* x + (g/k^2) * log(1 - k/a*x) + end + v0 = 50; theta = 45*pi/180; k = 1/5 a = v0 * cos(theta) - (g/(k*a) + tan(theta))* x + (g/k^2) * log(1 - k/a*x) + Δ = a/k + a = find_zero(trajectory, (50, Δ-1/10)) + plt = plot(;legend=false, ylims=(0,90)) + plot!(trajectory,0, a; line=(5, :blue)) + + u=25 + fu = trajectory(u) + v = find_zero(x -> trajectory(x) - fu, (50, Δ)) + plot!([u,v], [fu,fu]; line=(1, :black))#, linestyle = :dash) + + c = find_zero(trajectory', (50, 100)) + plot!([c,c],[0, trajectory(c)]; line=(1, :black)) + + + h(y)= tangent(trajectory, u)(y) - tangent(trajectory, v)(y) + d = find_zero(h, (u,v)) + plot!(tangent(trajectory, u), 5, 110; line=(1, :black)) + plot!(tangent(trajectory, v), 80, 150; line=(1, :black)) + + plot!(zero) + 𝒚 = 4 + annotate!([(0, 𝒚, text(L"a")), (152, 𝒚, text(L"b")), (u, 𝒚, text(L"u")), + (v, 𝒚, text(L"v")), (c, 𝒚, text(L"c",:right))]) + + plotly() + plt end -v0 = 50; theta = 45*pi/180; k = 1/5 -𝒂 = v0 * cos(theta) -Δ = 𝒂/k -a = find_zero(trajectory, (50, Δ-1/10)) -plot(trajectory,0, a, legend=false, linewidth=5) - -u=25 -fu = trajectory(u) -v = find_zero(x -> trajectory(x) - fu, (50, Δ)) -plot!([u,v], [fu,fu])#, linestyle = :dash) - -c = find_zero(trajectory', (50, 100)) -plot!([c,c],[0, trajectory(c)]) - - -h(y)= tangent(trajectory, u)(y) - tangent(trajectory, v)(y) -d = find_zero(h, (u,v)) -plot!(tangent(trajectory, u), 0, 110) -plot!(tangent(trajectory, v), 75, 150) - -plot!(zero) -𝒚 = 4 -annotate!([(0, 𝒚, "a"), (152, 𝒚, "b"), (u, 𝒚, "u"), (v, 𝒚, "v"), (c, 𝒚, "c")]) ``` Graph of function $f(x)$ with both $f$ and $f'$ strictly concave down. ::: -By Rolle's theorem there exists $c$ in $(a,b)$, a critical point, as in the picture. There must be a critical point by Rolle's theorem, and it must be unique, as the derivative, which exists by the assumptions, must be strictly decreasing due to concavity of $f$ and hence there can be at most $1$ critical point. +By Rolle's theorem there exists a critical point $c$ in $(a,b)$, as in the picture. There must be a unique critical point as the derivative, which exists by the assumptions, must be strictly decreasing due to concavity of $f$. Take $a < u < c < v < b$ with $f(u) = f(v)$. -Some facts about this picture can be proven from the definition of concavity: - - -> The slope of the tangent line at $u$ goes up slower than the slope of the tangent line at $v$ declines: $f'(u) < -f'(v)$. - - - -Since $f'$ is *strictly* concave, we have for any $a The slope of the tangent line at $u$ goes up slower than the slope of the tangent line at $v$ declines: $f'(u) < -f'(v)$. + + + +Since $f'$ is *strictly* concave, we have by integrating both sides of the concavity inequality: $$ -\begin{align*} -\int_0^1 (tf'(u) + (1-t)f'(v)) dt &< \int_0^1 f'(tu + (1-t)v) dt, \text{or}\\ -\frac{f'(u) + f'(v)}{2} &< \frac{1}{v-u}\int_u^v f'(w) dw, -\end{align*} +\int_0^1 (tf'(u) + (1-t)f'(v)) dt < \int_0^1 f'(tu + (1-t)v) dt. +$$ + +Integrating the left-hand side and using the substitution $w = tu + (1-t)v$ on the right yields: + +$$ +\frac{f'(u) + f'(v)}{2} < \frac{1}{v-u}\int_u^v f'(w) dw. $$ -by the substitution $w = tu + (1-t)v$. Using the fundamental theorem of calculus to compute the mean value of the integral of $f'$ over $[u,v]$ gives the following as a consequence of strict concavity of $f'$: +Using the fundamental theorem of calculus to compute the integral of $f'$ over $[u,v]$ gives the following as a consequence of strict concavity of $f'$: $$ @@ -768,7 +817,7 @@ $$ $$ -Letting $h=f(u \rightarrow c)$ we get the *inequality* +Letting $u \rightarrow c$ we get the *inequality*: $$ @@ -798,14 +847,22 @@ This comes from solving the projectile motion equations with a drag force *propo ```{julia} #| hold: true @syms gₑ::positive, k::positive, v₀::positive, θ::positive, x::positive -ex = (gₑ/(k*v₀*cos(θ)) + tan(θ))*x + gₑ/k^2 * log(1 - k/(v₀*cos(θ))*x) -diff(ex, x, x), diff(ex, x, x, x,) +ex = (gₑ/(k*v₀*cos(θ)) + tan(θ))*x + gₑ/k^2 * log(1 - k*x/(v₀*cos(θ))) +diff(ex, x, x) ``` -Both the second and third derivatives are negative (as $0 \leq x < (v_0\cos(\theta))/k$ due to the logarithm term), so, both $f$ and $f'$ are strictly concave down. Hence the results above apply. That is the arrow will fly further as it goes up, than as it comes down and will carve out more area on its way up, than its way down. The trajectory could also show time versus height, and the same would hold, e.g, the arrow would take longer to go up than come down. +So the second derivative is negative. Furthermore: + +```{julia} +diff(ex, x, x, x) +``` + +But the logarithm term in `ex` ensures $1 - k\cdot x / (v_0\cos(\theta)) > 0$, so $f'''$ is also negative and $f'$ is strictly concave down. Hence the results above apply. + +That is the arrow will fly further as it goes up, than as it comes down and will carve out more area on its way up, than its way down. The trajectory could also show time versus height, and the same would hold, e.g, the arrow would take longer to go up than come down. -In general, the drag force need not be proportional to the velocity, but merely in opposite direction to the velocity vector $\langle x'(t), y'(t) \rangle$: +In general, Kantorwitz and Neumann note that the drag force need not be proportional to the velocity---merely in opposite direction to the velocity vector $\langle x'(t), y'(t) \rangle$: $$ diff --git a/quarto/integrals/area.qmd b/quarto/integrals/area.qmd index de22c4b..1995287 100644 --- a/quarto/integrals/area.qmd +++ b/quarto/integrals/area.qmd @@ -8,72 +8,105 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia -using Plots -plotly() -using QuadGK +using Plots; plotly() using Roots ``` +```{julia} +#| echo: false +# for problems +using QuadGK +nothing +``` + --- -![A jigsaw puzzle needs a certain amount of area to complete. For a traditional rectangular puzzle, this area is comprised of the sum of the areas for each piece. Decomposing a total area into the sum of smaller, known, ones---even if only approximate---is the basis of definite integration.](figures/jigsaw.png) +::: {#fig-jigsaw-puzzle} +![](figures/jigsaw.png) + +When completed, a jigsaw puzzle fills a certain amount of area. For a traditional rectangular puzzle this area can be computed using base times height, but could also be found by adding the areas of each piece. Decomposing a total area into the sum of smaller areas---even if only approximate---is the basis of definite integration. +::: The question of area has long fascinated human culture. As children, we learn early on the formulas for the areas of some geometric figures: a square is $b^2$, a rectangle $b\cdot h$, a triangle $1/2 \cdot b \cdot h$ and for a circle, $\pi r^2$. The area of a rectangle is often the intuitive basis for illustrating multiplication. The area of a triangle has been known for ages. Even complicated expressions, such as [Heron's](http://tinyurl.com/mqm9z) formula which relates the area of a triangle with measurements from its perimeter have been around for 2000 years. The formula for the area of a circle is also quite old. Wikipedia dates it as far back as the [Rhind](http://en.wikipedia.org/wiki/Rhind_Mathematical_Papyrus) papyrus for 1700 BC, with the approximation of $256/81$ for $\pi$. -The modern approach to area begins with a non-negative function $f(x)$ over an interval $[a,b]$. The goal is to compute the area under the graph. That is, the area between $f(x)$ and the $x$-axis between $a \leq x \leq b$. +The calculus approach to computing areas begins with a non-negative function $f(x)$ over an interval $[a,b]$. The goal is to compute the area under the graph of $f(x)$. That is, the area between $f(x)$ and the $x$-axis for $a \leq x \leq b$. -For some functions, this area can be computed by geometry, for example, here we see the area under $f(x)$ is just $1$, as it is a triangle with base $2$ and height $1$: - - -```{julia} -#| hold: true -f(x) = 1 - abs(x) -plot(f, -1, 1) -plot!(zero) -``` - -Similarly, we know this area is also $1$, it being a square: - - -```{julia} -#| hold: true -f(x) = 1 -plot(f, 0, 1) -plot!(zero) -``` - -This one, is simply $\pi/2$, it being half a circle of radius $1$: - - -```{julia} -#| hold: true -f(x) = sqrt(1 - x^2) -plot(f, -1, 1) -plot!(zero) -``` - -And this area can be broken into a sum of the area of square and the area of a triangle, or $1 + 1/2$: - - -```{julia} -#| hold: true -f(x) = x > 1 ? 2 - x : 1.0 -plot(f, 0, 2) -plot!(zero) -``` - +For some functions this area can be computed by familiar geometry. Examples are shown in @fig-easy-to-compute-areas. But what of more complicated areas? Can these have their area computed? +::: {#fig-easy-to-compute-areas layout-ncol=2} +```{julia} +#| echo: false +gr(); +``` + +```{julia} +#| echo: false +let + f(x) = 1 - abs(x) + plot(; legend=false, xlims=(-5/4, 5/4)) + plot!(f; line=(1, :black)) + plot!(zero; line=(1, :black)) + plot!(f, -1, 1; line=(5, :black, 0.25)) + plot!(zero, -1, 1; line=(5, :black, 0.25)) +end +``` + +```{julia} +#| hold: true +#| echo: false +let + f(x) = 1 + plot(; legend=false, xlims=(-1/4, 5/4)) + plot!(f; line=(1, :black)) + plot!(zero; line=(1, :black)) + plot!([(0,0), (1,0), (1,1), (0,1), (0,0)]; line=(5, :black, 0.25)) +end +``` + +```{julia} +#| hold: true +#| echo: false +let + f(x) = sqrt(1 - x^2) + plot(; legend=false, xlims=(-5/4, 5/4), aspect_ratio=:equal) + plot!(f, -1, 1; line=(1,:black)) + plot!(f, -1, 1; line=(5,:black, 0.25)) + plot!(zero; line=(1, :black)) + plot!(zero, -1, 1; line=(5, :black, 0.25)) +end +``` + +```{julia} +#| hold: true +#| echo: false +let + f(x) = x > 1 ? 2 - x : 1.0 + plot(; legend=false, xlims=(-1/4, 2 + 1/4), aspect_ratio=:equal) + plot!(f; line=(1, :black)) + plot!(f, 0, 2; line=(5, :black, 0.25)) + plot!(zero; line=(1, :black)) + plot!(zero, 0, 2; line=(5, :black, 0.25)) + plot!([(0,0), (0, 1)]; line=(5, :black, 0.25)) +end +``` + +```{julia} +#| echo: false +plotly(); +``` + +Example of areas under functions over an interval that can readily be computed. The upper left shows the area under $1 - \lvert x \rvert$ over $[-1,1]$ (a triangle); the upper right shows the area under $f(x) = 1$ over $[0,1]$ (a square); the lower left shows the area under $\sqrt{1 - x^2}$ over $[-1,1]$ (a half circle); and the lower right graph shows an area comprised of a square and a triangle. +::: + ## Approximating areas -In a previous section, we saw this animation: - - +::: {#fig-archimedes-parabola-take-2} ```{julia} #| hold: true #| echo: false @@ -132,15 +165,16 @@ imgfile = tempname() * ".gif" gif(anim, imgfile, fps = 1) -caption = L""" -The first triangle has area $1/2$, the second has area $1/8$, then $2$ have area $(1/8)^2$, $4$ have area $(1/8)^3$, ... -With some algebra, the total area then should be $1/2 \cdot (1 + (1/4) + (1/4)^2 + \cdots) = 2/3$. -""" +caption = "" plotly() ImageFile(imgfile, caption) ``` -This illustrates a method of [Archimedes](http://en.wikipedia.org/wiki/The_Quadrature_of_the_Parabola) to compute the area contained in a parabola using the method of exhaustion. Archimedes leveraged a fact he discovered relating the areas of triangle inscribed with parabolic segments to create a sum that could be computed. +The first triangle has area $1/2$, the second has area $1/8$, then $2$ have area $(1/8)^2$, $4$ have area $(1/8)^3$, ... With some algebra, the total area then should be $1/2 \cdot (1 + (1/4) + (1/4)^2 + \cdots) = 2/3$. +::: + + +In a previous section, we saw the animation in @fig-archimedes-parabola-take-2. This animation illustrates a method of [Archimedes](http://en.wikipedia.org/wiki/The_Quadrature_of_the_Parabola) to compute the area contained in a parabola using the method of exhaustion. The area below the curve, just a subtraction away. Archimedes leveraged a fact he discovered relating the areas of triangle inscribed with parabolic segments to create a sum that could be computed. The pursuit of computing areas persisted. The method of computing area by finding a square with an equivalent area was known as *quadrature*. Over the years, many figures had their area computed, for example, the area under the graph of the [cycloid](http://en.wikipedia.org/wiki/Cycloid) (...Galileo tried empirically to find this using a tracing on sheet metal and a scale). @@ -149,9 +183,9 @@ The pursuit of computing areas persisted. The method of computing area by findin However, as areas of geometric objects were replaced by the more general question of area related to graphs of functions, a more general study was called for. -One such approach is illustrated in this figure due to Beeckman from 1618 (from [Bressoud](http://www.math.harvard.edu/~knill/teaching/math1a_2011/exhibits/bressoud/)) - +One such approach is illustrated in @fig-beeckman-1618 due to Beeckman from 1618. +::: {#fig-beeckman-1618} ```{julia} #| echo: false imgfile = "figures/beeckman-1618.png" @@ -167,13 +201,15 @@ Riemann sums. nothing ``` -![Figure of Beeckman (1618) showing a means to compute the area under a +![](./figures/beeckman-1618.png) + +Figure of Beeckman (1618) showing a means to compute the area under a curve, in this example the line connecting points $A$ and $B$. Using approximations by geometric figures with known area is the basis of -Riemann sums. -](./figures/beeckman-1618.png) +Riemann sums. (from [Bressoud](http://www.math.harvard.edu/~knill/teaching/math1a_2011/exhibits/bressoud/)) +::: -Beeckman actually did more than find the area. He generalized the relationship of rate $\times$ time $=$ distance. The line was interpreting a velocity, the "squares", then, provided an approximate distance traveled when the velocity is taken as a constant on the small time interval. Then the distance traveled can be approximated by a smaller quantity - just add the area of the rectangles squarely within the desired area ($6+16+6$) - and a larger quantity - by including all rectangles that have a portion of their area within the desired area ($10 + 16 + 10$). Beeckman argued that the error vanishes as the rectangles get smaller. +Beeckman actually did more than find the area. He generalized the relationship of rate $\times$ time $=$ distance. The line was interpreting a velocity, the "squares", then, provided an approximate distance traveled when the velocity is taken as a constant on the small time interval. Then the distance traveled can be approximated by a smaller quantity---just add the area of the squares within the desired area ($6+16+6$)---and a larger quantity---by including all the squares that have a portion of their area within the desired area ($10 + 16 + 10$). Beeckman argued that the error vanishes as the squares get smaller and smaller. Adding up the smaller "squares" can be a bit more efficient if we were to add all those in a row, or column at once. We would then add the areas of a smaller number of rectangles. For this curve, the two approaches are basically identical. For other curves, identifying which squares in a row would be added is much more complicated (though useful), but for a curve generated by a function, identifying which "squares" go in a rectangle is quite easy, in fact we can see the rectangle's area will be a base given by that of the squares, and height depending on the function. @@ -182,19 +218,19 @@ Adding up the smaller "squares" can be a bit more efficient if we were to add al ### Adding rectangles -The idea of the Riemann sum then is to approximate the area under the curve by the area of well-chosen rectangles in such a way that as the bases of the rectangles get smaller (hence adding more rectangles) the error in approximation vanishes. +The idea of the Riemann sum is to approximate the area under the curve of a non-negative function by the area of well-chosen rectangles in such a way that as the bases of the rectangles get smaller (hence adding more rectangles) the error in approximation vanishes. Define a partition of $[a,b]$ to be a selection of points $a = x_0 < x_1 < \cdots < x_{n-1} < x_n = b$. The norm of the partition is the largest of all the differences $\lvert x_i - x_{i-1} \rvert$. For a partition, consider an arbitrary selection of points $c_i$ satisfying $x_{i-1} \leq c_i \leq x_{i}$, $1 \leq i \leq n$. Then the following is a **Riemann sum**: $$ -S_n = f(c_1) \cdot (x_1 - x_0) + f(c_2) \cdot (x_2 - x_1) + \cdots + f(c_n) \cdot (x_n - x_{n-1}). +S_n = f(c_1) \cdot (x_1 - x_0) + f(c_2) \cdot (x_2 - x_1) + \cdots + f(c_n) \cdot (x_n - x_{n-1}). $$ -Clearly for a given partition and choice of $c_i$, the above can be computed. Each term $f(c_i)\cdot(x_i-x_{i-1}) = f(c_i)\Delta_i$ can be visualized as the area of a rectangle with base spanning from $x_{i-1}$ to $x_i$ and height given by the function value at $c_i$. The following visualizes left Riemann sums for different values of $n$ in a way that makes Beekman's intuition plausible – that as the number of rectangles gets larger, the approximate sum will get closer to the actual area. - +Clearly for a given partition and choice of $\{c_i\}$, the above can be computed. Each term $f(c_i)\cdot(x_i-x_{i-1}) = f(c_i)\Delta_i$ can be visualized as the area of a rectangle with base spanning from $x_{i-1}$ to $x_i$ and height given by the function value at $c_i$. @fig-left-riemann-sum-animation visualizes *left* Riemann sums for different values of $n$ in a way that makes Beekman's intuition plausible---that as the number of rectangles gets larger, the approximate sum will get closer to the actual area. +::: {#fig-left-riemann-sum-animation} ```{julia} #| hold: true #| echo: false @@ -262,16 +298,19 @@ end imgfile = tempname() * ".gif" gif(anim, imgfile, fps = 1) -caption = "Illustration of left Riemann sum for increasing ``n`` values" +caption = "" plotly() ImageFile(imgfile, caption) ``` +Illustration of left Riemann sum for increasing ``n`` values +::: + To successfully compute a good approximation for the area, we would need to choose $c_i$ and the partition so that a formula can be found to express the dependence on the size of the partition. -For Archimedes' problem - finding the area under $f(x)=x^2$ between $0$ and $1$ - if we take as a partition $x_i = i/n$ and $c_i = x_i$, then the above sum becomes: +For Archimedes' problem---finding the area under $f(x)=x^2$ between $0$ and $1$---if we take as a partition $x_i = i/n$ and $c_i = x_i$, then the above sum becomes: $$ @@ -297,25 +336,28 @@ The above approach, like Archimedes', ends with a limit being taken. The answer ::: ---- - - There is a more compact notation to $x_1 + x_2 + \cdots + x_n$, this using the *summation notation* or capital sigma. We have: $$ -\Sigma_{i = 1}^n x_i = x_1 + x_2 + \cdots + x_n +\sum_{i = 1}^n x_i = x_1 + x_2 + \cdots + x_n. $$ The notation includes three pieces of information: - * The $\Sigma$ is an indication of a sum - * The ${i=1}$ and $n$ sub- and superscripts indicate the range to sum over. - * The term $x_i$ is a general term describing the $i$th entry, where it is understood that $i$ is just some arbitrary indexing value. +* The $\sum$ is an indication of a sum. + +* The ${i=1}$ and $n$ sub- and superscripts indicate the range to sum over. + +* The term $x_i$ is a general term describing the $i$th entry, where it is understood that $i$ is just some arbitrary indexing value. -With this notation, a Riemann sum can be written as $\Sigma_{i=1}^n f(c_i)(x_i-x_{i-1})$. +With this notation, a Riemann sum can be written as + +$$ +\sum_{i=1}^n f(c_i)(x_i-x_{i-1}). +$$ ### Other sums @@ -324,20 +366,23 @@ With this notation, a Riemann sum can be written as $\Sigma_{i=1}^n f(c_i)(x_i- The choice of the $c_i$ will give different answers for the approximation, though for an integrable function these differences will vanish in the limit. Some common choices are: - * Using the right hand endpoint of the interval $[x_{i-1}, x_i]$ giving the right-Riemann sum, $R_n$. - * The choice $c_i = x_{i-1}$ gives the left-Riemann sum, $L_n$. - * The choice $c_i = (x_i + x_{i-1})/2$ is the midpoint rule, $M_n$. - * If the function is continuous on the closed subinterval $[x_{i-1}, x_i]$, then it will take on its minimum and maximum values. By the extreme value theorem, we could take $c_i$ to correspond to either the maximum or the minimum. These choices give the "upper Riemann-sums" and "lower Riemann-sums". +* Using the right hand endpoint of the interval $[x_{i-1}, x_i]$ giving the right-Riemann sum, $R_n$. + +* The choice $c_i = x_{i-1}$ gives the left-Riemann sum, $L_n$. + +* The choice $c_i = (x_i + x_{i-1})/2$ is the midpoint rule, $M_n$. + +* If the function is continuous on the closed subinterval $[x_{i-1}, x_i]$, then it will take on its minimum and maximum values. By the extreme value theorem, we could take $c_i$ to correspond to either the maximum or the minimum. These choices give the "upper Riemann-sums" and "lower Riemann-sums". When the area is well defined, it must lay between these two values for any given partition. -The choice of partition can also give different answers. A common choice is to break the interval into $n+1$ equal-sized pieces. With $\Delta = (b-a)/n$, these pieces become the arithmetic sequence $a = a + 0 \cdot \Delta < a + 1 \cdot \Delta < a + 2 \cdot \Delta < \cdots < a + n \cdot \Delta = b$ with $x_i = a + i (b-a)/n$. (The `range(a, b, length=n+1)` command will compute these.) An alternate choice made below for one problem is to use a geometric progression: +The choice of partition can also give different answers. A common choice is to break the interval into $n$ equal-sized pieces. With $\Delta = (b-a)/n$, the partition becomes the arithmetic sequence $a = a + 0 \cdot \Delta < a + 1 \cdot \Delta < a + 2 \cdot \Delta < \cdots < a + n \cdot \Delta = b$ with $x_i = a + i (b-a)/n$. (The `range(a, b, length=n+1)` command will compute these.) An alternate choice made below for one problem is to use a geometric progression: $$ a = a(1+\alpha)^0 < a(1+\alpha)^1 < a (1+\alpha)^2 < \cdots < a (1+\alpha)^n = b. $$ -The general statement allows for any partition such that the largest gap goes to $0$. +The general statement allows for any partition provide the largest gap goes to $0$. --- @@ -346,24 +391,22 @@ The general statement allows for any partition such that the largest gap goes to Riemann sums weren't named after Riemann because he was the first to approximate areas using rectangles. Indeed, others had been using even more efficient ways to compute areas for centuries prior to Riemann's work. Rather, Riemann put the definition of the area under the curve on a firm theoretical footing with the following theorem which gives a concrete notion of what functions are integrable: -::: {.callout-note icon=false} -## Riemann Integral +::: {.definition title="Riemann integral"} -A function $f$ is Riemann integrable over the interval $[a,b]$ and its integral will have value $V$ provided for every $\epsilon > 0$ there exists a $\delta > 0$ such that for any partition $a =x_0 < x_1 < \cdots < x_n=b$ with $\lvert x_i - x_{i-1} \rvert < \delta$ and for any choice of points $x_{i-1} \leq c_i \leq x_{i}$ this is satisfied: +A function $f$ is Riemann integrable over the interval $[a,b]$ and its integral will have value $A$ provided for every $\epsilon > 0$ there exists a $\delta > 0$ such that for any partition $a =x_0 < x_1 < \cdots < x_n=b$ with $\lvert x_i - x_{i-1} \rvert < \delta$ and for any choice of points $c_i$ with $x_{i-1} \leq c_i \leq x_{i}$ the following is satisfied: $$ -\lvert \sum_{i=1}^n f(c_i)(x_{i} - x_{i-1}) - V \rvert < \epsilon. +\lvert \sum_{i=1}^n f(c_i)(x_{i} - x_{i-1}) - A \rvert < \epsilon. $$ -When the integral exists, it is written $V = \int_a^b f(x) dx$. - +When the integral exists, it is written $A = \int_a^b f(x) dx$. ::: :::{.callout-note} ## History note -The expression $V = \int_a^b f(x) dx$ is known as the *definite integral* of $f$ over $[a,b]$. Much earlier than Riemann, Cauchy had defined the definite integral in terms of a sum of rectangular products beginning with $S=f(x_0) \cdot (x_1 - x_0) + f(x_1) \cdot (x_2 - x_1) + \cdots + f(x_{n-1}) \cdot (x_n - x_{n-1}) $ (the left Riemann sum). He showed the limit was well defined for any continuous function. Riemann's formulation relaxes the choice of partition and the choice of the $c_i$ so that integrability can be better understood. +The expression $A = \int_a^b f(x) dx$ is known as the *definite integral* of $f$ over $[a,b]$. Much earlier than Riemann, Cauchy had defined the definite integral in terms of a sum of rectangular products beginning with $S=f(x_0) \cdot (x_1 - x_0) + f(x_1) \cdot (x_2 - x_1) + \cdots + f(x_{n-1}) \cdot (x_n - x_{n-1})$ (the left Riemann sum). He showed the limit was well defined for any continuous function. Riemann's formulation relaxes the choice of partition and the choice of the $c_i$ so that integrability can be better understood. ::: @@ -375,10 +418,13 @@ The following formulas are consequences when $f(x)$ is integrable. These mostly The area under a constant function is found from the area of rectangle, a special case being $c=0$ yielding $0$ area: +::: {.relationship title="Area under a constant function"} -> $$ -> \int_a^b c dx = c \cdot (b-a). -> $$ +$$ +\int_a^b c dx = c \cdot (b-a). +$$ + +::: @@ -428,13 +474,14 @@ Illustration that the area under a constant function is that of a rectangle +::: {.relationship title="Area when there is no width"} -The area is $0$ when there is no width to the interval to integrate over: +$$ +\int_a^a f(x) dx = 0. +$$ - -> $$ -> \int_a^a f(x) dx = 0. -> $$ +The area is $0$ when there is no width to the interval to integrate over. +::: @@ -446,22 +493,23 @@ Even our definition of a partition doesn't really apply, as we assume $a < b$, b A jigsaw puzzle piece will have the same area if it is moved around on the table or flipped over. Similarly some shifts preserve area under a function. -The area is invariant under shifts left or right. - - -> $$ -> \int_a^b f(x - c) dx = \int_{a-c}^{b-c} f(x) dx. -> $$ +::: {.relationship title="Area invariant under shifts left or right"} +For integrable $f$: +$$ +\int_a^b f(x - c) dx = \int_{a-c}^{b-c} f(x) dx. +$$ +::: Any partition $a =x_0 < x_1 < \cdots < x_n=b$ is related to a partition of $[a-c, b-c]$ through $a-c < x_0-c < x_1-c < \cdots < x_n - c = b-c$. Let $d_i=c_i-c$ denote this partition, then we have: $$ \begin{align*} -f(c_1 -c) \cdot (x_1 - x_0) &+ f(c_2 -c) \cdot (x_2 - x_1) + \cdots\\ +f(c_1 -c) &\cdot (x_1 - x_0) + f(c_2 -c) \cdot (x_2 - x_1) + \cdots\\ &\quad + f(c_n -c) \cdot (x_n - x_{n-1})\\ - &= f(d_1) \cdot(x_1-c - (x_0-c)) + f(d_2) \cdot(x_2-c - (x_1-c)) + \cdots\\ + &= f(d_1) \cdot(x_1-c - (x_0-c)) \\ + &\quad + f(d_2) \cdot(x_2-c - (x_1-c)) +\cdots \\ &\quad + f(d_n) \cdot(x_n-c - (x_{n-1}-c)). \end{align*} $$ @@ -523,14 +571,16 @@ Illustration that the area under shift remains the same -Similarly, reflections don't effect the area under the curve, they just require a new parameterization: +::: {.relationship title="Area is invariant under reflections"} +For integrable $f$: -> $$ -> \int_a^b f(x) dx = \int_{-b}^{-a} f(-x) dx -> $$ - +$$ +\int_a^b f(x) dx = \int_{-b}^{-a} f(-x) dx +$$ +Under a reflection, trea stays the same if interval is re=parameterized +::: ::: {#fig-consequence-reflect-area} ```{julia} @@ -580,23 +630,29 @@ Illustration that the area remains constant under reflection through $y$ axis. + +::: {.relationship title="Area after reversing interval"} +For integrable $f$: + +$$ +\int_a^b f(x) dx = -\int_b^a f(x) dx. +$$ + The "reversed" area is the same, only accounted for with a minus sign. - - -> $$ -> \int_a^b f(x) dx = -\int_b^a f(x) dx. -> $$ - +::: #### Scaling Scaling the $y$ axis by a constant can be done before or after computing the area: +::: {.relationship title="Area of constant multiple of a function"} +For integrable $f$: +$$ +\int_a^b cf(x) dx = c \int_a^b f(x) dx. +$$ -> $$ -> \int_a^b cf(x) dx = c \int_a^b f(x) dx. -> $$ +::: @@ -606,10 +662,12 @@ Let $a=x_0 < x_1 < \cdots < x_n=b$ be any partition. Then we have $S_n= cf(c_1)( The scaling operation on the $x$ axis, $g(x) = f(cx)$, has the following property: - -> $$ -> \int_a^b f(c\cdot x) dx = \frac{1}{c} \int_{ca}^{cb}f(x) dx -> $$ +::: {.relationship title="Area under scaling operation"} +For integrable $f$: +$$ +\int_a^b f(c\cdot x) dx = \frac{1}{c} \int_{ca}^{cb}f(x) dx +$$ +::: @@ -623,13 +681,17 @@ Combining two operations above, the operation $g(x) = \frac{1}{h}f(\frac{x-c}{h} When two jigsaw pieces interlock their combined area is that of each added. This also applies to areas under functions. + +::: {.relationship title="Area is additive"} + +For $a < c < b$ and integrable $f$: + +$$ +\int_a^b f(x) dx = \int_a^c f(x) dx + \int_c^b f(x) dx. +$$ + The area between $a$ and $b$ can be broken up into the sum of the area between $a$ and $c$ and that between $c$ and $b$. - - -> $$ -> \int_a^b f(x) dx = \int_a^c f(x) dx + \int_c^b f(x) dx. -> $$ - +::: For this, suppose we have a partition for both the integrals on the right hand side for a given $\epsilon/2$ and $\delta$. Combining these into a partition of $[a,b]$ will mean $\delta$ is still the norm. The approximating sum will combine to be no more than $\epsilon/2 + \epsilon/2$, so for a given $\epsilon$, this $\delta$ applies. @@ -685,19 +747,24 @@ Illustration that the area between $a$ and $b$ can be computed as area between $ A consequence of the last few statements is: +::: {.relationship title="Areas under even and odd functions"} -> If $f(x)$ is an even function, then $\int_{-a}^a f(x) dx = 2 \int_0^a f(x) dx$. +If $f(x)$ is an integrable, even function, then $\int_{-a}^a f(x) dx = 2 \int_0^a f(x) dx$. -> If $f(x)$ is an odd function, then $\int_{-a}^a f(x) dx = 0$. +If $f(x)$ is an integrable, odd function, then $\int_{-a}^a f(x) dx = 0$. +::: Additivity works in the $y$ direction as well. -If $f(x)$ and $g(x)$ are two functions then -> $$ -> \int_a^b (f(x) + g(x)) dx = \int_a^b f(x) dx + \int_a^b g(x) dx -> $$ +::: {.relationship title="Area under a sum of functions"} +If $f(x)$ and $g(x)$ are two integrable functions then + +$$ +\int_a^b (f(x) + g(x)) dx = \int_a^b f(x) dx + \int_a^b g(x) dx +$$ +::: For any partitioning with $x_i, x_{i-1}$ and $c_i$ this holds: @@ -712,16 +779,21 @@ This leads to the same statement for the areas under the curves. The *linearity* of the integration operation refers to this combination of the above: -> $$ -> \int_a^b (cf(x) + dg(x)) dx = c\int_a^b f(x) dx + d \int_a^b g(x)dx -> $$ +::: {.relationship title="Linearity of integration"} +for $c, d$ scalars and $f(x)$ and $g(x)$ integrable functions: +$$ +\int_a^b (cf(x) + dg(x)) dx = c\int_a^b f(x) dx + d \int_a^b g(x)dx +$$ +::: -The integral of a shifted function satisfies: -> $$ -> \int_a^b \left(D + C\cdot f(\frac{x - B}{A})\right) dx = D\cdot(b-a) + C \cdot A \int_{\frac{a-B}{A}}^{\frac{b-B}{A}} f(x) dx -> $$ +::: {.relationship title="Integral of shifted function"} +The area of a shift of an integrable function is related to the area of the function. +$$ +\int_a^b \left(D + C\cdot f(\frac{x - B}{A})\right) dx = D\cdot(b-a) + C \cdot A \int_{\frac{a-B}{A}}^{\frac{b-B}{A}} f(x) dx +$$ +::: This follows from a few of the statements above: @@ -736,11 +808,15 @@ $$ #### Inequalities -Area under a non-negative function is non-negative +Some inequalities are always true for definite integrals, as the corresponding areas have obvious bounds. For example, the area under a non-negative function must be non-negative. -> $$ -> \int_a^b f(x) dx \geq 0,\quad\text{when } a < b, \text{ and } f(x) \geq 0 -> $$ +::: {.relationship title="Area under non-negative functions"} +When $a < b$ and $f(x) \geq 0$ and integrable then + +$$ +\int_a^b f(x) dx \geq 0. +$$ +::: Under this assumption, for any partitioning with $x_i, x_{i-1}$ and $c_i$ it holds the $f(c_i)\cdot(x_i - x_{i-1}) \geq 0$. So any sum of non-negative values can only be non-negative, even in the limit. @@ -748,9 +824,14 @@ Under this assumption, for any partitioning with $x_i, x_{i-1}$ and $c_i$ it ho If $g$ bounds $f$ then the area under $g$ will bound the area under $f$. -> $$ -> $\int_a^b f(x) dx \leq \int_a^b g(x) dx \quad\text{when } a < b\text{ and } 0 \leq f(x) \leq g(x) -> $$ +::: {.relationship title="Area when one function dominates another"} + +If $a < b$ and $f(x) \leq g(x)$ on $I=[a,b]$ and both functions are integrable then + +$$ +\int_a^b f(x) dx \leq \int_a^b g(x) dx. +$$ +::: For any partition of $[a,b]$ and choice of $c_i$, we have the term-by-term bound $f(c_i)(x_i-x_{i-1}) \leq g(c_i)(x_i-x_{i-1})$ So any sequence of partitions that converges to the limits will have this inequality maintained for the sum. @@ -803,42 +884,67 @@ nothing Illustration that if $f(x) \le g(x)$ on $[a,b]$ then the integrals share the same property. The excess area is clearly positive. ::: -(This also follows by considering $h(x) = g(x) - f(x) \geq 0$ by assumption, so $\int_a^b h(x) dx \geq 0$.) +(This also follows by considering $h(x) = g(x) - f(x) \geq 0$ by assumption, so $\int_a^b h(x) dx \geq 0$ and the result will follow from linearity.) -For non-negative functions, integrals over larger domains are bigger -> $$ -> \int_a^c f(x) dx \le \int_a^b f(x) dx,\quad\text{when } c < b \text{ and } f(x) \ge 0 -> $$ +::: {.relationship title="Integrals over larger domains"} + +if $a < c < b$ and $f(x)$ is non-negative, then + +$$ +\int_a^c f(x) dx \le \int_a^b f(x) dx. +$$ +::: + +For non-negative functions, integrals over larger domains are bigger. This follows as $\int_c^b f(x) dx$ is non-negative under these assumptions. -This follows as $\int_c^b f(x) dx$ is non-negative under these assumptions. ### Some known integrals Using the definition, we can compute a few definite integrals: +::: {.relationship title="Some integrals that can be directly computed using Riemann sums"} -> $$ -> \int_a^b c dx = c \cdot (b-a). -> $$ +The following integrals can be computed using Riemann sums: + +$$ +\int_a^b c dx = c \cdot (b-a). +$$ + +$$ +\int_a^b x dx = \frac{b^2}{2} - \frac{a^2}{2}. +$$ + +$$ +\int_a^b x^2 dx = \frac{b^3}{3} - \frac{a^3}{3}. +$$ + + +$$ +\int_a^b x^k dx = \frac{b^{k+1}}{k+1} - \frac{a^{k+1}}{k+1},\quad k \neq -1. +$$ + +$$ +\int_a^b x^{-1} dx = \log(b) - \log(a), \quad (0 < a < b). +$$ -> $$ -> \int_a^b x dx = \frac{b^2}{2} - \frac{a^2}{2}. -> $$ +::: +This first is just the area of a trapezoid with heights $a$ and $b$ and side length $b-a$, or $1/2 \cdot (b + a) \cdot (b - a)$. -This is just the area of a trapezoid with heights $a$ and $b$ and side length $b-a$, or $1/2 \cdot (b + a) \cdot (b - a)$. The right sum would be: +For the second, right Riemann sum is: $$ \begin{align*} S &= x_1 \cdot (x_1 - x_0) + x_2 \cdot (x_2 - x_1) + \cdots + x_n \cdot (x_n - x_{n-1}) \\ -&= (a + 1\frac{b-a}{n}) \cdot \frac{b-a}{n} + (a + 2\frac{b-a}{n}) \cdot \frac{b-a}{n} + \cdots + (a + n\frac{b-a}{n}) \cdot \frac{b-a}{n}\\ +&= (a + 1\frac{b-a}{n}) \cdot \frac{b-a}{n} + (a + 2\frac{b-a}{n}) \cdot \frac{b-a}{n} + \cdots \\ +&\quad + (a + n\frac{b-a}{n}) \cdot \frac{b-a}{n}\\ &= n \cdot a \cdot (\frac{b-a}{n}) + (1 + 2 + \cdots + n) \cdot (\frac{b-a}{n})^2 \\ &= n \cdot a \cdot (\frac{b-a}{n}) + \frac{n(n+1)}{2} \cdot (\frac{b-a}{n})^2 \\ & \rightarrow a \cdot(b-a) + \frac{(b-a)^2}{2} \\ @@ -847,24 +953,14 @@ S &= x_1 \cdot (x_1 - x_0) + x_2 \cdot (x_2 - x_1) + \cdots + x_n \cdot (x_n - x $$ -> $$ -> \int_a^b x^2 dx = \frac{b^3}{3} - \frac{a^3}{3}. -> $$ + + +The third is similar to the Archimedes case with $a=0$ and $b=1$ shown above. -This is similar to the Archimedes case with $a=0$ and $b=1$ shown above. - -> $$ -> \int_a^b x^k dx = \frac{b^{k+1}}{k+1} - \frac{a^{k+1}}{k+1},\quad k \neq -1 -> $$ -> -> . - - - -Cauchy showed this using a *geometric series* for the partition, not the arithmetic series $x_i = a + i (b-a)/n$. The series defined by $1 + \alpha = (b/a)^{1/n}$, then $x_i = a \cdot (1 + \alpha)^i$. Here the bases $x_{i+1} - x_i$ simplify to $x_i \cdot \alpha$ and $f(x_i) = (a\cdot(1+\alpha)^i)^k = a^k (1+\alpha)^{ik}$, or $f(x_i)(x_{i+1}-x_i) = a^{k+1}\alpha[(1+\alpha)^{k+1}]^i$, so, using $u=(1+\alpha)^{k+1}=(b/a)^{(k+1)/n}$, $f(x_i) \cdot(x_{i+1} - x_i) = a^{k+1}\alpha u^i$. This gives +Cauchy showed the fourth this using a *geometric series* for the partition, not the arithmetic series $x_i = a + i (b-a)/n$. The series defined by $1 + \alpha = (b/a)^{1/n}$, then $x_i = a \cdot (1 + \alpha)^i$. Here the bases $x_{i+1} - x_i$ simplify to $x_i \cdot \alpha$ and $f(x_i) = (a\cdot(1+\alpha)^i)^k = a^k (1+\alpha)^{ik}$, or $f(x_i)(x_{i+1}-x_i) = a^{k+1}\alpha[(1+\alpha)^{k+1}]^i$, so, using $u=(1+\alpha)^{k+1}=(b/a)^{(k+1)/n}$, $f(x_i) \cdot(x_{i+1} - x_i) = a^{k+1}\alpha u^i$. This gives $$ @@ -878,17 +974,16 @@ S &= a^{k+1}\alpha u^0 + a^{k+1}\alpha u^1 + \cdots + a^{k+1}\alpha u^{n-1}\\ $$ -> $$ -> \int_a^b x^{-1} dx = \log(b) - \log(a), \quad (0 < a < b). -> $$ - - -Again, Cauchy showed this using a geometric series. The expression $f(x_i) \cdot(x_{i+1} - x_i)$ becomes just $\alpha$. So the approximating sum becomes: +Finally, for the last example, Cauchy showed this using a geometric series. The expression $f(x_i) \cdot(x_{i+1} - x_i)$ becomes just $\alpha$. So the approximating sum becomes: $$ -S = f(x_0)(x_1 - x_0) + f(x_1)(x_2 - x_1) + \cdots + f(x_{n-1}) (x_n - x_{n-1}) = \alpha + \alpha + \cdots \alpha = n\alpha. +\begin{align*} +S &= f(x_0)(x_1 - x_0) + f(x_1)(x_2 - x_1) + \cdots + f(x_{n-1}) (x_n - x_{n-1}) \\ +&= \alpha + \alpha + \cdots \alpha\\ +&= n\alpha. +\end{align*} $$ But, letting $x = 1/n$, the limit above is just the limit of @@ -907,19 +1002,20 @@ Certainly other integrals could be computed with various tricks, but we won't pu ### Some other consequences -* The definition is defined in terms of any partition with its norm bounded by $\delta$. If you know a function $f$ is Riemann integrable, then it is enough to consider just a regular partition $x_i = a + i \cdot (b-a)/n$ when forming the sums, as was done above. It is just that showing a limit for just this particular type of partition would not be sufficient to prove Riemann integrability. +* The definition is defined in terms of any partition with its "norm" bounded by $\delta$. If you know a function $f$ is Riemann integrable, then it is enough to consider just a regular partition $x_i = a + i \cdot (b-a)/n$ when forming the sums, as was done above. It is just that showing a limit for just this particular type of partition would not be sufficient to prove Riemann integrability. -* The choice of $c_i$ is arbitrary to allow for maximum flexibility. The Darboux integrals use the maximum and minimum over the subinterval. It is sufficient to prove integrability to show that the limit exists with just these choices. +* The choice of $c_i$ is arbitrary to allow for maximum flexibility. The Darboux integrals use the maximum and minimum over the subinterval. It is sufficient to prove integrability to show that the limit exists with just these two choices. -* Most importantly, +* Most importantly: + +::: {.relationship title="Continuous functions are integrable"} + +A continuous function on $[a,b]$ is Riemann integrable on $[a,b]$. +::: -> A continuous function on $[a,b]$ is Riemann integrable on $[a,b]$. - - - -The main idea behind this is that the difference between the maximum and minimum values over a partition gets small. That is if $[x_{i-1}, x_i]$ is like $1/n$ is length, then the difference between the maximum of $f$ over this interval, $M$, and the minimum, $m$ over this interval will go to zero as $n$ gets big. That $m$ and $M$ exists is due to the extreme value theorem, that this difference goes to $0$ is a consequence of continuity. What is needed is that this value goes to $0$ at the same rate – no matter what interval is being discussed – is a consequence of a notion of uniform continuity, a concept discussed in advanced calculus, but which holds for continuous functions on closed intervals. Armed with this, the Riemann sum for a general partition can be bounded by this difference times $b-a$, which will go to zero. So the upper and lower Riemann sums will converge to the same value. +The main idea is that the difference between the maximum and minimum values over a partition gets small in a controlled manner. In particular, if $\epsilon$ is specified, a $\delta$ can be chosen so that if $\lvert x - y \rvert < \delta$ then $\lvert f(x) - f(y) \rvert < \epsilon/(2(b-a))$ for any pair $x,y$ in $[a,b]$. Now for a partition with maximum gap less than $\delta$ take $x$ and $y$ to be values where $f$ takes its maximum and minimum, respectively. Then the gap between the upper and lower Riemann sums on a partition is less than $\epsilon/(2(b-a)) \cdot (x_i - x_{i-1})$ and summing over all partitions, the upper and lower Riemann sums differ by no more than $\epsilon/2$, hence go to $0$. That fact that we can find such $x$ and $y$ in each partition comes from the extreme value theorem. That we can find a *uniform* bound for $f$, is a consequence of *uniform continuity*, a concept discussed in advanced calculus, but which holds for continuous functions on *closed* intervals. * A "jump", or discontinuity of the first kind, is a value $c$ in $[a,b]$ where $\lim_{x \rightarrow c+} f(x)$ and $\lim_{x \rightarrow c-}f(x)$ both exist, but are not equal. It is true that a function that is not continuous on $I=[a,b]$, but only has discontinuities of the first kind on $I$ will be Riemann integrable on $I$. @@ -931,99 +1027,6 @@ For example, the function $f(x) = 1$ for $x$ in $[0,1]$ and $0$ otherwise will b * Some functions can have infinitely many points of discontinuity and still be integrable. The example of $f(x) = 1/q$ when $x=p/q$ is rational, and $0$ otherwise is often used to illustrate this. -## Numeric integration - - -The Riemann sum approach gives a method to approximate the value of a definite integral. We just compute an approximating sum for a large value of $n$, so large that the limiting value and the approximating sum are close. - - -To see the mechanics, let's again return to Archimedes' problem and compute $\int_0^1 x^2 dx$. - - -Let us fix some values: - - -```{julia} -a, b = 0, 1 -f(x) = x^2 -``` - -Then for a given $n$ we have some steps to do: create the partition, find the $c_i$, multiply the pieces and add up. Here is one way to do all this: - - -```{julia} -n = 5 -xs = a:(b-a)/n:b # also range(a, b, length=n) -deltas = diff(xs) # forms x2-x1, x3-x2, ..., xn-xn-1 -cs = xs[1:end-1] # finds left-hand end points. xs[2:end] would be right-hand ones. -``` - -We want to sum the products $f(c_i)\Delta_i$. Here is one way to do so using `zip` to iterate over the paired off values in `cs` and `deltas`. - - -```{julia} -sum(f(ci)*Δi for (ci, Δi) in zip(cs, deltas)) -``` - -Our answer is not so close to the value of $1/3$, but what did we expect - we only used $n=5$ intervals. Trying again with $50,000$ gives us: - - -```{julia} -#| hold: true -n = 50_000 -xs = a:(b-a)/n:b -deltas = diff(xs) -cs = xs[1:end-1] -sum(f(ci)*Δi for (ci, Δi) in zip(cs, deltas)) -``` - -This value is about $10^{-5}$ off from the actual answer of $1/3$. - - -We should expect that larger values of $n$ will produce better approximate values, as long as numeric issues don't get involved. - - -Before continuing, we define a function to compute Riemann sums for us with an extra argument to specifying one of four methods for computing $c_i$: - - -```{julia} -#| eval: false -function riemann(f, xs; method="right") - Ms = (left = (f,a,b) -> f(a), - right = (f,a,b) -> f(b), - trapezoid = (f,a,b) -> (f(a) + f(b))/2, - simpsons = (f,a,b) -> (c = a/2 + b/2; (1/6) * (f(a) + 4*f(c) + f(b))) - ) - M = Ms[Symbol(method)} - xs′ = zip(xs[1:end-1], xs[2:end]) - sum(M(f, a, b) * (b-a) for (a,b) ∈ xs′) -end - -riemann(f, a, b, n; method="right") = - riemann(f, range(a,b,n+1); method) -``` - -(This function is defined in `CalculusWithJulia` and need not be copied over if that package is loaded.) - - -With this, we can easily find an approximate answer. We wrote the function to use the familiar template `action(function, arguments...)`, so we pass in a function and arguments to describe the problem (`a`, `b`, and `n` and, optionally, the `method`): - - -```{julia} -f(x) = exp(x) -riemann(f, 0, 5, 10) # S_10 -``` - -Or with more intervals in the partition - - -```{julia} -riemann(f, 0, 5, 50_000) -``` - -(The answer is $e^5 - e^0 = 147.4131591025766\dots$, which shows that even $50,000$ partitions is not enough to guarantee many digits of accuracy.) - - ## "Negative" area @@ -1050,25 +1053,9 @@ If we think of the area below the $x$ axis as "signed" area carrying a minus sig ##### Example -Consider a function $g(x)$ defined through its piecewise linear graph: - - -```{julia} -#| echo: false -g(x) = abs(x) > 2 ? 1.0 : abs(x) - 1.0 -plot(g, -3,3; legend=false) -plot!(zero) -``` - - * Compute $\int_{-3}^{-1} g(x) dx$. The area comprised of a square of area $1$ and a triangle with area $1/2$, so should be $3/2$. - * Compute $\int_{-3}^{0} g(x) dx$. In addition to the above, there is a triangle with area $1/2$, but since the function is negative, this area is added in as $-1/2$. In total then we have $1 + 1/2 - 1/2 = 1$ for the answer. - * Compute $\int_{-3}^{1} g(x) dx$: - - -We could add the signed area over $[0,1]$ to the above, but instead see a square of area $1$, a triangle with area $1/2$ and a triangle with signed area $-1$. The total is then $1/2$. - -This figure---using equal sized axes---may make the above decomposition more clear: +Consider a function $g(x)$ defined through its piecewise linear graph in @fig-piecewise-linear-over-minus3-to-3. +::: {#fig-piecewise-linear-over-minus3-to-3} ```{julia} #| echo: false let @@ -1084,13 +1071,17 @@ let end ``` +A piecewise linear graph over $[-3, 3]$ +::: + +* Compute $\int_{-3}^{-1} g(x) dx$. The area comprised of a square of area $1$ and a triangle with area $1/2$, so should be $3/2$. + +* Compute $\int_{-3}^{0} g(x) dx$. In addition to the above, there is a triangle with area $1/2$, but since the function is negative, this area is added in as $-1/2$. In total then we have $1 + 1/2 - 1/2 = 1$ for the answer. + +* Compute $\int_{-3}^{1} g(x) dx$. We could add the signed area over $[0,1]$ to the above, but instead see a square of area $1$, a triangle with area $1/2$ and a triangle with signed area $-1$. The total is then $1/2$. - - * Compute $\int_{-3}^{3} g(x) dx$: - - -We could add the area, but let's use a symmetry trick. This is clearly twice our second answer, or $2$. (This is because $g(x)$ is an even function, as we can tell from the graph.) +* Compute $\int_{-3}^{3} g(x) dx$. We could add the area, but let's use a symmetry trick. This is clearly twice our second answer, or $2$. (This is because $g(x)$ is an even function, as we can tell from the graph.) ##### Example @@ -1105,30 +1096,6 @@ An immediate consequence would be $\int_{-\pi}^\pi \sin(x) = 0$, as would $\int_ ##### Example -Numerically estimate the definite integral $\int_0^2 x\log(x) dx$. (We redefine the function to be $0$ at $0$, so it is continuous.) - - -We have to be a bit careful with the Riemann sum, as the left Riemann sum will have an issue at $0=x_0$ (`0*log(0)`) returns `NaN` which will poison any subsequent arithmetic operations, so the value returned will be `NaN` and not an approximate answer. We could define our function with a check: - - -```{julia} -h(x) = x > 0 ? x * log(x) : 0.0 -``` - -This is actually inefficient, as the check for the size of `x` will slow things down a bit. Since we will call this function 50,000 times, we would like to avoid this, if we can. In this case just using the right sum will work: - - -```{julia} -h(x) = x * log(x) -riemann(h, 0, 2, 50_000, method="right") -``` - -(The default is `"right"`, so no method specified would also work.) - - -##### Example - - Let $j(x) = \sqrt{1 - x^2}$. The area under the curve between $-1$ and $1$ is $\pi/2$. Using a Riemann sum with 4 equal subintervals and the midpoint, estimate $\pi$. How close are you? @@ -1166,10 +1133,12 @@ We have the well-known triangle [inequality](http://en.wikipedia.org/wiki/Triang This suggests that the following inequality holds for integrals: +::: {.relationship title="Triangle inequality for definite integrals"} -> $$ -> \lvert \int_a^b f(x) dx \rvert \leq \int_a^b \lvert f(x) \rvert dx$. -> $$ +$$ +\lvert \int_a^b f(x) dx \rvert \leq \int_a^b \lvert f(x) \rvert dx. +$$ +::: @@ -1183,284 +1152,10 @@ $$ While such bounds are disappointing, often, when looking for specific values, they are very useful when establishing general truths, such as is done with proofs. -## Error estimate - - -The Riemann sum above is actually extremely inefficient. To see how much, we can derive an estimate for the error in approximating the value using an arithmetic progression as the partition. Let's assume that our function $f(x)$ is increasing, so that the right sum gives an upper estimate and the left sum a lower estimate, so the error in the estimate will be between these two values: - - -$$ -\begin{align*} -\text{error} &\leq -\left[ -f(x_1) \cdot (x_{1} - x_0) + f(x_2) \cdot (x_{2} - x_1) + \cdots + f(x_{n-1})(x_{n-1} - x_{n-2}) + f(x_n) \cdot (x_n - x_{n-1})\right]\\ -&- -\left[f(x_0) \cdot (x_{1} - x_0) + f(x_1) \cdot (x_{2} - x_1) + \cdots + f(x_{n-1})(x_n - x_{n-1})\right] \\ -&= \frac{b-a}{n} \cdot (\left[f(x_1) + f(x_2) + \cdots + f(x_n)\right] - \left[f(x_0) + \cdots + f(x_{n-1})\right]) \\ -&= \frac{b-a}{n} \cdot (f(b) - f(a)). -\end{align*} -$$ - - -We see the error goes to $0$ at a rate of $1/n$ with the constant depending on $b-a$ and the function $f$. In general, a similar bound holds when $f$ is not monotonic. - - -There are other ways to approximate the integral that use fewer points in the partition. [Simpson's](http://tinyurl.com/7b9pmu) rule is one, where instead of approximating the area with rectangles that go through some $c_i$ in $[x_{i-1}, x_i]$ instead the function is approximated by the quadratic polynomial going through $x_{i-1}$, $(x_i + x_{i-1})/2$, and $x_i$ and the exact area under that polynomial is used in the approximation. The explicit formula is: - - -$$ -A \approx \frac{b-a}{3n} (f(x_0) + 4 f(x_1) + 2f(x_2) + 4f(x_3) + \cdots + 2f(x_{n-2}) + 4f(x_{n-1}) + f(x_n)). -$$ - -The error in this approximation can be shown to be - - -$$ -\text{error} \leq \frac{(b-a)^5}{180n^4} \text{max}_{\xi \text{ in } [a,b]} \lvert f^{(4)}(\xi) \rvert. -$$ - -That is, the error is like $1/n^4$ with constants depending on the length of the interval, $(b-a)^5$, and the maximum value of the fourth derivative over $[a,b]$. This is significant, the error in $10$ steps of Simpson's rule is on the scale of the error of $10,000$ steps of the Riemann sum for well-behaved functions. - - -:::{.callout-note} -## Note -The Wikipedia article mentions that Kepler used a similar formula $100$ years prior to Simpson, or about $200$ years before Riemann published his work. Again, the value in Riemann's work is not the computation of the answer, but the framework it provides in determining if a function is Riemann integrable or not. - -::: - -## Gauss quadrature - - -The formula for Simpson's rule was the *composite* formula. If just a single rectangle is approximated over $[a,b]$ by a parabola interpolating the points $x_1=a$, $x_2=(a+b)/2$, and $x_3=b$, the formula is: - - -$$ -\frac{b-a}{6}(f(x_1) + 4f(x_2) + f(x_3)). -$$ - -This formula will actually be exact for any 2nd degree polynomial. In fact an entire family of similar approximations using $n$ points can be made exact for any polynomial of degree $n-1$ or lower. But with non-evenly spaced points, even better results can be found. - - -The formulas for an approximation to the integral $\int_{-1}^1 f(x) dx$ discussed so far can be written as: - - -$$ -\begin{align*} -S &= f(x_1) \Delta_1 + f(x_2) \Delta_2 + \cdots + f(x_n) \Delta_n\\ - &= w_1 f(x_1) + w_2 f(x_2) + \cdots + w_n f(x_n)\\ - &= \sum_{i=1}^n w_i f(x_i). -\end{align*} -$$ - - -The $w$s are "weights" and the $x$s are nodes. A [Gaussian](http://en.wikipedia.org/wiki/Gaussian_quadrature) *quadrature rule* is a set of weights and nodes for $i=1, \dots n$ for which the sum is *exact* for any $f$ which is a polynomial of degree $2n-1$ or less. Such choices then also approximate well the integrals of functions which are not polynomials of degree $2n-1$, provided $f$ can be well approximated by a polynomial over $[-1,1]$. (Which is the case for the "nice" functions we encounter.) Some examples are given in the questions. - - -### The quadgk function - - -In `Julia` a modification of the Gauss quadrature rule is implemented in the `quadgk` function (from the `QuadGK` package) to give numeric approximations to integrals. The `quadgk` function also has the familiar interface `action(function, arguments...)`. Unlike our `riemann` function, there is no `n` specified, as the number of steps is *adaptively* determined. (There is more partitioning occurring where the function is changing rapidly.) Instead, the algorithm outputs an estimate on the possible error along with the answer. Instead of $n$, some trickier problems require a specification of an error threshold. - - -To use the function, we have: - - -```{julia} -#| hold: true -f(x) = x * log(x) -quadgk(f, 0, 2) -``` - -As mentioned, there are two values returned: an approximate answer, and an error estimate. In this example we see that the value of $0.3862943610307017$ is accurate to within $10^{-9}$. (The actual answer is $-1 + 2\cdot \log(2)$ and the error is only $10^{-11}$. The reported error is an upper bound, and may be conservative, as with this problem.) Our previous answer using $50,000$ right-Riemann sums was $0.38632208884775737$ and is only accurate to $10^{-5}$. By contrast, this method uses just $256$ function evaluations in the above problem. - - -The method should be exact for polynomial functions: - - -```{julia} -#| hold: true -f(x) = x^5 - x + 1 -quadgk(f, -2, 2) -``` - -The error term is $0$, the answer is $4$ up to the last unit of precision (1 ulp), so any error is only in floating point approximations. - - -For the numeric computation of definite integrals, the `quadgk` function should be used over the Riemann sums or even Simpson's rule. - - -Here are some sample integrals computed with `quadgk`: - - -$$ -\int_0^\pi \sin(x) dx -$$ - -```{julia} -quadgk(sin, 0, pi) -``` - -(Again, the actual answer is off only in the last digit, the error estimate is an upper bound.) - - -$$ -\int_0^2 x^x dx -$$ - -```{julia} -u(x) = x^x -quadgk(u, 0, 2) -``` - -$$ -\int_0^5 e^x dx -$$ - -```{julia} -quadgk(exp, 0, 5) -``` - -When composing the answer with other functions it may be desirable to drop the error in the answer. Two styles can be used for this. The first is to just name the two returned values: - - -```{julia} -#| hold: true -A, err = quadgk(cos, 0, pi/4) -A -``` - -The second is to ask for just the first component of the returned value: - - -```{julia} -#| hold: true -A = quadgk(tan, 0, pi/4)[1] # or first(quadgk(tan, 0, pi/4)) -``` - ---- - - -To visualize the choice of nodes by the algorithm, we have for $f(x)=\sin(x)$ over $[0,\pi]$ relatively few nodes used to get a high-precision estimate: - - -```{julia} -#| echo: false -function FnWrapper(f) - xs=Any[] - ys=Any[] - x -> begin - fx = f(x) - push!(xs, x) - push!(ys, fx) - fx - end -end -nothing -``` - -```{julia} -#| hold: true -#| echo: false -let - a, b= 0, pi - f(x) = sin(x) - F = FnWrapper(f) - ans,err = quadgk(F, a, b) - plot(f, a, b, legend=false, title="Error ≈ $(round(err,sigdigits=2))") - scatter!(F.xs, F.ys) -end -``` - -For a more oscillatory function, more nodes are chosen: - - -```{julia} -#| hold: true -#| echo: false -let - a, b= 0, pi - f(x) = exp(-x)*sinpi(x) - F = FnWrapper(f) - ans,err = quadgk(F, a, b) - plot(f, a, b, legend=false, title="Error ≈ $(round(err,sigdigits=2))") - scatter!(F.xs, F.ys) -end -``` - -##### Example - - -In probability theory, a *univariate density* is a function, $f(x)$ such that $f(x) \geq 0$ and $\int_a^b f(x) dx = 1$, where $a$ and $b$ are the range of the distribution. The [Von Mises](http://en.wikipedia.org/wiki/Von_Mises_distribution) distribution, takes the form - - -$$ -k(x) = C \cdot \exp(\cos(x)), \quad -\pi \leq x \leq \pi -$$ - -Compute $C$ (numerically). - - -The fact that $1 = \int_{-\pi}^\pi C \cdot \exp(\cos(x)) dx = C \int_{-\pi}^\pi \exp(\cos(x)) dx$ implies that $C$ is the reciprocal of - - -```{julia} -k(x) = exp(cos(x)) -A,err = quadgk(k, -pi, pi) -``` - -So - - -```{julia} -C = 1/A -k₁(x) = C * exp(cos(x)) -``` - -The *cumulative distribution function* for $k(x)$ is $K(x) = \int_{-\pi}^x k(u) du$, $-\pi \leq x \leq \pi$. We just showed that $K(\pi) = 1$ and it is trivial that $K(-\pi) = 0$. The quantiles of the distribution are the values $q_1$, $q_2$, and $q_3$ for which $K(q_i) = i/4$. Can we find these? - - -First we define a function, that computes $K(x)$: - - -```{julia} -K(x) = quadgk(k₁, -pi, x)[1] -``` - -(The trailing `[1]` is so only the answer - and not the error - is returned.) - - -The question asks us to solve $K(x) = 0.25$, $K(x) = 0.5$ and $K(x) = 0.75$. The `Roots` package can be used for such work, in particular `find_zero`. We will use a bracketing method, as clearly $K(x)$ is increasing, as $k(u)$ is positive, so we can just bracket our answer with $-\pi$ and $\pi$. (We solve $K(x) - p = 0$, so $K(\pi) - p > 0$ and $K(-\pi)-p < 0$.). We could do this with `[find_zero(x -> K(x) - p, (-pi, pi)) for p in [0.25, 0.5, 0.75]]`, but that is a bit less performant than using the `solve` interface for this task: - - -```{julia} -#| hold: true -Z = ZeroProblem((x,p) -> K(x) - p, (-pi, pi)) -solve.(Z, (1/4, 1/2, 3/4)) -``` - -The middle one is clearly $0$. This distribution is symmetric about $0$, so half the area is to the right of $0$ and half to the left, so clearly when $p=0.5$, $x$ is $0$. The other two show that the area to the left of $-0.809767$ is equal to the area to the right of $0.809767$ and equal to $0.25$. - -##### Example: Gauss nodes - -The `QuadGK.gauss(n)` function returns a pair of $n$ quadrature points and weights to integrate a function over the interval $(-1,1)$, with an option to use a different interval $(a,b)$. For a given $n$, these values exactly integrate any polynomial of degree $2n-1$ or less. The pattern to integrate below can be expressed in other ways, but this is intended to be direct: - - -```{julia} -xs, ws = QuadGK.gauss(5) -``` - -```{julia} -f(x) = exp(cos(x)) -sum(w * f(x) for (x, w) in zip(xs, ws)) -``` - -The `zip` function is used to iterate over the `xs` and `ws` as pairs of values. - - - ## Questions + ###### Question @@ -1542,9 +1237,9 @@ numericq(val) ###### Question -Using geometry, compute the definite integral between $-3$ and $3$ of this graph comprised of lines and circular arcs: - +Using geometry, compute the definite integral between $-3$ and $3$ of the graph in @fig-question-compute-area-lines-circles-minus3-to-3 comprised of lines and circular arcs. +::: {#fig-question-compute-area-lines-circles-minus3-to-3} ```{julia} #| hold: true #| echo: false @@ -1560,6 +1255,9 @@ end plot(f, -3, 3, aspect_ratio=:equal) ``` +Plot of function over $[-3, 3]$ whose definite integral can be readily computed using geometry +::: + The value is: @@ -1573,22 +1271,6 @@ numericq(val) ###### Question -For the function $f(x) = \sin(\pi x)$, estimate the integral for $-1$ to $1$ using a left-Riemann sum with the partition $-1 < -1/2 < 0 < 1/2 < 1$. - - -```{julia} -#| hold: true -#| echo: false -f(x) = sin(pi*x) -xs = -1:1/2:1 -deltas = diff(xs) -val = sum(map(f, xs[1:end-1]) .* deltas) -numericq(val) -``` - -###### Question - - Without doing any *real* work, find this integral: @@ -1707,295 +1389,3 @@ L" $F(x)$ is continuous, so between $a$ and $b$ has an extreme value, which must answ = 1 radioq(choices, answ) ``` - -###### Question - - -For the right Riemann sum approximating $\int_0^{10} e^x dx$ with $n=100$ subintervals, what would be a good estimate for the error? - - -```{julia} -#| hold: true -#| echo: false -choices = [ -"``(10 - 0)/100 \\cdot (e^{10} - e^{0})``", -"``10/100``", -"``(10 - 0) \\cdot e^{10} / 100^4``" -] -answ = 1 -radioq(choices, answ) -``` - -###### Question - - -Use `quadgk` to find the following definite integral: - - -$$ -\int_1^4 x^x dx . -$$ - -```{julia} -#| hold: true -#| echo: false -f(x) = x^x -a, b = 1, 4 -val, _ = quadgk(f, a, b) -numericq(val) -``` - -###### Question - - -Use `quadgk` to find the following definite integral: - - -$$ -\int_0^3 e^{-x^2} dx . -$$ - -```{julia} -#| hold: true -#| echo: false -f(x) = exp(-x^2) -a, b = 0, 3 -val, _ = quadgk(f, a, b) -numericq(val) -``` - -###### Question - - -Use `quadgk` to find the following definite integral: - - -$$ -\int_0^{9/10} \tan(u \frac{\pi}{2}) du. -$$ - -```{julia} -#| hold: true -#| echo: false -f(x) = tan(x*pi/2) -a, b = 0, 9/10 -val, _ = quadgk(f, a, b) -numericq(val) -``` - -###### Question - - -Use `quadgk` to find the following definite integral: - - -$$ -\int_{-1/2}^{1/2} \frac{1}{\sqrt{1 - x^2}} dx -$$ - -```{julia} -#| hold: true -#| echo: false -f(x) = 1/sqrt(1 - x^2) -a, b =-1/2, 1/2 -val, _ = quadgk(f, a, b) -numericq(val) -``` - - -###### Question - -Let $A=1.98$ and $B=1.135$ and - -$$ -f(x) = \frac{1 - e^{-Ax}}{B\sqrt{\pi}x} e^{-x^2}. -$$ - -Find $\int_0^1 f(x) dx$ - -```{julia} -#| echo: false -let - A,B = 1.98, 1.135 - f(x) = (1 - exp(-A*x))*exp(-x^2)/(B*sqrt(pi)*x) - val,_ = quadgk(f, 0, 1) - numericq(val) -end -``` - -###### Question - -A bound for the complementary error function ( positive function) is - -$$ -\text{erfc}(x) \leq \frac{1}{2}e^{-2x^2} + \frac{1}{2}e^{-x^2} \leq e^{-x^2} -\quad x \geq 0. -$$ - -Let $f(x)$ be the first bound, $g(x)$ the second. -Assuming this is true, confirm numerically using `quadgk` that - -$$ -\int_0^3 f(x) dx \leq \int_0^3 g(x) dx -$$ - - -The value of $\int_0^3 f(x) dx$ is - -```{julia} -#| echo: false -let - f(x) = 1/2 * exp(-2x^2) + 1/2 * exp(-x^2) - val,_ = quadgk(f, 0, 3) - numericq(val) -end -``` - -The value of $\int_0^3 g(x) dx$ is - -```{julia} -#| echo: false -let - g(x) = exp(-x^2) - val,_ = quadgk(g, 0, 3) - numericq(val) -end -``` - - - - - -###### Question - - -```{=html} -
-``` - -```{ojs} -//| echo: false -//| output: false -JXG = require("jsxgraph"); - -b = JXG.JSXGraph.initBoard('jsxgraph', { - boundingbox: [-0.5,0.3,1.5,-1/4], axis:true -}); - -g = function(x) { return x*x*x*x + 10*x*x - 60* x + 100} -f = function(x) {return 1/Math.sqrt(g(x))}; - -type = "right"; -l = 0; -r = 1; -rsum = function() { - return JXG.Math.Numerics.riemannsum(f,n.Value(), type, l, r); -}; -n = b.create('slider', [[0.1, -0.05],[0.75,-0.05], [2,1,50]],{name:'n',snapWidth:1}); - -graph = b.create('functiongraph', [f, l, r]); -os = b.create('riemannsum', - [f, - function(){ return n.Value();}, - type, l, r - ], - {fillColor:'#ffff00', fillOpacity:0.3}); - -b.create('text', [0.1,0.25, function(){ - return 'Riemann sum='+(rsum().toFixed(4)); -}]); -``` - -The interactive graphic shows the area of a right-Riemann sum for different partitions. The function is - - -$$ -f(x) = \frac{1}{\sqrt{ x^4 + 10x^2 - 60x + 100}} -$$ - -When $n=5$ what is the area of the Riemann sum? - - -```{julia} -#| hold: true -#| echo: false -numericq(0.1224) -``` - -When $n=50$ what is the area of the Riemann sum? - - -```{julia} -#| hold: true -#| echo: false -numericq(0.1187) -``` - -Using `quadgk` what is the area under the curve? - - -```{julia} -#| hold: true -#| echo: false -g(x) = 1/sqrt(x^4 + 10x^2 - 60x + 100) -val, tmp = quadgk(g, 0, 1) -numericq(val) -``` - -###### Question - - -Gauss nodes for approximating the integral $\int_{-1}^1 f(x) dx$ for $n=4$ are: - - -```{julia} -ns = [-0.861136, -0.339981, 0.339981, 0.861136] -``` - -The corresponding weights are - - -```{julia} -wts = [0.347855, 0.652145, 0.652145, 0.347855] -``` - -Use these to estimate the integral $\int_{-1}^1 \cos(\pi/2 \cdot x)dx$ with $w_1f(x_1) + w_2 f(x_2) + w_3 f(x_3) + w_4 f(x_4)$. - - -```{julia} -#| hold: true -#| echo: false -f(x) = cos(pi/2*x) -val = sum([f(ni)*wi for (wi, ni) in zip(wts, ns)]) -numericq(val) -``` - -The actual answer is $4/\pi$. How far off is the approximation based on 4 points? - - -```{julia} -#| hold: true -#| echo: false -choices = [ -L"around $10^{-1}$", -L"around $10^{-2}$", -L"around $10^{-4}$", -L"around $10^{-6}$", -L"around $10^{-8}$"] -answ = 4 -radioq(choices, answ, keep_order=true) -``` - -###### Question - - -Using the Gauss nodes and weights from the previous question, estimate the integral of $f(x) = e^x$ over $[-1, 1]$. The value is: - - -```{julia} -#| hold: true -#| echo: false -f(x) = exp(x) -val = sum([f(ni)*wi for (wi, ni) in zip(wts, ns)]) -numericq(val) -``` diff --git a/quarto/integrals/area_between_curves.qmd b/quarto/integrals/area_between_curves.qmd index ac7db6a..490b87c 100644 --- a/quarto/integrals/area_between_curves.qmd +++ b/quarto/integrals/area_between_curves.qmd @@ -1,4 +1,4 @@ -# Area between two curves +# Area between curves {{< include ../_common_code.qmd >}} @@ -8,81 +8,31 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia -using Plots -plotly() +using Plots; plotly() using Roots using QuadGK using SymPy ``` ---- +## Area enclosed between two curves -The definite integral gives the "signed" area between the function $f(x)$ and the $x$-axis over $[a,b]$. Conceptually, this is the area between two curves, $f(x)$ and $g(x)=0$. More generally, this integral: +The definite integral gives the "signed" area between the function $f(x)$ and the $x$-axis over $[a,b]$. Conceptually, this is the area between two curves, $f(x)$ and $g(x)=0$. More generally: +::: {.definition title="Area between two curves"} +Suppose $f(x)$ and $g(x)$ are integrable functions on $[a,b]$ and $f(x) \geq g(x)$. Then the area between the curves given by $f(x)$ and $g(x)$ over the interval $[a,b]$ is given by: $$ -\int_a^b (f(x) - g(x)) dx +\int_a^b \left(f(x) - g(x)\right) dx $$ -can be interpreted as the "signed" area between $f(x)$ and $g(x)$ over $[a,b]$. If on this interval $[a,b]$ it is true that $f(x) \geq g(x)$, then this would just be the area, as seen in this figure. The rectangle in the figure has area: $(f(x_i)-g(x_i)) \cdot (x_{i+1}-x_i)$ for some $x_i, x_{i+1}$ suggestive of a term in a left Riemann sum of the integral of $f(x) - g(x)$: + +In general, this integral yields the "signed" area between the two curves. +::: -```{julia} -#| hold: true -#| echo: false -#| label: fig-area-between-f-g-shade -#| fig-cap: "Area between two functions" -f1(x) = x^2 -g1(x) = sqrt(x) -a,b = 1/4, 3/4 - -xs = range(a, stop=b, length=250) -ss = vcat(xs, reverse(xs)) -ts = vcat(f1.(xs), g1.(reverse(xs))) - -plot(f1, 0, 1, legend=false) -plot!(g1) -plot!(ss, ts, fill=(0, :forestgreen, 0.25)) -plot!(xs, f1.(xs), linewidth=5, color=:royalblue) -plot!(xs, g1.(xs), linewidth=5, color=:royalblue) - - -plot!(xs, f1.(xs), legend=false, linewidth=5, color=:blue) -plot!(xs, g1.(xs), linewidth=5, color=:blue) -u,v = .4, .5 -plot!([u,v,v,u,u], [f1(u), f1(u), g1(u), g1(u), f1(u)], color=:black, linewidth=3) -``` - -In @fig-area-between-f-g we have $f(x) = \sqrt{x}$, $g(x)= x^2$ and $[a,b] = [1/4, 3/4]$. The shaded area is then found by: - - -$$ -\int_{1/4}^{3/4} (x^{1/2} - x^2) dx = (\frac{x^{3/2}}{3/2} - \frac{x^3}{3})\big|_{1/4}^{3/4} = \frac{\sqrt{3}}{4} -\frac{7}{32}. -$$ - -#### Examples - -Find the area between - -$$ -\begin{align*} -f(x) &= \frac{x^3 \cdot (2-x)}{2} \text{ and } \\ -g(x) &= e^{x/3} + (1-\frac{x}{1.7})^6 - 0.6 -\end{align*} -$$ - -over the interval $[0.2, 1.7]$. The area is illustrated in the figure below. - -```{julia} -f(x) = x^3*(2-x)/2 -g(x) = exp(x/3) + (1 - (x/1.7))^6 - 0.6 -a, b = 0.2, 1.7 -h(x) = g(x) - f(x) -answer, _ = quadgk(h, a, b) -answer -``` +Consider @fig-area-between-f-g. The shaded rectangle has area: $(f(c)-g(c)) \cdot (x_{i+1}-x_i)$ for some $x_i \le c \le x_{i+1}$. This is suggestive of a term in a Riemann sum of the integral of $f(x) - g(x)$. ::: {#fig-area-between-f-g} @@ -142,27 +92,88 @@ plotly() p ``` -Illustration of a Riemann sum approximation to estimate the area between $f(x)$ and $g(x)$ over an interval $[a,b]$. (Figure follows one by @Angenent.) +Illustration of a Riemann sum approximation to estimate the area between $f(x)$ and $g(x)$ over an interval $[a,b]$. ::: + ##### Example + +Find the area between $f(x) = \sqrt{x}$ and $g(x) = x^2$ over $[1/4, 3/4]$. As $f(x) \geq g(x)$ on this interval, we have: + + +$$ +\int_{1/4}^{3/4} (x^{1/2} - x^2) dx = (\frac{x^{3/2}}{3/2} - \frac{x^3}{3})\Big|_{1/4}^{3/4} = \frac{\sqrt{3}}{4} -\frac{7}{32}. +$$ + + +For the same functions, find the area between the functions over $[1/4, 4]$. + +The two graphs cross at $x=1$, so this would need to be done with two integrals: + +$$ +\begin{align*} +A &= \int_{1/4}^{1} (x^{1/2} - x^2) dx + \int_1^4 (x^2 - x^{1/2}) dx\\ +&= \left(\frac{x^{3/2}}{3/2} - \frac{x^3}{3}\right) \Big|_{1/4}^1 + +\left(\frac{x^3}{3} - \frac{x^{3/2}}{3/2}\right) \Big|_1^4\\ +&= \left(\left[\frac{2}{3} - \frac{1}{3}\right]- + \left[\frac{1}{12} - \frac{1}{192}\right] + + \left[\frac{64}{3} - \frac{16}{3}\right] - + \left[\frac{1}{3} - \frac{2}{3}\right]\right)\\ +&= \frac{49}{192} +\end{align*} +$$ + +##### Example + +Find the area between + +$$ +\begin{align*} +f(x) &= \frac{x^3 \cdot (2-x)}{2} \text{ and } \\ +g(x) &= e^{x/3} + (1-\frac{x}{1.7})^6 - 0.6 +\end{align*} +$$ + +over the interval $[0.2, 1.7]$. The area is illustrated in @fig-area-between-f-g. + +```{julia} +f(x) = x^3*(2-x)/2 +g(x) = exp(x/3) + (1 - (x/1.7))^6 - 0.6 +a, b = 0.2, 1.7 +h(x) = g(x) - f(x) +answer, _ = quadgk(h, a, b) +answer +``` + +##### Example + Find the area bounded by the line $y=2x$ and the curve $y=2 - x^2$. +::: {#fig-plot-parabola-and-line-where-intersect} +```{julia} +#| echo: false +let + f(x) = 2 - x^2 + g(x) = 2x + plot(; legend=false, xlims=(-3,3)) + plot!(f; line=(1, :black)) + plot!(g; line=(1, :black)) + a, b = find_zeros(x -> f(x) - g(x), (-3, 3)) + plot!(f, a, b; line=(5, :black, 0.25)) + plot!(g, a, b; line=(5, :black, 0.25)) +end +``` -We can plot to see the area in question: +Plot of $2-x^2$ and $2x$ over $[-3,3]$ to see the two intersection points. +::: + +We plot both curves in @fig-plot-parabola-and-line-where-intersect to see the two intersection points, $a$ and $b$, happen in $[-3, 3]$. These are found numerically through: + ```{julia} f(x) = 2 - x^2 g(x) = 2x -plot(f, -3,3) -plot!(g) -``` - -For this problem we need to identify $a$ and $b$. These are found numerically through: - - -```{julia} h(x) = f(x) - g(x) a,b = find_zeros(h, -3, 3) ``` @@ -180,20 +191,32 @@ first(quadgk(h, a, b)) Find the integral between $f(x) = \sin(x)$ and $g(x)=\cos(x)$ over $[0,2\pi]$ where $f(x) \geq g(x)$. -A plot shows the areas: - +@fig-plot-sin-cos-over-0-2pi-find-intersection-points shows the areas: +::: {#fig-plot-sin-cos-over-0-2pi-find-intersection-points} ```{julia} -f(x) = sin(x) -g(x) = cos(x) -plot(f, 0, 2pi) -plot!(g) +#| echo: false +let + f(x) = sin(x) + g(x) = cos(x) + plot(f, 0, 2pi; label="sin") + plot!(g; label="cos") + a, b= pi/4, 5pi/4 + plot!(f, a, b; label=nothing, line=(5, :black, 0.25)) + plot!(g, a, b; label=nothing, line=(5, :black, 0.25)) +end ``` +Plot of $\sin(x)$ and $\cos(x)$ over $[0, 2\pi]$ +::: + There is a single interval when $f \geq g$ and this can be found algebraically using basic trigonometry, or numerically: ```{julia} +f(x) = sin(x) +g(x) = cos(x) + a, b = find_zeros(x -> f(x) - g(x), 0, 2pi) # pi/4, 5pi/4 quadgk(x -> f(x) - g(x), a, b)[1] ``` @@ -204,7 +227,7 @@ quadgk(x -> f(x) - g(x), a, b)[1] Find the area between $x^n$ and $x^{n+1}$ over $[0,1]$ for $n=1,2,\dots$. -We have on this interval $x^n \geq x^{n+1}$, so the integral can be found symbolically through: +On this interval we have $x^n \geq x^{n+1}$, so the integral can be found symbolically through: ```{julia} @@ -213,40 +236,42 @@ ex = integrate(x^n - x^(n+1), (x, 0, 1)) together(ex) ``` -Based on this answer, what is the value of this +Based on this answer, what is the value of this sum: $$ -\frac{1}{2\cdot 3} + \frac{1}{3\cdot 4} + \frac{1}{4\cdot 5} + \cdots? +\frac{1}{2\cdot 3} + \frac{1}{3\cdot 4} + \frac{1}{4\cdot 5} + \cdots $$ -This should be no surprise, given how the areas computed carve up the area under the line $y=x^1$ over $[0,1]$, so the answer should be $1/2$. - +The answer is $1/2$. This is no surprise. In @fig-x-to-n-n-1-to-20) the areas computed carve up the area under the line $y=x^1$ over $[0,1]$ +::: {#fig-x-to-n-n-1-to-20} ```{julia} -p = plot(x, 0, 1, legend=false) -[plot!(p, x^n, 0, 1) for n in 2:20] -p +#| echo: false +plt = plot(x, 0, 1; legend=false, line=(1, :black)) +[plot!(plt, x^n; line=(1, :black)) for n in 2:20] +plt ``` +Plot of $x^n$ over $[0,1]$ for $n$ in $1$ to $20$. +::: + + We can check using the `summation` function of `SymPy` which is similar in usage to `integrate`: ```{julia} -summation(1/(n+1)/(n+2), (n, 1, oo)) +summation(1/((n+1) * (n+2)), (n, 1, oo)) ``` ##### Example +Verify [Archimedes'](http://en.wikipedia.org/wiki/The_Quadrature_of_the_Parabola) finding that the area of the parabolic segment is $4/3\text{rds}$ that of the triangle joining $a$, $(a+b)/2$ and $b$. @fig-area-between-f-g clearly shows the bigger parabolic segment area. -Verify [Archimedes'](http://en.wikipedia.org/wiki/The_Quadrature_of_the_Parabola) finding that the area of the parabolic segment is $4/3$rds that of the triangle joining $a$, $(a+b)/2$ and $b$. @fig-area-between-f-g clearly shows the bigger parabolic segment area. - - +::: {#fig-archimedes-triangle} ```{julia} #| hold: true #| echo: false -#| label: fig-archimedes-triangle -#| fig-cap: "Area of parabolic segment and triangle" f(x) = 2 - x^2 a,b = -1, 1/2 c = (a + b)/2 @@ -265,6 +290,9 @@ plot!(triangle, fill=(:forestgreen, 3, 0.25)) ``` +Area of parabolic segment and triangle +::: + For concreteness, let $f(x) = 2-x^2$ and $[a,b] = [-1, 1/2]$, as in the figure. Then the area of the triangle can be computed through: @@ -277,47 +305,60 @@ sac, sab, scb = secant(f, a, 𝐜), secant(f, a, b), secant(f, 𝐜, b) f1(x) = min(sac(x), scb(x)) f2(x) = sab(x) -A1 = quadgk(x -> f1(x) - f2(x), a, b)[1] +A1 = first(quadgk(x -> f1(x) - f2(x), a, b)) ``` -As we needed three secant lines, we used the `secant` function from `CalculusWithJulia` to create functions representing each. Once that was done, we used the `min` function to facilitate integrating over the top bounding curve, alternatively, we could break the integral over $[a,c]$ and $[c,b]$. +As we needed three secant lines, we used the `secant` function from `CalculusWithJulia` to create functions representing each. Once that was done, we used the `min` function to facilitate integrating over the top bounding curve, alternatively, we could break the integral over $[a,c]$ and $[c,b]$, as would be recommended for a symbolic approach. The area of the parabolic segment is more straightforward. ```{julia} -A2 = quadgk(x -> f(x) - f2(x), a, b)[1] +A2 = first(quadgk(x -> f(x) - f2(x), a, b)) ``` -Finally, if Archimedes was right, this relationship should bring about $0$ (or something within round-off error): +Finally, if Archimedes was right, these values should be equal (up to rounding errors) ```{julia} -A1 * 4/3 - A2 +A1 * 4/3 ≈ A2 ``` + ##### Example Find the area bounded by $y=x^4$ and $y=e^x$ when $x^4 \geq e^x$ and $x > 0$. -A graph over $[0,10]$ shows clearly the largest zero, for afterwards the exponential dominates the power. - +@fig-x-to-4-and-exp-x-over-0-10 graphs the two functions and shows clearly the largest zero, for afterwards the exponential dominates the power. +::: {#fig-x-to-4-and-exp-x-over-0-10} ```{julia} -h1(x) = x^4 -h2(x) = exp(x) -plot(h1, 0, 10) -plot!(h2) +#| echo: false +let + h1(x) = x^4 + h2(x) = exp(x) + plot(; xlims=(0, 10), legend=false) + plot!(h1; line=(1, :black)) + plot!(h2; line=(1, :black)) + a,b = find_zeros(x -> h1(x) - h2(x), 0, 10) + plot!(h1, a, b; line=(5, :black, 0.25)) + plot!(h2, a, b; line=(5, :black, 0.25)) +end ``` +Plot of $x^4$ and $e^x$ over $[0, 10]$ +::: + There must be another zero, though it is hard to see from the graph over $[0,10]$, as $0^4=0$ and $e^0=1$, so the polynomial must cross below the exponential to the left of $5$. (Otherwise, plotting over $[0,2]$ will clearly reveal the other zero.) We now find these intersection points numerically and then integrate: ```{julia} #| hold: true +h1(x) = x^4 +h2(x) = exp(x) a,b = find_zeros(x -> h1(x) - h2(x), 0, 10) quadgk(x -> h1(x) - h2(x), a, b)[1] ``` @@ -325,39 +366,54 @@ quadgk(x -> h1(x) - h2(x), a, b)[1] ##### Examples -The area between $y=\sin(x)$ and $y=m\cdot x$ between $0$ and the first positive intersection depends on $m$ (where $0 \leq m \leq 1$. The extremes are when $m=0$, the area is $2$ and when $m=1$ (the line is tangent at $x=0$), the area is $0$. What is it for other values of $m$? The picture for $m=1/2$ is: +The area between $y=\sin(x)$ and $y=m\cdot x$ between $0$ and the first positive intersection depends on $m$ (where $0 \leq m \leq 1$). The extremes are when $m=0$, the area is $2$ and when $m=1$ (the line is tangent at $x=0$), the area is $0$. What is it for other values of $m$? @fig-plot-sinx-and-mx-over-0-pi show the two functions when $m=1/2$. +::: {#fig-plot-sinx-and-mx-over-0-pi} ```{julia} -m = 1/2 -plot(sin, 0, pi) -plot!(x -> m*x) +#| echo: false +let + m = 1/2 + a, b = 0, find_zero(x -> sin(x) - m*x, 2) + plot(; xlims=(0, pi), legend=false) + plot!(sin, 0, pi; line=(1, :black)) + plot!(sin, a, b; line=(5, :black, 0.25)) + plot!(x -> m*x; line=(1, :black)) + plot!(x -> m*x, a, b; line=(5, :black, 0.25)) +end ``` +Plot of $\sin(x)$ and $m\cdot x$ over $[0, \pi]$ when $m=1/2$ +::: + For a given $m$, the area is found after computing $b$, the intersection point. We express this as a function of $m$ for later reuse: ```{julia} intersection_point(m) = maximum(find_zeros(x -> sin(x) - m*x, 0, pi)) a1 = 0 -b1 = intersection_point(m) -quadgk(x -> sin(x) - m*x, a1, b1)[1] +b1 = intersection_point(1/2) +first(quadgk(x -> sin(x) - x/2, a1, b1)) ``` In general, the area then as a function of `m` is found by substituting `intersection_point(m)` for `b`: ```{julia} -area(m) = quadgk(x -> sin(x) - m*x, 0, intersection_point(m))[1] +area(m) = first(quadgk(x -> sin(x) - m*x, 0, intersection_point(m))) ``` -A plot shows the relationship: - +@fig-plot-area-over-0-1-shows-monotonic shows the relationship and that the function is monotonically decreasing, as would be guessed. +::: {#fig-plot-area-over-0-1-shows-monotonic} ```{julia} -plot(area, 0, 1) +#| echo: false +plot(area, 0, 1; label="area") ``` +Plot of `area` over $[0,1]$ +::: + While here, let's also answer the question of which $m$ gives an area of $1$, or one-half the total? This can be done as follows: @@ -375,80 +431,18 @@ In an early 2023 article appearing in the [New York Times](https://www.nytimes.c ::: {#fig-excess-deaths} ![Excess deaths](./figures/excess-deaths.png) -Illustration of excess deaths. Figure from a Feb. 2023 New York Times article +Illustration of excess deaths. Figure from a Feb. 2023 New York Times article. ::: Consider the curve marked *Actual deaths*. The number of deaths per year is the sum over each day of the number of deaths per each day. Approximating this number with a curve and setting 1 day equal to 1 unit, the number of deaths is basically $\int_0^{365} d(t) dt$. This curve is *usually*, say, $u(t)$, so the expected number of deaths would be $\int_0^{365} u(t) dt$. The difference, $\int_0^{365} (d(t) - u(t))dt$ is interpreted as the number of *excess deaths*. This methodology has been used to estimate the true number of deaths attributable to the COVID-19 pandemic. - ##### Example - -Find the area bounded by the $x$ axis, the line $x-1$ and the function $\log(x+1)$. - - -A plot shows us the basic area: - - -```{julia} -j1(x) = log(x+1) -j2(x) = x - 1 -plot(j1, 0, 3) -plot!(j2) -plot!(zero) -``` - -The value for "$b$" is found from the intersection point of $\log(x+1)$ and $x-1$, which is near $2$: - - -```{julia} -ja = 0 -jb = find_zero(x -> j1(x) - j2(x), 2) -``` - -We see that the lower part of the area has a condition: if $x < 1$ then use $0$, otherwise use $g(x)$. We can handle this many different ways: - - - * break the integral into two pieces and add: - - -```{julia} -quadgk(x -> j1(x) - zero(x), ja, 1)[1] + quadgk(x -> j1(x) - j2(x), 1, jb)[1] -``` - - * make a new function for the bottom bound: - - -```{julia} -j3(x) = x < 1 ? 0.0 : j2(x) -quadgk(x -> j1(x) - j3(x), ja, jb)[1] -``` - - * Turn the picture on its side and integrate in the $y$ variable. To do this, we need to solve for inverse functions: - - -```{julia} -#| hold: true -a1=j1(ja) -b1=j1(jb) -f1(y)=y+1 # y=x-1, so x=y+1 -g1(y)=exp(y)-1 # y=log(x+1) so e^y = x + 1, x = e^y - 1 -quadgk(y -> f1(y) - g1(y), a1, b1)[1] -``` - -:::{.callout-note} -## Note -When doing problems by hand this latter style can often reduce the complications, but when approaching the task numerically, the first two styles are generally easier, though computationally more expensive. - -::: - -##### Example - -Consider two overlapping circles, one with smaller radius. How much area is in the larger circle that is not in the smaller? The question came up on the `Julia` [discourse](https://discourse.julialang.org/t/is-there-package-or-method-to-calculate-certain-area-in-julia-symbolically-with-sympy/99751) discussion board. A solution, modified from an answer of `@rocco_sprmnt21`, follows. +Consider two overlapping circles, one with smaller radius. How much area is in the larger circle that is not in the smaller?^[This question came up on the `Julia` [discourse](https://discourse.julialang.org/t/is-there-package-or-method-to-calculate-certain-area-in-julia-symbolically-with-sympy/99751) discussion board. A solution, modified from an answer of `@rocco_sprmnt21` is followed.] Without losing too-much generality, we can consider the smaller circle to have radius $a$, the larger circle to have radius $b$ and centered at $(0,c)$. -We assume some overlap---$a \ge c-b$, but not too much---$c-b \ge 0$ or $0 \le c-b \le a$. +We assume some overlap: $a \ge c-b$, but not too much: $c-b \ge 0$ or $0 \le c-b \le a$. ```{julia} @syms x::real y::real a::positive b::positive c::positive @@ -460,7 +454,9 @@ x₀ = sqrt(a - y₀^2) # point of intersection Plotting with $a=1, b=3/2, c=2$ we have: +::: {#fig-two-overlapping-circles-plotted} ```{julia} +#| echo: false let 𝑎 = 1, 𝑏=3/2, 𝑐=2 @assert 0 ≤ 𝑐 - 𝑏 ≤ 𝑎 @@ -482,9 +478,9 @@ let 𝑎 = 1, 𝑏=3/2, 𝑐=2 scatter!([-x, x], [y, y]) plot!([0, 𝑏], [𝑐, 𝑐]; linestyle=:dash) - annotate!([(0, 𝑐, text("(0, c)", 8, :left)), - (𝑏, 𝑐, text("(b, c)", 8, :left)), - (x, y, text("(x₀, y₀)", 8, :left))]) + annotate!([(0, 𝑐, text(L"(0, c)", 10, :left)), + (𝑏, 𝑐, text(L"(b, c)", 10, :left)), + (x, y, text(L"(x_0, y_0)", 10, :left))]) plot!([0, 0], [𝑎, 𝑐 + 𝑏]; color=:green) plot!([x, x], [y, 𝑐 + sqrt(𝑏^2 - x^2)]; color=:green) @@ -492,6 +488,9 @@ let 𝑎 = 1, 𝑏=3/2, 𝑐=2 end ``` +Plot of two overlapping circles showing a decomposition that might make finding the area tractable +::: + With this orientation, we can see by symmetry that the area is twice the integral from $[0,x_0]$ and from $[x_0, b]$ **provided** $0 \le c- b \le a$: ```{julia} @@ -513,6 +512,84 @@ A(c=>3, a=>1, b=>2) ``` +(This could also be done by computing the amount of overlap the two circles have---with a single integral---and subtracting from the area of the larger circle.) + +##### Example + + +Find the area bounded by the $x$ axis, the line $x-1$ and the function $\log(1 + x)$. + + +@fig-log1p-and-x-minus-1-over-0-3 shows the basic area. + +::: {#fig-log1p-and-x-minus-1-over-0-3} +```{julia} +#| echo: false +let + j1(x) = log(x+1) + j2(x) = x - 1 + plot(; legend=false, xlims=(0,3)) + plot!(j1; line=(1, :black)) + plot!(j2; line=(1, :black)) + plot!(zero; line=(1, :black)) + c = find_zero(x -> j1(x) - j2(x), 2) + P,Q,R = (0,0), (1,0), (c, j1(c)) + plot!([P,Q,R]; line=(5, :black, 0.25)) + plot!(j1, 0, c; line=(5, :black, 0.25)) + annotate!([(c, j1(c), text(L"(b, \log(1 + b))", :top,:left))]) +end +``` + +Plot of $\log(1 + x)$, $x-1$, and the $x$ axis over $[0,3]$ used to identify the location of the $x$ value ($b$) of the largest intersection point +::: + +The value for "$b$" is found from the intersection point of $\log(x+1)$ and $x-1$, which is near $2$: + + +```{julia} +j1(x) = log(1 + x) +j2(x) = x - 1 +ja = 0 +jb = find_zero(x -> j1(x) - j2(x), 2) +``` + +We see that the lower part of the area has this condition: if $x < 1$ then use $0$, otherwise use $g(x)$. We can handle this many different ways: + + +* break the integral into two pieces and add: + + +```{julia} +first(quadgk(x -> j1(x) - zero(x), ja, 1)) + first(quadgk(x -> j1(x) - j2(x), 1, jb)) +``` + +* make a new function for the bottom bound: + + +```{julia} +j3(x) = x < 1 ? 0.0 : j2(x) +first(quadgk(x -> j1(x) - j3(x), ja, jb)) +``` + +* Turn the picture on its side and integrate in the $y$ variable. To do this, we need to solve for inverse functions: + + +```{julia} +#| hold: true +a1=j1(ja) +b1=j1(jb) +f1(y)=y+1 # y=x-1, so x=y+1 +g1(y)=exp(y)-1 # y=log(x+1) so e^y = x + 1, x = e^y - 1 +quadgk(y -> f1(y) - g1(y), a1, b1)[1] +``` + +:::{.callout-note} +## Note +When doing problems by hand this latter style can often reduce the complications, but when approaching the task numerically, the first two styles are generally easier, though computationally more expensive. + +::: + + #### Integrating in different directions @@ -524,40 +601,41 @@ It has been noted that different symmetries can aid in computing integrals throu Another symmetry of the $x-y$ plane is the reflection through the line $y=x$. This has the effect of taking the graph of $f(x)$ to the graph of $f^{-1}(x)$ and vice versa. Here is an example with $f(x) = x^3$ over $[-1,1]$. - +::: {#fig-plot-inverse-function-of-x-cubed} ```{julia} -#| hold: true f(x) = x^3 xs = range(-1, stop=1, length=50) ys = f.(xs) plot(ys, xs) ``` +Plot of inverse function of $x^3$ +::: + By switching the order of the `xs` and `ys` we "flip" the graph through the line $x=y$. We can use this symmetry to our advantage. Suppose instead of being given an equation $y=f(x)$, we are given it in "inverse" style: $x = f(y)$, for example suppose we have $x = y^3$. We can plot this as above via: - +::: {#fig-plot-function-in-inverse-style-y-cubed} ```{julia} #| hold: true ys = range(-1, stop=1, length=50) xs = [y^3 for y in ys] plot(xs, ys) ``` +Plot of function given in inverse style +::: -Suppose we wanted the area in the first quadrant between this graph, the $y$ axis and the line $y=1$. What to do? With the problem "flipped" through the $y=x$ line, this would just be $\int_0^1 x^3dx$. Rather than mentally flipping the picture to integrate, instead we can just integrate in the $y$ variable. That is, the area is $\int_0^1 y^3 dy$. The mental picture for Riemann sums would be have the approximating rectangles laying flat and as a function of $y$, are given a length of $y^3$ and height of "$dy$". - - ---- - +Suppose we wanted the area in the first quadrant between this graph, the $y$ axis and the line $y=1$. What to do? With the problem "flipped" through the $y=x$ line, this would just be $\int_0^1 x^3dx$. Rather than mentally flipping the picture to integrate, instead we can just integrate in the $y$ variable. That is, the area is $\int_0^1 y^3 dy$. The mental picture for Riemann sums would be have the approximating rectangles laying flat and as a function of $y$, are given a length of $y^3$ and height of "$dy$" (cf. @fig-integrate-in-x-or-in-y-it-is-all-just-area). +::: {#fig-integrate-in-x-or-in-y-it-is-all-just-area} ```{julia} #| hold: true #| echo: false f(x) = x^(1/3) f⁻¹(x) = x^3 -plot(f, 0, 1, label="f", linewidth=5, color=:blue, aspect_ratio=:equal) +plot(f, 0, 1; label="f", linewidth=5, color=:blue, aspect_ratio=:equal) plot!([0,1,1],[0,0,1], linewidth=1, linestyle=:dash, label="") x₀ = 2/3 Δ = 1/16 @@ -575,26 +653,36 @@ box(f⁻¹(x₀-1Δ), x₀-2Δ, 1 - f⁻¹(x₀-1Δ), Δ, colᵣ) box(f⁻¹(x₀-2Δ), x₀-3Δ, 1 - f⁻¹(x₀-2Δ), Δ, colᵣ) ``` -The figure above suggests that the area under $f(x)$ over $[a,b]$ could be represented as the area between the curves $f^{-1}(y)$ and $x=b$ from $[f(a), f(b)]$. +The figure suggests that the area under $f(x)$ over $[a,b]$ could be represented as the area between the curves $f^{-1}(y)$ and $x=b$ from $[f(a), f(b)]$ +::: --- -For a less trivial problem, consider the area between $x = y^2$ and $x = 2-y$ in the first quadrant. - +For a less trivial problem, consider the area between $x = y^2$ and $x = 2-y$ in the first quadrant shown in @fig-plot-x-y-squared-x-2-minus-y. +::: {#fig-plot-x-y-squared-x-2-minus-y} ```{julia} -#| hold: true -ys = range(0, stop=2, length=50) -xs = [y^2 for y in ys] -plot(xs, ys) -xs = [2-y for y in ys] -plot!(xs, ys) -plot!(zero) +#| echo: false +let + ys = range(0, stop=2, length=50) + xs = [y^2 for y in ys] + plot(xs, ys; label=L"x = y^2") + xs = [2-y for y in ys] + plot!(xs, ys; label=L"x = 2-y") + plot!(zero; label=nothing) + + P,Q,R = (0,0), (2,0), (1, 1) + plot!([P,Q,R]; label=nothing, line=(5, :black, 0.25)) + plot!(sqrt, 0, 1; label=nothing, line=(5, :black, 0.25)) +end ``` -We see the bounded area could be described in the "$x$" variable in terms of two integrals, but in the $y$ variable in terms of the difference of two functions with the limits of integration running from $y=0$ to $y=1$. So, this area may be found as follows: +Plot of $x=y^2$ and $x = 2-y$ +::: + +In the figure, the bounded area could be described in the "$x$" variable in terms of two integrals, but in the $y$ variable in terms of the difference of two functions with the limits of integration running from $y=0$ to $y=1$. So, this area may be found as follows: ```{julia} @@ -607,43 +695,229 @@ quadgk(y -> f(y) - g(y), a, b)[1] ## The area enclosed in a simple polygon -A simple polygon is comprised of several non-intersecting line segments, save for the last segment ends where the first begins. These have an orientation, which we take to be counterclockwise. Polygons, as was seen when computing areas related to Archimedes efforts, can be partitioned into simple geometric shapes, for which known areas apply. +We now turn our attention to a variation of the area bounded between two curves. +A simple polygon is comprised of several *non-intersecting* line segments, save for the last segment ends where the first begins. These have an orientation, which we take to be counterclockwise. Polygons, as was seen when computing areas related to Archimedes efforts, can be partitioned into simple geometric shapes, for which known areas apply. In this section we discuss partitions that allow the enclosed area to be computed. + + +### The triangle formula + +First, let's work out a formula for the area of a triangle formed by three points: $O=(0,0)$, $P=(x_1, y_1)$, and $Q = (x_2, y_2)$ where we assume the angles for $P$ and $Q$ are as in @fig-area-of-triangle-3-points (the angle $0 < \theta < \pi$). + +::: {#fig-area-of-triangle-3-points} +```{julia} +#| echo: false +let + gr() + function plot_arc!(plt, r, θ₁, θ₂, txt, i=50) + ts = range(θ₁, θ₂, 100) + xs, ys = r * cos.(ts), r * sin.(ts) + plot!(plt, xs, ys; line=(1, :dot, :black), arrow=:right) + annotate!(plt, (xs[i], ys[i], text(txt, :left, :bottom))) + end + + O = (0, 0) + x1, y1 = P = (2, 2) + x2, y2 = Q = (1, 3) + plt = plot(; legend=false, aspect_ratio=:equal, framestyle=:orgin) + plot!(plt, [O,P,Q,O]; line=(1, :black)) + scatter!(plt, [O, P, Q]; marker=(5, :black)) + annotate!(plt, [ + (O..., text(L"O", :right, :bottom)), + (P..., text(L"P", :left, :top)), + (Q..., text(L"Q", :right, :bottom))]) + θ₁, θ₂ = atan(y1, x1), atan(y2, x2) + plot_arc!(plt, 0.4, 0, θ₁, L"\theta_1", 1) + plot_arc!(plt, 0.6, 0, θ₂, L"\theta_2", 80) + plot_arc!(plt, 0.9, θ₁, θ₂, L"\theta", 50) + plotly() + plt +end +``` + +Triangle $\triangle OPQ$ with $\theta_2 > \theta_1$ +::: + +The area of a triangle is $1/2 \cdot b \cdot h$. The base of the triangle in @fig-area-of-triangle-3-points is the length of $OP$, the height is the length of $OQ$ times $\sin(\theta)$. We can then compute in terms of coordinates: + +$$ +\begin{align*} +A &= \frac{1}{2} b \cdot h\\ +&= \frac{1}{2} \lvert \overline{PQ} \rvert \cdot \lvert\overline{OQ}\rvert \sin(\theta) \\ +&= \frac{1}{2} \lvert \overline{PQ} \rvert \cdot \lvert\overline{OQ}\rvert \sin(\theta_2 - \theta_1) \\ +&= \frac{1}{2} \lvert \overline{PQ} \rvert \cdot \lvert\overline{OQ}\rvert \left(\sin(\theta_2)\cos(\theta_1) - \cos(\theta_2)\sin(\theta_1)\right) \\ +&= \frac{1}{2} \lvert\overline{PQ}\rvert \cdot \lvert\overline{OQ}\rvert +\left(\frac{y_2}{\lvert\overline{OQ}\rvert}\frac{x_1}{\lvert\overline{PQ}\rvert} - \frac{x_2}{\lvert\overline{OQ}\rvert}\frac{y_1}{\lvert\overline{PQ}\rvert}\right)\\ +&= \frac{1}{2} \left(x_1 y_2 - y_1 x_2 \right). +\end{align*} +$$ + +The expression $x_1 y_2 - y_1 x_2$ is sometimes written using *determinant* notation + +$$ +x_1 y_2 - y_1 x_2 = +\left| +\begin{align*} +x_1 &\quad x_2\\ +y_1 &\quad y_2\\ +\end{align*} +\right|. +$$ + +Now consider a *simple*, convex polygon with $(0,0)$ in its interior. We can create triangles to partition the polygon as in @fig-simple-case-for-triangle-formula-to-compute-area-simple-polygon. + +::: {#fig-simple-case-for-triangle-formula-to-compute-area-simple-polygon} +```{julia} +#| echo: false +let + xs = [-1, 1, 2, 0, -1] + ys = [-1, -1, 1/2, 1, -1] + + gr() + S = Shape(xs, ys) + plt = plot(; legend=false, aspect_ratio=:equal, framestyle=:origin) + plot!(plt, xs, ys) + scatter!(plt, xs, ys; marker=(7, :circle)) + for i in 1:4 + col = xs[i]*ys[i+1] - xs[i+1]*ys[i] > 0 ? :yellow : :blue + S = Shape([(0,0), (xs[i],ys[i]), (xs[i+1],ys[i+1])]) + plot!(plt, S, fill=(col, 0.25)) + end + plotly() + plt +end +``` + +Partition of simple example using triangles to compute the area inside a simple polygon +::: + +The triangles do not overlap, so the area inside the polygon is the sum of the areas of each triangle. This is found from: + +$$ +\frac{1}{2} \cdot \left( +(x_1 y_2 - y_1 x_2) + (x_2 y_3 - y_2 x_3) + (x_3 y_4 - y_3 x_4) + (x_4 y_1 - y_4 x_1) +\right) +$$ + + +For the points in @fig-simple-case-for-triangle-formula-to-compute-area-simple-polygon this value is: + +```{julia} +xs = [-1, 1, 2, 0, -1] +ys = [-1, -1, 1/2, 1, -1] + +(1/2) * sum(xs[i]*ys[i+1] - xs[i+1]*ys[i] for i in 1:4) +``` + +More generally, we have: + +::: {.definition title="Area of a simple polygon described by Cartesian coordinates"} +The area enclosed in a simple polygon with $n$ points (labeled by $P_i = (x_i, y_i)$) traversed in a counter-clockwise manner is given by: + +$$ +A = \sum_{i=1}^{n-1} \left(x_i y_{i+1} - y_i x_{i+1}\right) + \left( x_n y_1 - y_n x_1\right). +$$ +::: + +This formula is motivated above by a convex polygon with the origin in its interior, but extends to any simple polygon. +This formula is often called Gauss's shoelace formula, so called as diagrams like @fig-gauss-shoelace-formula, which are used to help remember what gets multiplied and what terms are subtracted, resemble the laces of shoes. + +::: {#fig-gauss-shoelace-formula} +```{julia} +#| echo: false +let + gr() + function plot_circles!(plt, i, l1, l2, col=:blue) + S = Plots.scale(Shape(:circle), 0.3) + for j in 0:1 + S′ = Plots.translate(S, i, j) + plot!(plt, S′, fill=(0, 0.0), line=(2, col)) + annotate!([(i,1,text(l1)), (i,0, text(l2))]) + end + end + function criss_cross!(plt, i) + ts = range(0,1, 100)[25:75] + plot!(plt, i .+ ts, 1 .- ts; line=(3, :blue), arrow=true) + plot!(plt, i .+ ts, ts; line=(3, :red), arrow=true) + end + plt = plot(; legend=false, aspect_ratio=:equal, xaxis=([], false), + yaxis=([], false)) + plot_circles!(plt, 0, L"x_1", L"y_1") + plot_circles!(plt, 1, L"x_2", L"y_2") + plot_circles!(plt, 2, L"x_3", L"y_3") + plot_circles!(plt, 3, L"x_4", L"y_4") + plot_circles!(plt, 4, L"x_1", L"y_1", :green) + for i in 0:3 + criss_cross!(plt, i) + end + plotly() + plt +end +``` + +Representation to remember Gauss's shoelace formula for computing the area of a simple polygon +::: + +The above formula computes the area for simple polygon described by Cartesian coordinates and traversed in a counter-clockwise manner. Traversing in a clockwise manner will produce the same number multiplied by $-1$. This formula extends to compute the area of any simple polygon traversed in a counter-clockwise manner. For non-simple polygons, the formula can be applied with care to account for "negative" area when the traversal is in the clockwise manner. ### The trapezoid formula -In this example, we see how trapezoids can be used to find the interior area encolosed by a simply polygon, avoiding integration. +In this example, we see how trapezoids can be used to find the interior area encolosed by a simple polygon, avoiding integration. -The trapezoid formula to compute the area of a simple polygon is +Consider a trapezoid formed by points $P_1 = (x_1, y_1)$, $P_2 = (x_2, y_2)$, $P_3 = (x_2, 0)$, and $P_4 = (x_1, 0)$. Assume $x_1 > x_2$. The area is the average of the heights times the length of the base: $$ -A = - \sum_{i=1}^n \frac{y_{i+1} + y_i}{2} \cdot (x_{i+1} - x_i). +A = \frac{1}{2}\left(y_2 + y_1\right) \cdot (x_1 - x_2) = +- \frac{1}{2}\left(y_2 + y_1\right) \cdot (x_2 - x_1). $$ -Where the polygon is described by points $(x_1,y_1), (x_2,y_2), \cdots, (x_n, y_n), (x_{n+1}, y_{n+1})$ *with* $(x_1,y_1) = (x_{n+1}, y_{n+1})$. +::: {.definition title="Area inside a simple polygon"} -Each term describes the area of a trapezoid, possibly signed. +Consider a simple polygon described by points $(x_1,y_1), (x_2,y_2), \dots, (x_n, y_n)$ Set $(x_{n+1}, y_{n+1}) = (x_1,y_1)$. Assume the points traverse the simple polygon in a counter-clockwise manner. Then the area contained within the polygon is given by the formula: -This figure illustrates for a simple case: +$$ +A = \sum_{i=1}^n \frac{1}{2} \left(y_{i+1} + y_i\right) \cdot \left(x_{i+1} - x_i \right). +$$ +Each term describes the *signed* area of a trapezoid, with a sign of $1$ if $x_{i+1} \le x_i$ and $-1$ otherwise. +::: + +We illustrate the formula with a simple polygon shown in @fig-simple-case-for-trapezoid-formula-to-compute-area-simple-polygon. (The same shape as before, but shifted up and over by $2$ units.) + +::: {#fig-simple-case-for-trapezoid-formula-to-compute-area-simple-polygon} ```{julia} -xs = [1, 3, 4, 2, 1] # n = 4 to give 5=n+1 values -ys = [1, 1, 2, 3, 1] -p = plot(xs, ys; line=(3, :black), ylims=(0,4), legend=false) -scatter!(p, xs, ys; marker=(7, :circle)) +xs = [1, 3, 4, 2, 1] # n = 4 to give 5=n+1 values +ys = [1, 1, 5/2, 3, 1] +plt = plot(; line=(3, :black), framestyle=:origin, aspect_ratio=:equal, legend=false) + +plot!(plt, xs, ys) +scatter!(plt, xs, ys; marker=(5, :black)) ``` -Going further, we draw the four trapezoids using different colors depending on the sign of the `xs[i+1] - xs[i]` terms: +A simple polygon whose area can be computed by the trapezoid formula +::: + +Going further, we draw the four trapezoids associated to @fig-simple-case-for-trapezoid-formula-to-compute-area-simple-polygon using different colors (blue or yellow) depending on the sign of the `xs[i+1] - xs[i]` terms in @fig-simple-case-for-trapezoid-formula-to-compute-area-simple-polygon-with-colors. + +::: {#fig-simple-case-for-trapezoid-formula-to-compute-area-simple-polygon-with-colors} ```{julia} +#| echo: false for i in 1:4 col = xs[i+1] - xs[i] > 0 ? :yellow : :blue S = Shape([(xs[i],0), (xs[i+1],0), (xs[i+1],ys[i+1]), (xs[i], ys[i])]) - plot!(p, S, fill=(col, 0.25)) + plot!(plt, S, fill=(col, 0.25)) end -p +P1, P2, P3, P4, _ = zip(xs, ys) +annotate!(plt, [(P1..., text(L"P_1", :right, :top)), + (P2..., text(L"P_2", :left, :top)), + (P3..., text(L"P_3", :left, :bottom)), + (P4..., text(L"P_4", :right, :bottom))]) +plt ``` +Simple polygon of @fig-simple-case-for-trapezoid-formula-to-compute-area-simple-polygon with trapezoids used to compute area filled in +::: -The yellow trapezoids appear to be colored grey, as they completely overlap with parts of the blue trapezoids and blue and yellow make grey with lights. As the signs of the differences of the $x$ values is different, these areas add to $0$ in the sum, leaving just the area of the interior when the sum is computed. +The yellow trapezoids appear to be colored grey, as they completely overlap with parts of the blue trapezoids and blue and yellow display as grey. As the signs of the differences of the $x$ values are different, these areas add to $0$ in the sum, leaving just the area of the interior when the sum is computed. For this particular figure, the enclosed area is @@ -651,31 +925,87 @@ For this particular figure, the enclosed area is - sum((ys[i+1] + ys[i]) / 2 * (xs[i+1] - xs[i]) for i in 1:length(xs)-1) ``` -### The triangle formula +### Area between a parameterized curve -Similarly, we can create triangles to partition the polygon. The *signed* area of a triangle with vertices $(0,0), (x_i, y_i), (x_{i+1}, y_{i+1})$ can be computed by $\frac{1}{2} \cdot (x_i \cdot y_{i+1} - x_{i+1}\cdot y_i)$. (A formula that can be derived from a related one for the area of a parallelogram. +Now suppose a closed, simple curve is parameterized in the counterclockwise direction by $(f(t), g(t))$ for $t$ in $[a,b]$. The area contained in the curve is *approximated* by the area contained in the polygon given by the points $(f(t_i), g(t_i))$ where $a = t_0 < t_1 < \cdots < t_n = b$ is some partition. The approximate area is given by: -Visualizing, as before, we have the shape and the triangles after centering around the origin: +$$ +\begin{align*} +A &= - \sum_{i=1}^n \frac{y_{i+1} + y_i}{2} \cdot (x_{i+1} - x_i)\\ +&= -\frac{1}{2} \sum_{i=1}^n \left(g(t_{i+1}) + g(t_i)\right)\cdot\left(f(t_{i+1}) - f(t_i)\right)\\ +&= -\frac{1}{2} \sum_{i=1}^n \left(g(t_{i+1}) + g(t_i)\right)\frac{f(t_{i+1}) - f(t_i)}{t_{i+1} - t_i}\left(t_{i+1} - t_i\right)\\ +&\approx -\frac{1}{2} \int_a^b 2g(t) f'(t) dt\\ +&= -\int_a^b g(t) f'(t) dt \\ +&= \int_a^b f(t) g'(t) dt. +\end{align*} +$$ + +The last line follows by integration by parts. + +::: {.definition title="Area of bounded by a parameterized curve"} +Let $(f(t), g(t))$, $a \le t \leg b$ parameterize a closed, simple curve. Then the area bounded by the curve is given by: + +$$ +A = \sigma \cdot \int_a^b f(t) g'(t) dt, +$$ + +where $\sigma = 1$ if the parameterization is in the counter-clockwise direction, and $\sigma = -1$ if the parameterization is in the clockwise direction. +::: + +##### Example + +The area of a circle is well known, $\pi r^2$. Here we compute it with the above formula using the following parameterization of the perimeter: ```{julia} -S = Shape(xs, ys) -c = Plots.center(S) # find centroid of the polygon -xs, ys = xs .- c[1], ys .- c[2] - -p = plot(xs, ys; line=(3, :black), legend=false) -scatter!(p, xs, ys; marker=(7, :circle)) -for i in 1:4 - col = xs[i]*ys[i+1] - xs[i+1]*ys[i] > 0 ? :yellow : :blue - S = Shape([(0,0), (xs[i],ys[i]), (xs[i+1],ys[i+1])]) - plot!(p, S, fill=(col, 0.25)) -end -p +@syms t, R +u = R * cos(t) +v = R * sin(t) +integrate(u * diff(v,t), (t, 0, 2PI)) ``` -Here the triangles are all yellow, as each has a positive area to contribute to the following sum: +##### Example + +[Talbot's curve](https://mathshistory.st-andrews.ac.uk/Curves/Talbots/) is parametrically described by + +$$ +\begin{align*} +u(t) &= \frac{1}{a} \cdot (a^2 + c^2 \cdot \sin(t)^2) \cdot \cos(t)\\ +v(t) &= \frac{1}{b} \cdot (a^2 - 2c^2 + c^2 \cdot \sin(t)^2)\cdot \sin(t) +\end{align*} +$$ + +Find the area bounded by the curve when $a=2$, $b=1$, and $c=3$. + +@fig-talbots-curve-a-2-b-1-c-3 shows the curve with the given set of constants. + +::: {#fig-talbots-curve-a-2-b-1-c-3} +```{julia} +#| echo: false +let + @syms a b c t + u = (a^2 + c^2*sin(t)^2)*cos(t)/a + v = (a^2 - 2c^2 + c^2*sin(t)^2)*sin(t)/b + d = (a=>2, b=>1, c=>3) + plot(u(d...), v(d...), 0, 2pi; legend=false, line=(1, :black)) +end +``` +Talbot's curve with $a=2$, $b=1$, and $c=3$ +::: + + +We can integrate in terms of the parameters. *However*, this parameterization is in the clockwise direction, so we reverse our limits of integration below: ```{julia} -(1/2) * sum(xs[i]*ys[i+1] - xs[i+1]*ys[i] for i in 1:4) +@syms a b c t +u = (a^2 + c^2*sin(t)^2)*cos(t)/a +v = (a^2 - 2c^2 + c^2*sin(t)^2)*sin(t)/b +A = integrate(u * diff(v, t), (t, 2PI, 0)) # reverse limits to adjust for parameterization +``` + +Finally, we substitute in the specific parameter values: + +```{julia} +A(a=>2, b=>1, c=>3) ``` @@ -861,8 +1191,8 @@ Is the guess that the entire sculpture is more than two tons? #| hold: true #| echo: false choices=["Less than two tons", "More than two tons"] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` :::{.callout-note} @@ -909,9 +1239,9 @@ Consider the following code which sets up the area of an inscribed triangle, `A1 @syms x::real A::real B::real C::real a::real b::real c = (a + b) / 2 f(x) = A*x^2 + B*x + C -Secant(f, a, b) = f(a) + (f(b)-f(a))/(b-a) * (x - a) -A1 = integrate(Secant(f, a, c) - Secant(f,a,b), (x,a,c)) + integrate(Secant(f,c,b)-Secant(f,a,b), (x, c, b)) -A2 = integrate(f(x) - Secant(f,a,b), (x, a, b)) +Sec(f, a, b) = f(a) + (f(b)-f(a))/(b-a) * (x - a) +A1 = integrate(Sec(f, a, c) - Sec(f,a,b), (x,a,c)) + integrate(Sec(f,c,b) - Sec(f,a,b), (x, c, b)) +A2 = integrate(f(x) - Sec(f,a,b), (x, a, b)) out = 4//3 * A1 - A2 ``` @@ -940,8 +1270,11 @@ Figure from Martin showing the companion curve to the cycloid. As the generatin # ImageFile(:integrals, imgfile, caption) nothing ``` +::: {#fig-marting-figure-for-cycloid} +![](./figures/cycloid-companion-curve.png) -![Figure from Martin showing the companion curve to the cycloid. As the generating circle rolls, from ``A`` to ``C``, the original point of contact, ``D``, traces out an arch of the cycloid. The companion curve is that found by congruent line segments. In the figure, when ``D`` was at point ``P`` the line segment ``PQ`` is congruent to ``EF`` (on the original position of the generating circle).](./figures/cycloid-companion-curve.png) +Figure from Martin showing the companion curve to the cycloid. As the generating circle rolls, from ``A`` to ``C``, the original point of contact, ``D``, traces out an arch of the cycloid. The companion curve is that found by congruent line segments. In the figure, when ``D`` was at point ``P`` the line segment ``PQ`` is congruent to ``EF`` (on the original position of the generating circle). +::: In particular, it can be read that Roberval proved that the area between the cycloid and its companion curve is half the area of the generating circle. Roberval didn't know integration, so finding the area between two curves required other tricks. One is called "Cavalieri's principle." From the figure above, which of the following would you guess this principle to be: @@ -1050,3 +1383,83 @@ choices = ["The two enclosed areas should be equal", "The two enclosed areas are clearly different, as they do not overap"] radioq(choices, 1) ``` + +##### Question + +Consider the polygon defined by the points $P_1 = (0,0), P_2 = (1, 1/2), P_3 = (2,2)$, and $P_4=(1,3)$. What is the enclosed area? + +```{julia} +#| echo: false +xs = [0, 1, 2, 1] +ys = [0, 1.2, 2, 3] +push!(xs, first(xs)); push!(ys, first(ys)) + +answer = (1/2) * sum(xs[i]*ys[i+1] - xs[i+1]*ys[i] for i in 1:4) +numericq(answer) +``` + +##### Question + +Consider the Polygon defined by the points $P_1 =(0,0), P_2 = (2, 0), P_3 = (0,1), P_4=(2,2)$, and $P_5=(0,2)$. What is the enclosed area? + + +```{julia} +#| echo: false +xs = [0, 2, 0, 2, 0] +ys = [0,0,1,2,2] +push!(xs, first(xs)); push!(ys, first(ys)) + +answer = (1/2) * sum(xs[i]*ys[i+1] - xs[i+1]*ys[i] for i in 1:4) +numericq(answer) +``` + +Now move $P_3$ to $(4,1)$. Will the enclosed area by $4$ units more? + +```{julia} +#| echo: false +xs = [0, 2, 0, 2, 0] +ys = [0,0,1,2,2] +push!(xs, first(xs)); push!(ys, first(ys)) +a1 = (1/2) * sum(xs[i]*ys[i+1] - xs[i+1]*ys[i] for i in 1:4) +xs[3] = 4 +a2 = (1/2) * sum(xs[i]*ys[i+1] - xs[i+1]*ys[i] for i in 1:4) +val = a2 - a1 ≈ 4 +radioq(["No", "Yes"], 1 + val) +``` + +##### Question + +A [folium](https://mathshistory.st-andrews.ac.uk/Curves/Folium/) can be parameterized by +$(f(t), g(t)) = (r(t)\cdot \cos(t), r(t) \cdot \sin(t)), where $r(t) = -\cos(t) + 4 \cdot \cos(t) \cdot \sin^2(t)$. + +Compute the area between $a=$`0.5236$ and $b=\pi/2$ (cf. @fig-folium-with-some-parameters). + +```{julia} +#| echo: false +a, b = 0.5236, pi/2 +r(t) = -cos(t) + 4 * cos(t) * sin(t)^2 +f(t) = r(t) * cos(t) +g(t) = r(t) * sin(t) +val, _ = quadgk(f ∘ g', a, b) +numericq(val) +``` + +::: {#fig-folium-with-some-parameters} +```{julia} +#| echo: false +let + gr() + r(t) = -cos(t) + 4 * cos(t) * sin(t)^2 + f(t) = r(t) * cos(t) + g(t) = r(t) * sin(t) + + plt = plot(; legend=false) + plot!(plt, f, g, 0, 2pi; line=(1, :black)) + a, b = 0.5236, pi/2 + plot!(plt, f, g, a, b; line=(5, :blue)) + plotly() + plt +end +``` +Plot of a folium---a parameterized curve---over $[0, 2\pi]$ with interval $[$`0.5236`$,\pi/2]$ emphasized +::: diff --git a/quarto/integrals/center_of_mass.qmd b/quarto/integrals/center_of_mass.qmd index b19f409..9a1d058 100644 --- a/quarto/integrals/center_of_mass.qmd +++ b/quarto/integrals/center_of_mass.qmd @@ -8,18 +8,17 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia -using Plots -plotly() +using Plots; plotly() using Roots using QuadGK using SymPy - ``` --- +::: {#fig-seesaw-image-for-center-of-mass} ```{julia} #| hold: true #| echo: false @@ -37,15 +36,18 @@ distance, the balance will tip in favor of the heavier. nothing ``` -![A silhouette of two children on a seesaw. The seesaw can be balanced +![ +](./figures/seesaw.png) + +A silhouette of two children on a seesaw. The seesaw can be balanced only if the distance from the central point for each child reflects their relative weights, or masses, through the formula $d_1m_1 = d_2 m_2$. This means if the two children weigh the same the balance will tip in favor of the child farther away, and if both are the same distance, the balance will tip in favor of the heavier. -](./figures/seesaw.png) +::: -The game of seesaw is one where children earn an early appreciation for the effects of distance and relative weight. For children with equal weights, the seesaw will balance if they sit an equal distance from the center (on opposite sides, of course). However, with unequal weights that isn't the case. If one child weighs twice as much, the other must sit twice as far. +@fig-seesaw-image-for-center-of-mass shows the game of seesaw. One where children earn an early appreciation for the effects of distance and relative weight. For children with equal weights, the seesaw will balance if they sit an equal distance from the center (on opposite sides, of course). However, with unequal weights that isn't the case. If one child weighs twice as much, the other must sit twice as far. The key relationship is that $d_1 m_1 = d_2 m_2$. This come from physics, where the moment about a point is defined by the mass times the distance. This balance relationship says the overall moment balances out. When this is the case, then the *center of mass* is at the fulcrum point, so there is no impetus to move. @@ -58,7 +60,7 @@ In general, we use position of the mass, rather than use distance from some fixe $$ -\bar{\text{cm}} = \frac{m_1 x_1 + m_2 x_2 + \cdots + m_n x_n}{m_1 + m_2 + \cdots + m_n}. +\overline{\text{cm}} = \frac{m_1 x_1 + m_2 x_2 + \cdots + m_n x_n}{m_1 + m_2 + \cdots + m_n}. $$ Writing $w_i = m_i / (m_1 + m_2 + \cdots + m_n)$, we get the center of mass is just a weighted sum: $w_1 x_1 + \cdots + w_n x_n$, where the $w_i$ are the relative weights. @@ -68,10 +70,10 @@ With some rearrangement, we can see that the center of mass satisfies the equati $$ -w_1 \cdot (x_1 - \bar{\text{cm}}) + w_2 \cdot (x_2 - \bar{\text{cm}}) + \cdots + w_n \cdot (x_n - \bar{\text{cm}}) = 0. +w_1 \cdot (x_1 - \overline{\text{cm}}) + w_2 \cdot (x_2 - \overline{\text{cm}}) + \cdots + w_n \cdot (x_n - \overline{\text{cm}}) = 0. $$ -The center of mass is a balance of the weighted signed distances. This property of the center of mass being a balancing point makes it of intrinsic interest and can be - in the case of sufficient symmetry - easy to find. +The center of mass is a balance of the weighted signed distances. This property of the center of mass being a balancing point makes it of intrinsic interest and can be---in the case of sufficient symmetry---easy to find. ##### Example @@ -80,7 +82,7 @@ The center of mass is a balance of the weighted signed distances. This property A set of weights sits on a dumbbell rack. They are spaced 1 foot apart starting with the 5, then the 10-, 15-, 25-, and 35-pound weights. Where is the center of mass? -We begin by letting $m_1=5$, $m_2=10$, $m_3=15$, $m_4=25$ and $m_5=35$. Our positions will be labeled $x_i = i-1$, so the five-pound weight is at position $0$ and the $35$-pound one at $4$. The center of mass is then given by: +We begin by letting $m_1=5$, $m_2=10$, $m_3=15$, $m_4=25$ and $m_5=35.$ Our positions will be labeled $x_i = i-1$, so the five-pound weight is at position $0$ and the $35$-pound one at $4$. The center of mass is then given by: $$ @@ -106,17 +108,21 @@ The center of mass shifts slightly, but since the removed weight was already clo Consider now a more general problem, the center of mass of a solid figure. We will restrict our attention to figures that can be represented by functions in the $x-y$ plane which are two dimensional. For example, consider the region in the plane bounded by the $x$ axis and the function $1 - \lvert x \rvert$. This is triangle with vertices $(-1,0)$, $(0,1)$, and $(1,0)$. -This graph shows that the figure is symmetric: - +@fig-graph-1-absx-over-minus-3-over-2-to-3-over-2 shows that the graph is symmetric: +::: {#fig-graph-1-absx-over-minus-3-over-2-to-3-over-2} ```{julia} #| hold: true +#| echo: false f(x) = 1 - abs(x) a, b = -1.5, 1.5 -plot(f, a, b) +plot(f, a, b; legend=false) plot!(zero, a, b) ``` +Plot of symmetric function $1 - \lvert x \rvert$ over $[-3/2, 3/2]$ +::: + As the center of mass should be a balancing value, we would guess intuitively that the center of mass in the $x$ direction will be $x=0$. @@ -125,7 +131,7 @@ But what should the center of mass formula be? As with many formulas that will end up involving a derived integral, we start with a sum approximation. If the region is described as the area under the graph of $f(x)$ between $a$ and $b$, then we can form a Riemann sum approximation, that is a choice of $a = x_0 < x_1 < x_2 \cdots < x_n = b$ and points $c_1$, $\dots$, $c_n$. If all the rectangles are made up of a material of uniform density, say $\rho$, then the mass of each rectangle will be the area times $\rho$, or $\rho f(c_i) \cdot (x_i - x_{i-1})$, for $i = 1, \dots , n$. - +::: {#fig-center-of-mass-of-1-minus-absx} ```{julia} #| hold: true #| echo: false @@ -152,34 +158,38 @@ plot!(p, [-1,1], [0,0]) p ``` -The figure shows the approximating rectangles and circles representing their masses for $n=20$. +Approximating rectangles with circles representing their masses for a equal sized partition of $[-3/2, 3/2$ with $n=20$ +::: -Generalizing from this figure shows the center of mass for such an approximation will be: +Generalizing from @fig-center-of-mass-of-1-minus-absx shows the center of mass for such an approximation will be: $$ \begin{align*} &\frac{\rho f(c_1) (x_1 - x_0) \cdot x_1 + \rho f(c_2) (x_2 - x_1) \cdot x_1 + \cdots + \rho f(c_n) (x_n- x_{n-1}) \cdot x_{n-1}}{\rho f(c_1) (x_1 - x_0) + \rho f(c_2) (x_2 - x_1) + \cdots + \rho f(c_n) (x_n- x_{n-1})} \\ -&=\\ -&\quad\frac{f(c_1) (x_1 - x_0) \cdot x_1 + f(c_2) (x_2 - x_1) \cdot x_1 + \cdots + f(c_n) (x_n- x_{n-1}) \cdot x_{n-1}}{f(c_1) (x_1 - x_0) + f(c_2) (x_2 - x_1) + \cdots + f(c_n) (x_n- x_{n-1})}. +&= +\frac{f(c_1) (x_1 - x_0) \cdot x_1 + f(c_2) (x_2 - x_1) \cdot x_1 + \cdots + f(c_n) (x_n- x_{n-1}) \cdot x_{n-1}}{f(c_1) (x_1 - x_0) + f(c_2) (x_2 - x_1) + \cdots + f(c_n) (x_n- x_{n-1})}. \end{align*} $$ But the top part is an approximation to the integral $\int_a^b x f(x) dx$ and the bottom part the integral $\int_a^b f(x) dx$. The ratio of these defines the center of mass. -::: {.callout-note icon=false} -## Center of Mass +::: {.definition title="Center of mass"} The center of mass (in the $x$ direction) of a region in the $x-y$ plane described by the area under a (positive) function $f(x)$ between $a$ and $b$ is given by $$ \text{Center of mass} = -\text{cm}_x = \frac{\int_a^b xf(x) dx}{\int_a^b f(x) dx}. +\overline{\text{cm}}_x = \frac{\int_a^b xf(x) dx}{\int_a^b f(x) dx}. $$ -For regions described by a more complicated set of equations, the center of mass is found from the same formula where $f(x)$ is the total height in the $x$ direction for a given $x$. +For a region bounded between $g(x) \le f(x)$ over $[a,b]$ the center of mass is given by: + +$$ +\overline{\text{cm}}_x = \frac{\int_a^b x(f(x)-g(x)) dx}{\int_a^b (f(x)-g(x)) dx}. +$$ ::: @@ -189,51 +199,63 @@ For the triangular shape, we have by the fact that $f(x) = 1 - \lvert x \rvert$ ##### Example -What about the center of mass of the triangle formed by the line $x=-1$, the $x$ axis and $(1-x)/2$? This too is defined between $a=-1$ and $b=1$, but the center of mass will be negative, as a graph shows more mass to the left of $0$ than the right: - +What about the center of mass of the triangle formed by the line $x=-1$, the $x$ axis and $(1-x)/2$? This too is defined between $a=-1$ and $b=1,$ but the center of mass will be negative, as a graph shows more mass to the left of $0$ than the right: +::: {#fig-plot-1-x-over-2-and-center-of-mass} ```{julia} #| hold: true +#| echo: false f(x) = (1-x)/2 -plot(f, -1, 1) -plot!(zero, -1, 1) +plot(f, -1, 1; legend=false, line=(1, :black)) +plot!(zero; line=(1, :black)) +plot!([(-1,0), (-1, f(-1))]; line=(1, :black)) ``` +Plot of $(1-x)/2$ over $[-1, 1]$ +::: + The formulas give: $$ -\int_{-1}^1 xf(x) dx = \int_{-1}^1 x\cdot (1-x)/2 = (\frac{x^2}{4} - \frac{x^3}{6})\big|_{-1}^1 = -\frac{1}{3}. +\int_{-1}^1 xf(x) dx = \int_{-1}^1 x\cdot (1-x)/2 = \left(\frac{x^2}{4} - \frac{x^3}{6}\right)\Big|_{-1}^1 = -\frac{1}{3}. $$ -The bottom integral is just the area (or total mass if the $\rho$ were not canceled) and by geometry is $1/2 (1)(2) = 1$. So $\text{cm}_x = -1/3$. +The bottom integral is just the area (or total mass if the $\rho$ were not canceled) and by geometry is $1/2 (1)(2) = 1$. So $\overline{\text{cm}}_x = -1/3$. ##### Example -Find the center of mass formed by the intersection of the parabolas $y=1 - x^2$ and $y=(x-1)^2 - 2$. - - -The center of mass (in the $x$ direction) can be seen to be close to $x=1/2$: +Find the center of mass formed by the intersection of the parabolas $y=1 - x^2$ and $y=(x-1)^2 - 2$. @fig-center-of-mass-of-two-parabola-1-minus-xsquared-and-x-minus-1-sqared-minus-2 shows that for the $x$ direction, it is close to $1/2$. +::: {#fig-center-of-mass-of-two-parabola-1-minus-xsquared-and-x-minus-1-sqared-minus-2} ```{julia} +#| echo: false f1(x) = 1 - x^2 f2(x) = (x-1)^2 -2 -plot(f1, -3, 3) -plot!(f2, -3, 3) +plot(f1, -3, 3; legend=false) +plot!(f2) ``` -To find it, we need to find the intersection points, then integrate. We do so numerically. +Plot of $1-x^2$ and $(x-1)^2-2$ used to find intersection points for a center of mass calculation +::: + +We first find the intersection points numerically, though where two quadratics can readily be solved algebraically: ```{julia} #| hold: true h(x) = f1(x) - f2(x) a,b = find_zeros(h, -3, 3) -top, err = quadgk(x -> x * h(x), a, b) -bottom, err = quadgk(h, a, b) +``` + +With these, the computation of the center of mass involves two integrations: + +```{julia} +top = first(quadgk(x -> x * h(x), a, b)) +bottom = first(quadgk(h, a, b)) cm = top / bottom ``` @@ -259,17 +281,17 @@ We need to compute $\int_{-\infty}^\infty xf(x) dx$, but in this case since $f$ $$ -\mu = \int_0^\infty x e^{-x} dx = -(1+x) \cdot e^{-x} \big|_0^\infty = 1 +\mu = \int_0^\infty x e^{-x} dx = -(1+x) \cdot e^{-x} \Big|_0^\infty = 1 $$ For fun, we compare this to the median, which is the value $M$ so that the total area is split in half. That is, the following formula is satisfied: $\int_0^M f(x) dx = 1/2$. To compute, we have: $$ -\int_0^M e^{-x} dx = -e^{-x} \big|_0^M = 1 - e^{-M}. +\int_0^M e^{-x} dx = -e^{-x} \Big|_0^M = 1 - e^{-M}. $$ -Solving $1/2 = 1 - e^{-M}$ gives $M=\log(2) = 0.69...$, The median is to the left of the mean in this example. +Solving $1/2 = 1 - e^{-M}$ gives $M=\log(2) = 0.69\cdots$, The median is to the left of the mean in this example. :::{.callout-note} @@ -281,21 +303,29 @@ In this example, we used an infinite region, so the idea of "balancing" may be a ##### Example -A figure is formed by transformations of the function $\phi(u) = e^{2(k-1)} - e^{2(k-u)}$, for some fixed $k$, as follows: +@fig-center-of-mass-between-two-shifted-exponentials shows a region formed by transformations of the function $\phi(u) = e^{2(k-1)} - e^{2(k-u)}$, for some fixed $k$ ($k=3$ in the figure) between $0$ and $3$. The region is basically the graph of $\phi(u)$ and the graph of its shifted value $\phi(u+1)$, only truncated on the top and bottom. + +We have + ```{julia} -k = 3 phi(u) = exp(2(k-1)) - exp(2(k-u)) f(u) = max(0, phi(u)) g(u) = min(f(u+1), f(k)) - -plot(f, 0, k, legend=false) -plot!(g, 0, k) -plot!(zero, 0, k) ``` -(This is basically the graph of $\phi(u)$ and the graph of its shifted value $\phi(u+1)$, only truncated on the top and bottom.) +::: {#fig-center-of-mass-between-two-shifted-exponentials} +```{julia} +#| echo: false +k = 3 +plot(f, 0, k; legend=false, line=(1, :black)) +plot!(g; line=(1, :black)) +plot!(zero; line=(1, :black)) +``` + +Plot of shifts of $\phi(u) = e^{2(k-1)} - e^{2(k-u)}$ over $[0,k]$ +::: The center of mass of this figure is found with: @@ -308,9 +338,9 @@ bottom, _ = quadgk(h, 0, k) top/bottom ``` -This figure has constant slices of length $1$ for fixed values of $y$. If we were to approximate the values with blocks of height $1$, then the center of mass would be to the left of $1$ - for any $k$, but the top most block would have an overhang to the right of $1$ - out to a value of $k$. That is, this figure should balance: - +This figure has constant slices of length $1$ for fixed values of $y$. If we were to approximate the values with blocks of height $1$, then the center of mass would be to the left of $1$---for any $k$, but the top most block would have an overhang to the right of $1$---out to a value of $k$. That is, the blocks in @fig-max-hangover-figure should balance. +::: {#fig-max-hangover-figure} ```{julia} #| echo: false u(i) = 1/2*(2k - log(exp(2(k-1)) - i)) @@ -324,6 +354,8 @@ plot!(p, f, 0, e, linewidth=5); plot!(p, g, 0, 3, linewidth=5) p ``` +Figure of block arrangement that should be stable +::: See this [paper](https://math.dartmouth.edu/~pw/papers/maxover.pdf) and its references for some background on this example and its extensions. @@ -331,56 +363,69 @@ See this [paper](https://math.dartmouth.edu/~pw/papers/maxover.pdf) and its refe ### The $y$ direction. -We can talk about the center of mass in the $y$ direction too. The approximating picture uses horizontal rectangles - not vertical ones - and if we describe them by $f(y)$, then the corresponding formulas would be +We can talk about the center of mass in the $y$ direction too. Suppose our region is bounded by $g(x) \le f(x)$ over $[a,b]$. The center of mass can be computed different ways. If the region can be described by two functions in the $y$ direction, the same formulas as in the $x$ direction can be used. + +However, with the region as described, we can use this form -> $\text{center of mass} = \text{cm}_y = \frac{\int_a^b y f(y) dy}{\int_a^b f(y) dy}.$ - - - -For example, consider, again, the triangle bounded by the line $x=-1$, the $x$ axis, and the line $y=(1-x)/2$. In terms of describing this in $y$, the function $f(y)=2 -2y$ gives the total length of the horizontal slice (which comes from solving $y=(1-x)/2$for $x$, the general method to find an inverse function, and subtracting $-1$) and the interval is $y=0$ to $y=1$. Thus our center of mass in the $y$ direction will be +::: {.definition title="Center of mass in y direction"} +For a region bounded between $g(x) \le f(x)$ over $[a,b]$, the center of mass in the $y$ direction can be computed by: $$ -\text{cm}_y = \frac{\int_0^1 y (2 - 2y) dy}{\int_0^1 (2 - 2y) dy} = \frac{(2y^2/2 - 2y^3/3)\big|_0^1}{1} = \frac{1}{3}. -$$ - -Here the center of mass is below $1/2$ as the bulk of the area is. (The bottom area is just $1$, as known from the area of a triangle.) - - -As seen, the computation of the center of mass in the $y$ direction has an identical formula, though may be more involved if an inverse function must be computed. - - -::: {.callout-note} -#### An alternative formula - -An alternative formula, which is easily derived once double integrals are introduced, to find the center of mass in the $y$ direction is - -$$ -\text{cm}_y = \frac{\int_a^b \frac{1}{2}(f(x)^2 - g(x)^2) dx}{\int_a^b (f(x) -g(x)) dx}. +\overline{\text{cm}}_y = \frac{\int_a^b \frac{1}{2}(f(x)^2 - g(x)^2) dx}{\int_a^b (f(x) -g(x)) dx}. $$ ::: +This formula follows readily once two dimensional integrals are discussed.^[If $\rho(x,y)$ describes the density of a region $A$, then the center of mass in the $y$ direction is found by $\iint_A y \rho(x,y) dy dx / \iint_A \rho(x,y) dy dx$. With constant density, as assumed herein, and the region described by $g(x) \le f(x)$, the top integral becomes $\int_a^b \int_{g(x)}^{f(x)} y dy dx = \int_a^b (1/2)\left(f(x)^2 - g(x)^2\right) dx$. Similarly. for the $x$ direction, the top integral is $\iint x \rho(x,y) dy dx$ which becomes under these assumptions $\int_a^b \int_{g(x)}^{f(x)} x dy dx = \int_a^b x\left(f(x) - g(x)\right) dx$.] + + +For example, consider, again, the triangle bounded by the line $x=-1$, the $x$ axis, and the line $y=(1-x)/2$. In terms of describing this in $y$, the function $u(y)=2 -2y$ gives the total length of the horizontal slice (which comes from solving $y=(1-x)/2$for $x$, the general method to find an inverse function, and subtracting $-1$) and the interval is $y=0$ to $y=1$. Thus our center of mass in the $y$ direction will be + + +$$ +\overline{\text{cm}}_y = \frac{\int_0^1 y (2 - 2y) dy}{\int_0^1 (2 - 2y) dy} = \frac{(2y^2/2 - 2y^3/3)\Big|_0^1}{1} = \frac{1}{3}. +$$ + +Here the center of mass is below $1/2$ as the bulk of the area is. (The bottom area is just $1$, as known from the area of a triangle.) + +Using the other formula, where $f(x) = (1-x)/2$ and $g(x)=0$, we have + +$$ +\begin{align*} +\frac{1}{2} \int_{-1}^1 (f(x)^2 - g(x)^2) dx +&= \frac{1}{2} \int_{-1}^1 \frac{(1-x)^2}{4} dx\\ +&= \frac{1}{2}\frac{1}{4} \left(-\frac{(1-x)^3}{3}\right) \big|_{-1}^1\\ +&= \frac{1}{24} \left((1-x)^3\right) \big|_1^{-1}\\ +&= \frac{8}{24} - 0 = \frac{1}{3} +\end{align*} +$$ + +The total area is + +$$ +\int_{-1}^1 \frac{1-x}{2} dx = -\frac{(1-x)^2}{4}\big|_{-1}^1 = 0 - (-1) = 1 +$$ + +Leaving, $\overline{\text{cm}}_y = 1/3$, as before. ##### Example -More generally, consider a right triangle with vertices $(0,0)$, $(0,a)$, and $(b,0)$. The center of mass of this can be computed with the help of the equation for the line that forms the hypotenuse: $x/b + y/a = 1$. We find the center of mass symbolically in the $y$ variable by solving for $x$ in terms of $y$, then integrating from $0$ to $a$: +More generally, consider a right triangle with vertices $(0,0)$, $(0,a)$, and $(b,0)$. The center of mass of this can be computed with the help of the equation for the line that forms the hypotenuse: $x/b + y/a = 1$. We find the center of mass symbolically in the $y$ variable by writing $f(x) = a \cdot (1 - x)/b$ and $g(x) = 0$ ```{julia} @syms a b x y -eqn = x/b + y/a - 1 -fy = solve(eqn, x)[1] -integrate(y*fy, (y, 0, a)) / integrate(fy, (y, 0, a)) +fx = only(solve(x/b + y/a ~ 1, y)) +(1//2) * integrate(fx^2, (x, 0, b)) / integrate(fx, (x, 0, b)) ``` -The answer involves $a$ linearly, but not $b$. If we find the center of mass in $x$, we *could* do something similar: +The answer involves $a$ linearly, but not $b$. If we find the center of mass in $x,$ we *could* do something similar: ```{julia} -fx = solve(eqn, y)[1] integrate(x*fx, (x, 0, b)) / integrate(fx, (x, 0, b)) ``` @@ -389,7 +434,7 @@ But really, we should have just noted that simply by switching the labels $a$ an :::{.callout-note} ## Note -The [centroid](http://en.wikipedia.org/wiki/Centroid) of a region in the plane is just $(\text{cm}_x, \text{cm}_y)$. This last fact says the centroid of the right triangle is just $(b/3, a/3)$. The centroid can be found by other geometric means. The link shows the plumb line method. For triangles, the centroid is also the intersection point of the medians, the lines that connect a vertex with its opposite midpoint. +The [centroid](http://en.wikipedia.org/wiki/Centroid) of a region in the plane is just $(\overline{\text{cm}}_x, \overline{\text{cm}}_y)$. This last fact says the centroid of the right triangle is just $(b/3, a/3)$. The centroid can be found by other geometric means. The link shows the plumb line method. For triangles, the centroid is also the intersection point of the medians, the lines that connect a vertex with its opposite midpoint. ::: @@ -398,29 +443,21 @@ The [centroid](http://en.wikipedia.org/wiki/Centroid) of a region in the plane i Compute the $x$ and $y$ values of the center of mass of the half circle described by the area below the function $f(x) = \sqrt{1 - x^2}$ and above the $x$-axis. - -A plot shows the value of cm$_x$ will be $0$ by symmetry: +As $f(x)$ is even, $x \cdot f(x)$ would be odd, so the center of mass in the $x$ direction is $0$. -```{julia} -#| hold: true -f(x) = sqrt(1 - x^2) -plot(f, -1, 1) -``` - -($f(x)$ is even, so $xf(x)$ will be odd.) - - -However, the value for cm$_y$ will - like the last problem - be around $1/3$. The exact value is compute using slices in the $y$ direction. Solving for $x$ in $y=\sqrt{1-x^2}$, or $x = \pm \sqrt{1-y^2}$, if $f(y) = 2\sqrt{1 - y^2}$. The value is then: - +The value for $\overline{\text{cm}}_y$ will certainly be less than $1/2$ as the circle narrows as $y$ increases to $1$. The exact value is given by: $$ -\text{cm}_y = \frac{\int_{0}^1 y 2 \sqrt{1 - y^2}dy}{\int_{0}^1 2\sqrt{1-y^2}} = -\frac{-2(1-y^2)^{3/2}/3\big|_0^1}{\pi/2} = \frac{4}{3\pi}. +\begin{align*} +\frac{1}{2}\frac{2}{\pi} \cdot \int_{-1}^1 (\sqrt{1 - x^2})^2 dx +&= \frac{1}{\pi} \int_{-1}^1 ( 1 - x^2) dx\\ +&= \frac{1}{\pi} (x - \frac{x^3}{3})\big|_{-1}^1 = \frac{1}{\pi}\frac{4}{3}\\ +&= 0.424413\cdots. +\end{align*} $$ -The top calculation is done by $u$-substitution, the bottom by using the area formula for a half circle, $\pi r^2/2$. - +The value $2/\pi$ comes from the area of the figure being the area of half the unit circle, which has area $\pi$. ##### Example @@ -428,24 +465,34 @@ The top calculation is done by $u$-substitution, the bottom by using the area fo A disc of radius $2$ is centered at the origin, as a disc of radius $1$ is bored out between $y=0$ and $y=1$. Find the resulting center of mass. -A picture shows that this could be complicated, especially for $y > 0$, as we need to describe the length of the red lines below for $-2 < y < 2$: - +@fig-disc-radius-2-hole-bored-out shows that this could be complicated, especially for $y > 0$, as we need to describe the length of the red lines below for $-2 < y < 2$: +::: {#fig-disc-radius-2-hole-bored-out} ```{julia} #| hold: true #| echo: false -a,b = 0, 2pi -ts = range(a, stop=b, length=50) -p = plot(t -> 2cos(t), t->2sin(t), a, b, legend=false, aspect_ratio=:equal); -plot!(p, cos.(ts), 1 .+ sin.(ts), linetype=:polygon, color=:red); -plot!(p, [-sqrt(3), sqrt(3)], [-1,-1], color=:orange); -plot!(p, [-sqrt(3), -1], [1,1], color=:orange); -plot!(p, [sqrt(3), 1], [1,1], color=:orange); -p +let + gr() + a,b = 0, 2pi + ts = range(a, stop=b, length=50) + p = plot(t -> 2cos(t), t->2sin(t), a, b; legend=false, line=(1, :black), aspect_ratio=:equal); + plot!(p, cos.(ts), 1 .+ sin.(ts), linetype=:polygon, color=:red); + plot!(p, [(-sqrt(3), -1), (sqrt(3), -1)], line=(1, :orange)) + plot!(p, [(-sqrt(3), 1), (-1, 1)]; line=(1, :orange)) + plot!(p, [( 1, 1), (sqrt(3), 1)]; line=(1, :orange)) + plotly() + p +end ``` -We can see that cm$_x = 0$, by symmetry, but to compute cm$_y$ we need to find $f(y)$, which will depend on the value of $y$ between $-2$ and $2$. The outer circle is $x^2 + y^2 = 4$, the inner circle $x^2 + (y-1)^2 = 1$. When $y < 0$, $f(y)$ is the distance across the outer circle or, $2\sqrt{4 - y^2}$. When $y \geq 0$, $f(y)$ is *twice* the distance from the bigger circle to the smaller, of $2(\sqrt{4 - y^2} - \sqrt{1 - (y-1)^2})$. +Disk of radius $2$ with a hole of radius $1$ bored out +::: + + +We can see that $\overline{\text{cm}}_x = 0$, by symmetry. + +To compute $\overline{\text{cm}}_y$ we choose to find $f(y)$, which will depend on the value of $y$ between $-2$ and $2$. The outer circle is $x^2 + y^2 = 4$, the inner circle $x^2 + (y-1)^2 = 1$. When $y < 0$, $f(y)$ is the distance across the outer circle or, $2\sqrt{4 - y^2}$. When $y \geq 0$, $f(y)$ is *twice* the distance from the bigger circle to the smaller, of $2(\sqrt{4 - y^2} - \sqrt{1 - (y-1)^2})$. We use this to compute: @@ -453,13 +500,48 @@ We use this to compute: ```{julia} #| hold: true -f(y) = y < 0 ? 2*sqrt(4 - y^2) : 2* (sqrt(4 - y^2)- sqrt(1 - (y-1)^2)) -top, _ = quadgk( y -> y * f(y), -2, 2) -bottom, _ = quadgk( f, -2, 2) +f(y) = y < 0 ? 2 * sqrt(4 - y^2) : 2 * (sqrt(4 - y^2)- sqrt(1 - (y-1)^2)) +top, _ = quadgk(y -> y * f(y), -2, 2) +bottom, _ = quadgk(f, -2, 2) top/bottom ``` -The nice answer of $-1/3$ makes us think there may be a different way to visualize this. Were we to rearrange the top integral, we could write it as $\int_{-2}^2 y 2 \sqrt{4 -y^2}dy - \int_0^2 2y\sqrt{1 - (y-1)^2}dy$. Call this $A - B$. The left term, $A$, is part of the center of mass formula for the big circle (which is this value divided by $M=4\pi$), and the right term, $B$, is part of the center of mass formula for the (drilled out) smaller circle (which is this value divided by $m=\pi$. These values are weighted according to $(AM - Bm)/(M-m)$. In this case $A=0$, $B=1$ and $M=4m$, so the answer is $-1/3$. +The nice answer of $-1/3$ makes us think there may be a different way to compute this quantity. + + +Indeed, let $A$ be the big circle with the bite taken out, $B$ be the smaller circle, $C$ the big circle. Clearly, the center of mass of $C$ in the $y$ direction is $0$ and the center of mass of $B$ in the $y$ direction is $1$. The center of mass of $C$ is the *weighted average* of the center of mass of $A$ plus that of $B$: + +$$ +\overline{\text{cm}_C} = \frac{4\pi - \pi}{4\pi} \overline{\text{cm}_A} + \frac{\pi}{4\pi}\overline{\text{cm}_B} +$$ + +Or + +$$ +0 = \frac{3}{4}\overline{\text{cm}_A} + \frac{1}{4} +$$ + +which is solved by $\overline{\text{cm}_A}=-1/3$. + + +This above works as: + +::: {.relationship title="Center of mass of complex shapes"} +If a complicated shape can be partitioned into simpler shapes for which the center of mass can be computed, then the resulting center of mass is the weighted sum of the centers of mass of the simpler shapes. The weights are given by the relative masses.^[Again, this follows readily from the center of mass formula in two dimensions. For example, if $A$ can be partitioned into $B$ and $C$ then we have in the $y$ direction: +$$ +\begin{align*} +\overline{\text{cm}}_A &= +\frac{\iint_A y \rho da}{\iint_A \rho da}\\ +&= \frac{\iint_B y \rho da}{\iint_A \rho da} + \frac{\iint_C y \rho da}{\iint_A \rho da}\\ +&= \frac{\iint_B y \rho da}{\iint_B \rho da} \cdot \frac{\iint_B \rho da}{\iint_A \rho da} + +\frac{\iint_C y \rho da}{\iint_C \rho da} \cdot \frac{\iint_C \rho da}{\iint_A \rho da}\\ +&= \overline{\text{cm}}_B \frac{\iint_B\rho da}{\iint_A \rho da} + +\overline{\text{cm}}_C \frac{\iint_C \rho da}{\iint_A \rho da} +\end{align*} +$$ +] +::: + ## Questions @@ -613,37 +695,45 @@ numericq(val) ###### Question -A penny, nickel, dime and quarter are stacked so that their right most edges align and are centered so that the center of mass in the $y$ direction is $0$. Find the center of mass in the $x$ direction. - +@fig-penny-nickel-dime-quarter visualizes a penny, nickel, dime and quarter that are stacked so that their right most edges align and are centered so that the center of mass in the $y$ direction is $0$. Find the center of mass in the $x$ direction. +::: {#fig-penny-nickel-dime-quarter} ```{julia} #| hold: true #| echo: false -ds = [0.75, 0.835, 0.705, 0.955] -rs = ds/2 -xs = rs[4] .- rs -ts = range(0,stop=2pi, length=50) -p = plot(legend=false, aspect_ratio=:equal); -for i in 1:4 - plot!(p, xs[i] .+ rs[i]*cos.(ts), rs[i]*sin.(ts)); +let + ds = [0.75, 0.835, 0.705, 0.955] + rs = ds/2 + xs = rs[4] .- rs + ts = range(0,stop=2pi, length=50) + p = plot(legend=false, aspect_ratio=:equal); + for i in 1:4 + plot!(p, xs[i] .+ rs[i]*cos.(ts), rs[i]*sin.(ts); line=(1, :black)); + end + + p end - -p ``` -You will need some specifications, such as these from the [US Mint](http://www.usmint.gov/about_the_mint/?action=coin_specifications) +Sketch of a dime, penny, nickel, and quarter stacked with an edge aligned +::: + +You will need some specifications, such as the one from the [US Mint](http://www.usmint.gov/about_the_mint/?action=coin_specifications) in @tbl-diameter-mass-coins. -```{eval=false} - diameter(in) weight(gms) -penny 0.750 2.500 -nickel 0.835 5.000 -dime 0.705 2.268 -quarter 0.955 5.670 +::: {#tbl-diameter-mass-coins .striped .hover} -``` +| | diameter(in) | weight(gms) | +|:---------:|:--------------:|:--------------:| +| penny | 0.750 | 2.500 | +| nickel | 0.835 | 5.000 | +| dime | 0.705 | 2.268 | +| quarter | 0.955 | 5.670 | -(Hint: Though this could be done with integration, it is easier to treat each coin as a single point (its centroid) with the given mass and then apply the formula for sums.) +Size and weight of US coins +::: + +(Hint: Though this could be done with integration, it is easier to treat each coin as a single point (its centroid) with the given mass and then apply the center of mass formula for sums.) ```{julia} diff --git a/quarto/integrals/figures/ice-cream.jpg b/quarto/integrals/figures/ice-cream.jpg new file mode 100644 index 0000000000000000000000000000000000000000..8a3bb7eeab069d54f91c31d98efefa188aac1a2c GIT binary patch literal 344901 zcmbTdcUV)+);=5rM7l^(ibO#`K%^IG5s@we(rctiuTlbpL;*pm0s;yMBE3eW*U(XV z?=2uD)PxcO1o*}0ocEmfpYOWf>sw@J?#x~@%$iwy%DrbUr!G-|4HbyfD*!-S8z2Ax z0ImV9lH3505)ldU4?w~OApZvg07fM2|G`!y_y4VP1pv6h_+R}9Cji;M^@(}@BOm|$ z`(NQTVn9KRArlc36OAYN7bdwv3P=e52j3vx>-+XUI-(*X-$?$O`?o9qlRH`MH`4#X zq(wPb|E)tz{8yCpl{GYop}wuRy}kQuM-LwW;BxuOeGO%0uTRgGsV01~2gS8iE* zdb<8Cu$#M&_Y1WrcT7yp?oh1#%l5xHS=+w$RMORb_80$8_!s|Ia1(#)2>=)s`kU8( zttkM&I7Cci{`!@;*gyA4Xr2H7oC`HXdHS`}_hDl2Xz#k7Sjfs;H`|YiQ~l z7#bN9tKHVl{*{BHlk;mIUq63HKwxOtyYPtjA0iVHKP4rneEyR9Jv%2iFTbF$=x0S` zRdr2mU42`7M`u^}uirf*qhsR}lT*_(sO6PEt842U=uPbY!Qm0^7=Lm~e8&G}4)Ob6 z7X2@K7>M>=1mExwo6zoEWqQz_H?<4$I75X3bV+9P7@KQ^>*RzUSxzy zFUd@>hqxnFmjzAgY=V3c6)k^!9C>7m_6hW#JJ%A5=A1U-OvZeBwWw9cd;>bXVom2J z-!4EkvHl&!E2zwOQrmah=r5C_pJX<|LXbQ#dDwI_4|aQGLy`KmeLg@{ z=_K{1`ZV`0JPugR7k0s6t{8`sHs5*n$3DOS@k{tUk{N1=6*o-O!!nQlGWqimrf22p z?$D4v=)L;KU4u7*^cBjiK&a=bF#91#YFJJIUZKcK9GINwFR^QG%w_z_l{2K0c|0XD zd$j@RT$zkwl(1wfyU}&)g?VB0Qsh@ATV|P>S8;+9iLCS~_1c9C&9NAt>ktjjd#cljb~7z)8aY)`A4z)U?_nNyBD73*pch;=9Cg( zdR5~JJGzTtva~DPUDQnOPY^^;&=>LHd^=>Qk4>odDv@-(cExOk0-_dQT%zuS zhI2^Xc9#i`SAwp0%y`ev$Bejby%BJ?&@;jb3p^+~ERvO6?b|=`emFcEx#70MJNBm| z_-3MMO}+i$1Nn&SZ`2gWZkEqLg4(4fCx6*R-?aVR&0##WkFHLZef+`Tz6IO9C7-S~ zpSH#WuLrfI$=L7O^ofBhFp6HZ1|#oFd~e5$Us7IWMP3j<@VEQjM8i?oqN|kP-L~Vh zgBL$*#$O$*fhFkcn~LTbOlO$RY{-<}#*U5Nd>HY&0$ovA=0G9I`SrWTYyVa96*O2C zU#vBp-)wSdy>|0fTkQG|<|zm@&L3a5;`ARViBMt55LxvHQrqVp*Pvqr z>54&}-IkA-PGQR6!Df6#O?9Y4r~fR==-MS(m6sC8&vWeRE$Q1ct|P^Jmbt~d!=6A0 z?}A$~-4~SfSe8X4otyYP9ILPKv~WVpgAGlV;PePt_c4cY%8QbWbwgRN4{L$aZ{&Hhe!Uv4JcC%)@o?*9jx?C; zlwJXRd#F&fohrF)5$lCE-E&j)i;o;w>uFz*pnfAj$`sr}D4O%Dc_yL;Fm&qpm@cxM zw9UqPLk$4EHMw7+t=1k$zR%$A!5%tQ(CJI2BA;QQw$|mP@$f74-GiyZl9SQt$nF77 zuJcsbHAoq}aw~(Qyw*FgQUIPC5LQ*J|bMj`ex7JMOTN*29;TR;@WdzV1gs@j`hVb7_d(4jzUpQ zqno&L+U{RHwWQ}6;m8uEy1}cN3QbLOulRMvYRDMk4Vi6xK#?9au#wZa2*lvK3n4Qx z3EI`hG|x>nj>t2L_WYI%vC$M`%<9q?KgP2>_P58(r%7j)@K%v#M%d7(&a(Dhy0}s? z4lMwQX@mQ1TDZ_9fLW0fHUun>==gA`VCYbFT^(BYSn=sES)fi^1HVm9<(Dut4y5-0 z3NCci2WSbZ!c1-DENq&wc(t>aE>dO&R_s`UC_X~NtMY>ZtDz1vJ|EMILt^;ixZ~9D z(V;rcGJ2lr{0f5TD2VX3lnWb8anP$N!3PxbL2oS;2W}jH+wMC_9XkJzhPv4Xik8|b zA<&N{bm3$A^XE8dPH7GaAOXy6{m1wSOrj;JUbvgorucsS`9yaX2ipnXQ!K2`eR$LC zY2Qyixhy$LuR}y-^X-c%<54r=@QVi~-DPO}jF}3jb7ZXzAFY5OVk_%mI=DSsfk9Cc zqP*<>9?Ed0?C3XxOA|P;Ul1{&%gF3O3SI(S7V&p8TlZp4X$Rv#_Oz;$_N*VBQnU}^ zaC#RQ9BpO+VnQo33`_^(o05KiLE-$K_pCpEz_WeIA3@O+(R~S^sX}mBd*#mG{nbr1 zbx!M^MR_*lcehT6w+t=-6TsVJU2u>uFjh$KJvKrFH_Mj!+M6pEBeUJL#UNsjWyc;%3h3F#UxJb--S-I1lX71E#>x=+Iq36nsYf;@S>M2e{#Z zxadnwaISuF*EWBkMf>`p>Rz1k$#CCFPI8E@c2GW6GPpn|&m)0irae!lGl-&fa8_^` z-d6nXX9fzfEhf&?xD;euelZ68aS7-o`sTv@#*w|shWCv>{guU-;$dIkA2XkvNJRwS zKAyVwA=%S!+d6zOPvH9sPgdxQC{Xs|^g3=B@368OxiGWs6d3mTd@P?NZOC{bHFJlS z@2v#Oc$K@2%G~YeN{F>fz*6^ie}Fv-izV%%b0brRGWhT@?@Ga|UNxZRfCUCN9J^xiTHyx3_RKVObbe}ycLqjS1oz^jm$5 z=FxjpoyTYN0SCt^)d3;`Q~~d(BlyWnF87*BYzR%U=6SuF-3~M!hS3unz>8psq3;N< z9YZWW|55IC;p0~M1DqRdntL%aBOG2u*+{(-#WU`DJwVt~ZO>EYm&9Hc|HFK>l^~+C zz^L)+8T1$nCtR=H$=a>C=a1HqyuPJwM?zNSU*^hxBSzww2@Dvu4$G@sR(ST}sJxHv zQu@vJls5$TQ@R5wq6j1vsnaMKu&pr2jP>jp6f&;3o0i~@{ay0wuP5v#f$TL1v+y{bM{!eG2d)1;l zb9GB1WxdAu6USELLy$A48y+cF{K?Ac09(K=|)qT4hFI!B*5ShzUE*4de zI!#V-@k=ilbl%_Eqj#2do6eYgkewkrSq(PTp-D3j5eWMjA;VSDnys@`69x71TE2D| z{OD`7cgLZw;2wyc;O{~2tgJC%&uRhf@#=>LVNNIZ+DH0J%$J3>YH|aR(f7vO-De=M zSGKc-lIh>Sl$fiZPhcXtv=rRCWcmu_9f15>oxWE0qgsDDYM0fW$=9BEiq=l9#DzV8 z8ZYUDBUa{(4bCLnepj=zfaKI-o5yE>GZS#8scE!qTY%SWUSFwqPewC(oUJ~ndwRxW zhj~Rfygt==;dVp9RM(1awwrvg?Bq7z_Rb}MTt2Zg)c|%2KbzlKV*2yl!yfq-5ZlEo z5b9ACw&uNCC0^r99{lJS(Ey`?Sao;U55L;C^t|p0`Y-a^Hb%3)u zFL+?y3A%Eme%No=bqT2Wb9ep_p@0KPkoTtw^tYBr+!&B*W;Z@gjLip1X^$eSocfup3#UFu;{KuI`+V#P!< z=ymaI5&9Gn-&C)*3kpQq)8mDtwcB7hLe zX9=`ZGzBw`&G;DOj8IS)Fh)w{;KWs28N7O~s?+XP$e6{MpCh+zZ9Gn!~P z^zeH_$YWM3UYctl&BF~F^cb{QZPwu&Mv-Pr`y#WWl0XMurfAcN$Yhc@uwJ1R6HHM2 zQ#&o;Fn-TMEPnQCvrkG3sFJ?*ffi;|kNEtzKaD=f+w`c$GLAetp^t5Cxda3@_tI~@ zpkW(mzVcOVplRCQ41@=X724#J_cjJ8AeW$VsRW9%=Xlc2nHmBY=beK_IYjsI;f}Wa z2UuZd(&RiP^P#RmhR`o1imcm$G1$d6WW*w)&uA`WRx9K}9-61ZK6F~_tG_D>mR80j zuK2f{kT142SvP8At4%(gu;W$HQFpI<`FP_V#VA)@I@Um*L3465=$qWqci+w4+D2qV-$Ka%^vvNwJ3-l(D>K{KPJsLL<2Qdh zTms5Gj_(0?MG?^b5ktECJBOVfs--kfJyea)6^xZ12~=9I_#Wqebn)ic%ubw6 z#dBscWLxqU>q8r3&gGxUY`Pch<-;6L?Q;hA!?6Buj*!%w-e|U+^a*1I4hj9cs=&7h z(b|eszSi4<=ln>gJv{I6Humg^Q-#tBEgi73R?hwV<3#==8U?`<_<0#}DIClSyVdWz zOdryLkI;Pf<88C~JI>Gw{day}^GZ+X3`UKkKws~^_P??$-koa_svr4Seuz8}7<0jj zuV@YIXy(LPWh$lxPznrN+>HrWtXivZa=ShoxvOt9wB{x8P{`)%YgUTb@!o|g=Ckjm z721HdKQ`&Y(q%k(A08e3{755dJvS{@Hz(LE#BhhlcJmth8~!e90ELpUUx*S-Qe}}{=Be~qTHl_zJ0rFFXs{K>) zBz5#*!HC97Xm^)@>$Nknb%V&4<@nSgHwC`PEJfzXjo|F3pDonT>5|aV9H`2qkN^IR z@;%Gpx?2eCVR2>SF@UbBD1F7qJa0LE6VYTq-_Jp#LNp8fE_jLeaN z=+F2F<;XKz)Z~taXA~>rN#X8XBq69Fc5OIJzq=};AkW+-<@D+Iq2gmzQ(JKu4bAIWa^}Xj!b-aA`iKeN@7**er?Plo*BM;H@Uu+E0Vs1QF-=5 zIwgKU@F};T`+`7DF?V|8D3t!9>Q1t>oR`YSTPO?9@mC8I=3k!b(dlRoiwM%No$C_k zC9h#!!vs65CVsZF-962oYbQFk=@_1o6k}TCygHiqXFK*L&h0$1(Q?I{2Wy0M`Pr|! z#9>>*VLYA!zb4!$$uch~|0{ciG@S%o$AS$hcYSMsJ9wd(AW|gYo<> zQVt^*s)q>C$xmMgI<=Lb!j!eLJ96%A=>0let}AtUqUX3%DfSke&G`^7t1GrR3vU^N zYo3UA!)XzfXMS#?zYR8%oXb+LzUPuV7a)EKDDe!| zQDGiti;&BMA`0++zw`DV+PXYCzqVZGLR<)MN`b`?pEHWV;6)wA7UD>6-&9`p)%eu- zQ2|t1=V*bWz8kn{hc@H?A|n9}|JCCs(^}B|+iJKJ@AWO^_vFqqii^NI%e67eZN^@F znRy}^Aor^8j>(>`uMN$coU}deX;&Kx1b4nd20Hc?oC`Ea$uK*7GQuYD~km=@-Q2$k1c zp11v7{%$w#Yema|_?A9Yppvb<^L5d{oi|5(g@_?A8NQduw7nBb9@X3*#~!EcvEd1H zTKAiW6_JdoZ40|4t1voPk5N5klsPt!w+<*0g=cAQGnM#yrKMLcx2lpm0Ro^pne7Bs z?uA!IXe+jy*Izv6uBtgn!ye}0ja#hl;ByOj8?aCIw*BtyYAf>YTnqzyniJBkHM~bZ zERCmc%JrumFJ70o#Y&wwieq3?C#=)F-&Xzrcd;`~gQ&P|Y!*aOsmvv3(=5&VK>Oh3 z8Qu7121`_%;;m7A?P0E1;+v!^GzQHN@R>_UQYYaONGeVaGG@mQ5)UQ^OCH~dRI{i? zGuU^m;U^iJF?oE#JjRA z%9tw-CZfvE6~628u7)&RCdc#>G{>gwq$d;f%z(w`Se7GDUU028LG?C;^Y18X>cBB< z?rMJ=E=o||QrcJp1)^}_6h)eA^NO{D#7SD1=;j&XQS)pErsLc-RaV}V3U8Q?^TP>p zZ0H}P;R0{GOf%>(w8+qxXCk3K(U1H%60IW0I86=sTmq?jJhITJ0`KCVCT=(8rK(wS zqEzbZ-MT+EiJA{NeJJ=E(-ml}Bx!m znZXVQGPra1kj;(f?nl%fb&F`LQ0sg=_(ij{7DKGzxtB(A4bR732fi=%3B>~nE<=^t z?(9*I9l{=}cK)cr@I>{4rk&KrL! z!*|DYvNSnPg`Jn>&A-9hOmo<^1y*{zMO3{~%P}>r$IY}xA^Wf5?Ksw}NX@4XGpW0S zz28~RJmTi|cNavCegBa9oZTea>BjR^vJQz5Iy%{Mv;gnulkNKQ`tIjdH*zn!` z1NB8B9dNV5!(TGFlx~=sS?!1!cL~^@$4Hx;=RsX~-{CSVrp@0R(;*?GLsTa^sN?p` z>svM4&oJldHF0%@sgm=RjIl1KX6MmJ`xwS0t1w^4AE&(|iI_ZxZLcAEC1dmCPd;-P z=aqcJ{dV3T$S83NpEyBvp9JV=Xsm-m8 zQ@{x`IHi13d9_rx4dF^HdiTK^{0QP;m2qJm5GNE$WP!dPp4)Wyu^ivK;sSJBEt)Db zzg6FbI8Wo;h;VlqD1)l5M0mfp%3p4vHZ2V+kB{~{MBErVoci{1IDI`j&^Qgu`hvDp zLONgdVc$rUX@q}jx}=5dr(;bc#r%mMZm!Zc_4gkb%F-pJ3;s@(b?mMgK6)>PmHpj# zK4(+!;`RtvG|xM+OYS_~_3Oz*)?7Djzh7H}UU^tyHpj-tX8DT9zKkvxiye6!^Yqh9 zsV6H;7BeTqf9O=)o>v{(X;NpktL|TyU+oO=??AqRQ^BO17#+b35}zfn<=#49zM?kL zbhBfU?Oif$Nx*H&5?Y|?mz<7oS%(UOCjHVqY7Zk0&d!`K0j)RWrF~HoSkQK#c1Er< zHM^SaYL5a%)*)hjx1GpQRRXu0R!kd(%GA8gr)d={me`O%wpbm2x}JFTA&*nSza&|T8oa>qMT6`JQQzTtU|nN^v}%?C$?Al zW_8Iop+7GJ3U|HE1y$aEk|?O;8P!>O=)9naSHSMI?=2x`q}y@FA=XtM%a3_=Jk{=; z)56p*CO{aKAeS{Du_o+IOB;1)fK6u{pUsm<(`h^xb73v`P#k0ws+WU|RHWGCFPd5L zpEg;x<4ii3NL$VXLLW?qc}koH_wlX8yao3K8a4mcF>~*DRFyMKsKw7d#+T-}Fhcmc zQ!Tp^U%Q;uRHcfA*-phKVRSmbQf(`p^K!8htlQxGVxRNhoNm|8^ zH;ZZ~iE^*@WhCMIc?ohU#-W3aKfmQd9SURWR)+J#207t-H0^TkNQFIHQ7w6MFG(WeA+(*VO{#@(?h1Jgw zq!l~uf2#4m#Uy}^?aMJ9xVV8SD=c%?Mb}#h-P>&;BTYs47Jm*Jb~rM{0CAu2{#ec~ z7K~~ES3{TC@L(T^3K#bQRP`0vX33;oQfMkmGBvs65Uxo38>80F0NLC;nl!VnvsG=; z>1pCx#8p;c$)l~9@vSzJf;U(CE&<6O;xvDEb!%q6-V)}#7|r0pj|WtDHb!a4wE34k zpTy}+Kz$u&`XhP<9<7urErBCobdxWmOzMtN zs2D>xFS1aJo)RM5nJW5oqI9L!)F3o|1}TMf%Ea@;wO;~mcY@wS0pAC9e$MLJrY0*I zRf9Ms7pGs)vuuQRfk}MtAGtWN&;0tqm|rb83bNmuz8HKynN6Hzyr1cr8QvHAbE3tt zNgH%b%xan!K5eT~kBxYLWJ;G4A<`D}_U*P{+0Z4xagPl8ba=KMc+=+x7ppZ!2Znl-aH@UEQ1LQzmYTn|8 zfZk~YUpE7oa7pid3}n~=Cvo1X9rwBYTwBg4T+94@8aapCoycM5XJ6y$6gdd`Sc)+F z$Ssuyh$c?I6;bCS9KA>b6GU%)Czy|p2xhl5h5=g6CYMz|tZ00*#)U>3S2-#w#9EeT z`*w-GK&P;71Rr7faL*!7N3>s~2*aDz@StOdb<#QGj+sk9dtoJYB3({6uMWyO6s$|& z!|p{9_=9hwGF;UCcBu#yz?eYr2SqUg6Mh)7z=@A?lsK!Ddx4v?&GgdqtK{zlW-waJ z1=tMC-H;_GMT|6PGb#-=vb$XZD5axtzaaoep;$%mlkr1`_st~EiZny+jkZg=&D`%) zH>_wxl{OzAA_f+jTxJ&Xi+`V2-sYuPQ@z!VdKT93l_FAd7RkP4?XfnMC?!e|5d4b& zkmn@pJEjZq0}63Rlg2G`H(Ag8dWMvCMk5&Zcw-;9+guf>bLF*xigWUe5e(_RX1gzr zr=1mpzai7 z?918-dqdzkFZ1a|Jftm{vcgOQ?-A@fB3!>*I_&l zA_pyHE!?fFifcnMxn)lr?n{XcfFL(#ny+GjT|2@CRSaF&c@;iG_o@Zw1qKW-=DGzs zAvk=U^#sD)eZQ(XLNf^m(uS&>DZqL&tEAFTk*zy=5<*U0u%0Ok>2J)a3=V82fl2&= zax8^p9efo&IZ&wleG@W9pm+ojbJJBaNRBC9umr^PzX;^-885ZcT!G?(i!bG?@ zTywBZQ&UhpW`|!FJeF(A5XB6}96CakQ0yBX+xfb~y`&LO=(CtNb-P{^oak3JC;*Sp zS!+gZWBJ8q{^ksjS=8j)3vRTU3O4VHM}c{b$`?{M#iGCvIR>zFO`Asyn1U-|eJT8> z;w7oRZk55KxB_+e<97H$tO~eU$L2? zjjSsQyxNRiX2H$bPIpeSzLFv~We9@H{W>V=uEo%sjGLJiWo5#ZCA&m26mIk?fd?02 zqt(4lXW+*?*dLjlGg%pzoN!hR7Ytodc)ce(Apkj(FZ zrSYV=0Zhz7DSZe`%w%vW^ZKu+HjBFK)=r1xfGvWtP169HijUKrcPR1X<<_W&<;ZzWx#2`3R+e{i9 zFo?sB^E{*-VtQ}IBjWoAOk%?b=C;cn`y$IugZ-P)?X8o7^HSmqx)4QqOh7$WZF@c7 zywtY$JOsw0w^2GY8qPbTUZ}SX7m1aMwq;IRPl9r~%s2BxS@EHmV#}0vnBvGopR`}U zEfc%+WeyeSQC2s7oh00DFA|lW6nyvLyU?fRv3KNM5`Xr=ywhJx(}9+xjE9o~*lU0J&g zrztkI&%Lfgp7*0k;ujj&8%U;X%6ThE<6HAR)mL|iSW29@6XfOXu$HuA?(A#J756N% zx^vY{YQ7VcuvbIxi#!sLnF+LM3Vn*t`sVCPzU#V6PpFK3>3{~yTbfJF_Xx@>-I4ct zF%$rVm_zbku$-jx=DM7t#^}-73>&EjZo_N)xtD;97n~qcFwikWU_wHb)HLb2Y19pZea@|&!|Tiz@`8lTPl{v&e1 zg0|=?MKysnt%c^Ec27?xfnr`2C$;l&jyum&$vLgq-Sx;iRZk*CFF5o&E5|(#FE4FW z^L1=~mMIpOi!&9enw42*D2dEVJo>40^FWr6cjd;w>=vMAqz47j*-ftXS{%$}7)+z1UJwFWTpt=M@%{*9PNpicw> zKHuX!sld{y-nYn^(&b%Jgys78hjNpjdzVTX_3PubjrKiE<=aU%krzd$J$voVz*yW$ zXKGT#v@=bb4L7f7mvoJdOT<_svlnVd(b5c(dTJh~$I}7087|(-5jf%H=)VCiU8WB^ z{)o&ZvwC&C%)pxxaTUwg`c)B2lc656DS7wWt=Uaj)$v%W-343!M4X~+JvYlFh0qO> zF zvwJ)9;W*h#(z&yV_yVms<^4;*PJPoCmeU+cf9J=&zT65F6Lng9RCutNADr&lg#J1? zJt;`@jJh+E#J68y+7^>m@^QAp+AHzxc-pCPsB<%u2w0)1zoGjfwQ8W({OssVjlQ_2 z+JNjOAoM6kfy!m#JEU8~mU&Hj;d+R@_{RR#OMrF#-5*C?6GE2LZ1CmRt$@^Mk> zaZ9sd|Qb# z0ypRVsLW&iVjXR@TI5ta#0qFJyf9o{w2^!azZ||`egNaz!U+^Ib&zk<`UqjuI^$e$ zOm&XB7*m}-waF^T&mY|q4arfo;+r~T1mB3(?tvd+th0NX?GNE>r;Ja|f9^4iSu_t( zSWK?9+&Rp*8(CFeci4PbdKR!Hl^njo(>+{dpk@3Aeg@7-AZ?cK3X z)9rXlbrRI)?A_NEYhYaE&J>lJ$hSiy%c!wlQ_90+TpfSN91;BqbkC-t z7j&cBlUY@C2h82>huyywxhXLZ3$me<*XuEGIaaI*F-#Td@iH-{U}o+ zxJ*hkJ!h}H)l#x!Tc17_BZIm&C2yOnp4hdsTG(m$wVifr;tO+yhq1gtYP?n)h&W5a zoS_y9SNb(xe$brdKVQLh=i1^uo{u7Yf9dN6Cod7!nwLiGN8~}ds`4^VX4RXFStS(c zBAFg7#)+HE+lmVdAjV98_CX%&0 zMCQ8|bA2XK?}@5%&REeqhZa6X&>0ky&knjL&7iA;^3F_MLj>uTdv>o)++TUfp8nRH zP0;f3+gCA3YNfp5@NF&D8|L=iUS>|Ej)4ZeYp*}tjHya;fkC^~jW-0sq1?GPZ7S|Eu$bfD}lJ`G?7dD zQ<%&l^5~4z$>SB*ohwZOTDqHB1$c+Hc-8FRRQRRRfL9g1PN=o3ORMyxu`WZqdn-4J zdRlhTRvHKwS@MDM^LX!;p7a8VM2Trfzo4DYpH^+25fme3?UH%<;ojL4$n4Qf>0AmG7EI+P3 z>AX)!9@!-PYtT7Wm}1s^HauQLt<>9}t)%)U?|rpj5zI*(Dym3m?^hY2J0IL_8ch3V z%jWz_v`3v8Keu6mY#PYMN<5xun#U6)N2gk?Ukh|!r}9ed^ifNxW81M!AC^Ebb*YY% z<_0qP8-rinP`^?cyb0Oy!H1(2bp-|Wn?g{~2L0~mf&$}B$%Pv-?P*$U)v|<9eyieIzgyzOV9CFK zdTM;FK7c+#$31DwpCNDWwP9@fy*oQ-%Sc{-`FV-?Zs%x2{*sA1%5>xC5j3kSg86Ns zNwj^CQDe?Lr-MSe@cu_pa3T<+i$`01tav%Ps}0H?f2I%8IN z?`aa7)P_&S9yX|cG4_s?8tYb?x)31pb9>hzX>GC>rAy~VkM2fJ*d&<$60!00isno@ z<-vVk)1HT!Ey0(v%oO7>TPxi71C@8KnTDi0dfxz#fSuierKn)n#SNoL;@dx=>ug7! zr9FH?I=)kAk5acaxi0~nN&Sog`TgrA>bBe?^~SY*;E{6oNx#P&lg7fz{7!|3gTyTz zk*VX&-;$G4;%wo!dgR^S;Bjd1H!r9$CO_6EZ1_SN&F_=tS%X`mG@iQz92H|7w8qR; zCbp}X?ml~cpMXbH%X?w7TjBIjNvZ5*jd&)tkK#KP`uV?wUq8u~E7ke#tMjJ%H1|^m zFJ7pyb6H3sz%f4J>#bwpMbW_`J$|K2kz`DBMoTEYJY7Pl`+27}Oo0GA&xhT0LNe!L zjoX@@xG9oy-*~T+eDMaMc^-ig-g$>PSe{7F0G{W_%bZ%S&uLqYizr0d=I@t|`&AZI z-pI>r@l}+=Ah!3;^c}&|KsFzBx1*bu#RcXC8YBeSb6Ml%^&R5oI5G%vwGm;FxV*TH zrco3a{lsEdqx+CKcrpv5(}VP_%6!8XT`zz~pzH}&Tq7p#^dwgwH!m5pqdacoD_Ux- zAMhrySaI^A5ATfc=hc0?iRM1JhSdp!f(v^6ot7H{N_X$h-3zxD_@!n;Jn?Xd-6=U$ zEqVEonaWe}oTQlxDsi3)cbfI9#^>j#<+wa-(0@@$!Xe!(d%?|&cRO=|x3V_6S?v=6B(su6+Kx84Jay+Ff+T+a!GvHD#B9emjqWP`xv5%hDe%K$4S8J1vp~1P zMfrAi^XOMdR(U?_Wu55=t&lUdF-wGd`T8Bu9USrOL$60H=vuSri5XvKkf?P5M}Yqj zmbmGZU9xUyH8}2NGgh?@``RdZ#l+9c6K1#*iZ-u-m&aBVIwn}5!Q8U$dXe(nOSiS! zQjAW!!`HNJpIJRE_x3xBJR|nB^aR({hjFd>3f6itK{F0awO^IxRBp5L<|Se9l_ft=Z(*Z%&}@^Cx3vQrs7OISR;W zZZ-qq9KI{H*zlI?R?h`9W)0!tu~f>nY~EktoFaHY`s+P#3#<>-#0xGHSqrP{Zb5CL z`R~6y+TSZ}4vx>X2B%%z#VQs;M5$0?)xvY8h;JFcR{OJR_v6#5%#nU)Fi8O(iL*M-Uc2bUs{FuYR6l3g zpCe(csmJQ(yY-i3bf?0)WNX?jrI*L^ny?L;373F2EFrO{_|vl zvXnzHs(9HkN$9JJCMh7P3QGys@tqj`HnZG_a5e>(_S7HbWu-0g>VP*8bSq*o#ytA~$V=)-iMI9K_gY1iZr{T&I>#X8dj0WZ+tt zK%3$vAff)*qvsJQP=^l3%oc-Fszri$2npwB#1N7iHc|lz+*RP~d@lX)>Z*JsUcYr> zQr2(}*PM~{e58)67Rkn&{b0|(u_CeG`aZ9Pv&=T^2%@tR++6yKv*(Y6s@jtvV{NGl z6p)E81ro*aRa%%rl5hl(TV8)p;1A#s(Oj~ld4;rv(Lia>bLHJIs0$$_JD~bgG1AON z3Xf9nf#{dM`XzsYuiU}h#^lOMpr7XgHB`4g)imdbqc=Ux90UWaLTQP+qe zXxGl9_rF44ta}tKBX7ajq3YO&Y0JPWbjl6Yk3Xon$|*3CXDXPR7gWHNJt{Ne`#O#i z81Ha8nWOL{PleP^StHI{r)SMiqEn`a&U1)oCIL&$i&qtS6p*3B8K0c^QoqYxwm%22 zb}o($2r97sjZ8*)+T}R9_T^0KO}}HDG!+~FH_wgMh+AC9I0Q4{%6SyrjasS!!}ABm zgnbYSav&BxkO~vSB0P({pq$8@4!rl;rI{(xL*$-aJ-Nc8z)=w$l;1GOj;pc{^F$E0yMOn? zX7Iu*z;NhmkE{8pGyGauG$Ds$tw{j#zE?RM;@3nMx+3)fgRsJ2Mw;)Lm~VPo%6z!v zOSj!W>Hv*%K8c-0QFPIV9D6Uo`ZMmG*?)q(!r*?cnY`KNN=%6x^5q;q6mV(2IYzK6 z4Yo5{1DH|%0p7XwC${}$G83vae8D#}t5iS`3377ZMOY_q?z>`LGa=+wjh+b11ayaoNo&?yd} z^O=Zp79{e%uN+)pi9VIzn0CyMd!|_zn_Rji%w#r8JV?R(uDQzEm-0+m`_^4^4jS9S zZ3<0{4BP?n9!|Z-g9UC!Ew5FFuqZl2_^L^>kj46ConhfE&na^4`>j$dj#n-LiSR@M z)t`I?`wxA<)f!Ws4JV4MwWkEDhe&P2}TdxNdJB~9# zU5`8J_FE|M@9*gK8l@d4RnEhWh=&UVGtyHUj)7|mSbrP@pM=pe$!u!|4e$CCzjJlV z5F@oc%g;h&lvPf|-B)71NP#oMZWCzncwquPHm+T22ge896k@ai9vlWiFqmcyK z^J3&uNOfRoX?G0h)opcoZ~Kw9*LBwp)yg7GDlY+up$|hYwkN_N{M_u?PQ0nd?Y(cT zn*=u&V?Y(cor`Y*5AZ?Dei5&8`eG_23eT#$kIl2nh%0N66w`O^A~(5H(YVOT4-tY* zaZFO5m^J7JnQf>NqG9Jp&UFxv2A%K0TsX}p&EjKY)-gitObD8?lGin4@W$K z5<*S5hvkbv$opcb4T9t})>c2c`ZjWu`oMug*L=|?{!-Z- zRHt&pfwP5p!hn>Z?}ZZPoT{qCh+3|Eat&m|MiqOj1bQVnZ`snBI}Sc*F6a4urS8i& zyA|fm$sgqX-YG00%JIs!YIN9Q18u5kH;YLXp2FuLg}88**yG@2b#GEagDg4W=MLiKu;K#~=O8zE!)vh6KAog@0ZbE0?`KUD zOx+Sk(S}*Gzx10#@fNvk&-@n?y}#wApERG<0T5mbLZKUWx7WS>OAN)JnFjXeK0T2}gQf6i(cKY-%-ZTu*Hp z=IJd%Jozw2*@OCSkG8hujQR(1QRjx8$jFje?FP{PI7JVf?(?bs3+6cSig95Hf*UkW z^DJV?W9N1bK?0NWJ1g_YmVWi980JaF(Y{gsQV@!vX~fh+s2i&?$+=SHNy~((Q%>Sa zi%cySO^Yo!*Tn2~o*OJk75YohHm=PLJU8cUPRICn#)w@4JQAgo3D7(weI-4!i!9T3 z6CQcL9q?m_@rpuYuFZH#uk{-aPp2S3R}OS$WhvIiuEALl>nI&=Ipl0>@ST*{d1a7s0j&77=%?OA zLJ(Z?xaPbd_@iFBOHV4^^_zt9&$sWMD&{u@lcR*+%laXLy?FDGHsJ$L*)5x`0=Y6W z%~nfQOcv5-rcNh%_{)|U5^D5FLYfyo7e0_k3yP{~yz!gbgU;IH$SxMpoolx-DQY37 zO)6BY-cHCA3km#xES+amQ(YIWQ4~}JM5LGaQUr{EbVVQ{Lg*+h^r(O|0qH$ak={fF z1cj)S&_bklA{_;!x6q07mQX?Q-ddw+z%$e+XDoU`{@bIti|>7aDg&-T_K_7Q7V7=nb@Xia>nS{h* zAE|@a)Pvf8L}ZpKv3PGYHKiQw{Zy`$yJJOJb;YVT5ig&}eDu6^gFc+uKnkmt`l;#xM;l8&Aw2;Ew53{Y_)5=1gvX(O@>$EMRGLe zo;$Z8t$Y~y+^Ir(NnJW>aJuGB+(3Hd53V*KN7uyX>a`%JV#on644r5fQZMjGxjdtR zy7Tud6-~s7l5S)a+D3XT^`24n-(LdtS;iOEKR(vN(A04wamrAZb$k=FQ-ig3)bY5&;C0*?B|5#d+x}i zl1CKBHq*`ka#4os5!X@6H)4$xdYbzI zSNiX?Ye?1*_@uxfY%1qA1rp|6&?}QgO&(~Nt6rQZma3TnM)@euKXGK}zhU~BweRr^ zzBTy3e+=%)@xVS>EpS?_DtXq3KI%;_T>E-)^siYvf9XXbsSR7TL8j%TgYw?g&d`DA zaP#$>9VT3hgyggmwB6^BbM>o=R&^#HsgC;bo0i(|$8WrhVCLF&V5B@QqI~za{WQh7 z?r?@!vZS47>9fuOZE#?~)+){?@Yl9yiwHrtL^c$TYRH{_1G9riKL!(rJ+rt`j0g6@ zTFwahJ+?Dt)^wpwe(jkoBg_ad;=u95oS7g`GbKHP+veiC-M`in6Hj1+IRYph(v_=t z!CxR|wS&^K(9y+n%_E&J1eSyo3Zc{qe4evU*ZGLoi{2bExgu6#h6XZvrPwXL?WGyg zanuQi$#Cb~hu~V(&{w&;m(BS*g%W-3I>SB79H&6fMY*P-xtUKaD}Uaosq#pIba3Z* zZZPO5Y??II4&T~I2fIx_rJup)H}dD{ro$TSV?ImH=`eF$v-xoRj3?#4l8_xtRLaNQ z)Sr*7@b*onb^;;1x^s~=?)ufyZWESAsug&AXF9V$j6RtaV&nnk{J~U63A;a1iX47M z?M^Cam~we&cbza^u#%b-ZEG0xgtcXi?6Ortm2{vFH(+LJC*E7Rjca)r@SaOQ;9F|S z0HR4*)h2j>rM|!}Ju?HKNs=d?&52j5zl=S!U3gk=r?GlW3sNj6{GMjEv_WR@UTb zJbo{3$vl1bd$PFrZU>C3lYLIbO)1uTuTmf#Ynrdcpg$u%?(;B z#Lf+{Pz)Ap+d-E)b17isH=XZq;~MTO93EnjSAxs8{?Z3VaQ>=K0=as$uGN})eg1}3 z5XkzS`|@?vE%D~{tJ8lh;mVJASM$TPF$Jg)XMvu&C%MEj1>@=wOnj#33}I)&#g*D( zdA4Z{+BfH*{mOI6L_(lA@d&2OFk|yo?IrkI{BIt%PIXnbF&Zq`gm@ZNKK-cCGS#Qu92zHADFVQ z?-W0$?n%zaqYI6p+ecV63btZ}4CZzX>_D%A{?>4R^am3Z7-$s#^um?$^;tho$w<2<mLtS^6!tjg%uYjIx*BI~F!4mHkmX zfNFgWe&IHB;rnsMT%%IOeL2|H_s6{80j~UO0xgz*VMFcgyq%IIqWx()Qv>-KLN}GG zpQ7|Pj2f^E7Iz5(FVCy!^w@QSZ`s~&Elfn+PR7p23W3k=3B=iQqc5L-hLUK0hKXP? zwGDZz0%V?)H-aw5rx;{G6Z3oUo1KrERkoEoK+=ID7#=P~XUVFq0J+X;qpa}e77c(*$mPv$Jp zmQ02LDd-AHZ|8087b_}MTz_Alo!vC~u6nLP^gp!D*ILZK@)v-!wJ+Aqfvv`sTy+K6+nNQX4NO;4%+jl5 zHOz2C)E3ubue#DsQv@b_QR8p0J(}w{m7wcyB-)8jPx{`!Iz44fD#=+yq#Z(l80NHu z9qG^OTf#h*o-#|1v~;)IBG=<-9we0xdpmdE;CpSi@XA{&BP>Oyannxw$~qn<%i2hD2xPlAQGCDwUNU(DqeomcW_k>Qtps;gp0(v z<04UfX^N1f2G_wuowghKo*hmnwdg%;e|>3Mw7vV6hM?Zp!#~JU9;_@l9Yi&yaRC%} zkfa{2x2R`kKP<*)@Y$4?Fbh2>gSA1f&}y=vNJesLu*`^3 z?wO`aSMK9`f*!PwxUB<`RQ9(mknqct{hIbAp&!IQbqkUWF{cK#&9E8|{rHXsWRDM= z2k9a^1tR|*$*Z8G<#jLaUeT|!*BN`T+}!;@<2&vG(iKG!gLgxwOv$TVxJxfYNzj;; z^HH;JcHWC0r4$lIlulVG0}I^=YU1qV^ei6p5!+8Y7=&|oozP8eBmx}V-UeAC`(;X} zuzSl&LJL?;-%!Og`azu&@!>)x8#ZFKs(r>41%9|fkrux3+|R~imkh?(xlWI0cBh#T zXC%HD@1|sXjfjf%0tRY^)sM;>QM*{JW7vKp^(C@Z2q90q@dn8qY&t@9F=nWkvu0hd zJ*^hR3V$*6eLNJIG&V%ZT)^T@mf&sDo?bMCaJueYSHTr!Kph(eOlQ^efR9^*ij7Pliv)cQMd)05-RXL@Y=4IC(q#eFBE z7?m%2FHmXP;k_j8-Ur0Kk+m$v9RHt-nW2|MVRbaqm@XU>?H1lS|1)bE^MepT;Ixsbor zP$BV{iPjczpAO=y>w!mHd&a0->p6+^-vXkq zS&q8Y3~L1pf8`J-5Q9t!fz^>}Z;U6q@we zzAe|aIdU~_qP%Avo;|^z8+}}y@yxCi5l!PeEkyiBlQxeuY*gWfp4mnp=QV#_dx z15>r+sFVOa_HFfgaL+%6lpsf12}!_&WHoqWzMG%MZXB+_!Js=mEx{jnN?Vz$hxL4TXFNiLh`@|Du zmDnNCpri;{z%=}Vl1co_r&I#b`1OxrY)*5(jZl6L)`yRd3~{ykjk|eb(U~i3nScNh zx$~F6?+I*Y_Z3ix+nbS1Pji$^m?>rSd;JjeA6m}Wpy!=Zf!Daa zd)a}H$V^V8s1spimr;M5aPV`E()DSA+%^#lzMEq8A%Y|RXF`b;^-0f zLc-)3D?QcZS&k9zqKzb14G9XN!)7JFFxXdWIq71S%Y7gI4>X4~v@@w-S=y*{OQo)B zSHGv0@{`U^WTB}O#h%T$4>1<@+@IUs@_8qRZ}DobXnx%^Ijm7X1>9?q8KFF`IkTsXY&S5H(;*kVEmh(hlo-=J#e zJ7mw}(J{1#Kr$~G0jtEV+;53LT*NbXprVo7ifapF<5CI9Hw>A$I)UX|e3V1Ea$zcJ z>ztD$b*UZ%4qf|`pck&1djRSLF#`LE?6f{pGtp^tVV>*Gz|`Nd#w4RT|as8LlisBVf6)qtB zPIWnE3Jn0}tgx-!w@t8o%A66T6U2Tp4>xrX~$MJiTY%6jyjEt`d zQMtoW?m}VQ>30qZWAnW1if5QzS3bx=yhiEIX1b z(w{B?O-1aKd~Uf(Atg?@2BQXj0c3~0x0RE`Fc+1){$qEHw!;Z%`%YxRk|PzFmS#kZSJVz{9BQoz+C zZ|~+%NRQO$%P{;u2A`xw)O4>6C8QYp7UD6_VzSVi-KIFU(BJjp?%~zKKBV5+iMbQY z$^~qk%{iOoxB$=M_5D}tVz}=UQL0Q z)<0Q${(j+{guRp9!lGSOo9MSr6Pw_&Z>q1QHEm0RAO@L(i;?#cYXf{Ko_<{ci{FnX zTyhUgp4+1JZD_K+-i_$PZ%5wAh;wTQiCT}avo-PU;J3_9k(3E_0YO+t5QGiq3OX0= zJlq6cn0{!&3@tEe{<0zvqV`j(k+XiL>?O0{`%swXSNaTyL*t6O$D;Q%XPfWZ;gAX-UDOs|VE8Wczz+Hv#Q`nbMVsaits$QNg)U7Wun9_5lki*|xMXq~clu>6An&CJ z2A75N>$|VvES%Q7fQ^C)3smq6z&i<9-txmA8IKnlNBtGNsC$F^$WqX(F?%j@%+BX{spTHZc<&0%58r(A=0zE!#GCdz zepv4n-}|Lf{`=Wu!^iUAzic1786hfMgF_$flcl#flEJF0MvpYoCVxA|MUx&MpS4i) zv7Q{=uUFg&x`YR-_GcFV6; zxp%IPrk?nIghy^v4tI!5ji}@quat`tn)&OPIa9a!^_rtc1oh5+X zE<)QCUM7gB895t6zbqF1rM>&&2A+Pv*G(HL+kO{e`HwZIx%71;x0K9J4>(fW2;cswrf3_NJizkP38c;y2=nCS>dgw>fUuQ#%HJ3x5H%ZX1IU>U+)-z%c`Wa2D z-`U`%R@R=q^K{=gp=d#AhoEeY=e7lz|3?0w5G)CZ!TqWNt4h$Hz;q1i?d>kShJTEm7EoHt;U;C1 zqpt7$SsSn@{2~+h)JLNqAU#QVFT&Y0S?tz1%6^3whn<3ph){eJs4${xy1dr!=>8gj zql-ogeK1MB<1$04xwZ-{mYc9y4Ng*0PRTX)bM;AGhR}0#u70th!)xF+pTUz1qtWkU zPRT3d@`I#0jzO5MXYM+WSd$RdQsu2uGt-2T&UE>gCxraK2G4s@RkD)`grHDIip(NxG z+@5&z?ERCRH@HXP7XjN@2c>(d*D65v>O6j3mm+)d{NL$Akc`J&J?V~(HV`L7S^vJ3 zo@Wo&8i*C(`=JRE3DPvyMPBPNFJl(RW$hi4uUu671uR7zFuI&_9|grq)Z@$&b+)5G zM}NQvE{j;8291HR63iB*Z_Evm&&!KFkz<^z$TqKnN*lIa3@hBnO?@DH69=|{A^1Hw z+qxwG->)8-vB!BI>Xi;9re3E6Jy5?nnBv-&cxg3UZ>bT)HYJxClbMnhv6*G|7#1dM zT#@S7fe?v#P+^d%`|CP~18B+Qs>2MNo0}7EbV{q zdf|DcR}Mf?95~fS{j!3dZSv?G8t^Wz-a_4WCNK`yS6&)MYh|&`e=FdZ8a+xSm{Qtd zx_Ie8iASr^LNUXryiNX(JVvZlb50VeDE+N6e=!mpA8Cc7jcWOX7_i+)8|0`gV0J&Q zqw1D<%nG9#4)i-bA_}W}GOd`nHf>2%22&bso=ClX%t^!BIFD~Xb=Vrl6yePb9?ul>~i z;1a_sBefaHcE@-^xQd)LIet6bLIr;?TYmY4;7T2m{Z)pQm%?xf$79U`Nu9MLdx^)X z&mK?00$#KZ%AdgA=bzZLacZQEx~kUA{O36FwJe@XEMwOY%{90q7*wosRg$+v-tb%S zK^bB%JiKVGq2Pp1``QlEz6E9*`bF?CFM5elB17)d*jI`8i50(YjMLZX<;h$hre7>< zb7@6hB3UE;pLol>t;W(>0vi&YCI;x@q=ERVZjTbKc~F3GTKHhWyW^0Q+=a@k`ZrSd z;t`KLI6Eg+w+is1ujYrF2yF;;ltm@$OXZhxhr1>dN?cLG8%1Y)T=E!mVqlAlSXp6JyS>Cl@wRujvhBsHkt zQj+~nHPb!Kdptk56o33Ft)l2>8$a|2zPZ`<06aawJJsBGq#gS*lyij0wTzG&RrF+& z!j}ED`kM02HW|#)g}h3;w;;`JaQfUii9_#^u|hoM1*C%=pJLyM%PgSy-Z=jJ?PFLN ztXBUf$75*rUFDsAcD`^O61+>@lg35tzZ|F^IAc#(W*iKir~xpl_r8Z~jI{mCah^`> zRq*W_+ibZ+@%~ETfhOn@Oy6jdE-|j_bT2qSHDypov(to-?d;&x#BkYSGSQO)EsL0asPsWPU3RZ z`^ginMdob@lER`;TYK*8=g-0_wnqCpfKnPUyEq>QXJ~#DOw&MKNsb7nmWP%T0ENTrr2<%fr(&O|Y+9W$zQNlNsVoB1Yj^=y!w65o+ zp6C%$BIzo`tEa`fQOl9xzXCk>yI{n1=$jU4N=jS2^ttaFLloB=0`F4vkJ}fHwrxo) zi{;dFM@;0_I**VR*PerJcd9IO9OAN@qD)6*Tqpji0mAA37;XvS^4+%IAGw!sY~d;F z8Hjs`Hc0vEc7mh&Z)@#^T~6b?l6(mOXp3f*jE$z3tZ z`tG5@`y6I7Y-Y!>gZ!o4XoNfkDXIEJVEglqUHksS_6=3b`8O?c!5Qu4*{TPa!-SXQ zBgRmU^+=mAb=j)|J4j}P@<>2JqghCV!13E7P*m`0QWcy6eS5p3d;7^nq0s{kR1`7* zDF|6WAuD9JJ)HfJi@F3NG|uY`q=9j(q9?-aD@_jG7qtBW*f=Xs(KzQ*%MEI$s_iB} z*r~h7S}Of&c4bN@vr zKs8x>{=I>_G(?$)`GW0`5dZ>IKf7y;`dt^7yDFIIU)pHI+{rn6U+vEO)BMSEd;9N@ zmr6MO4!sv}tpfARo9mDcs>>PP7eD)4g!bW{!5QQ-BD#IrY~l30sqNQKaBVEWpo>Nc zsYMX|^s9;JjwxL2Vw$Gw^4~7?Gh~dX)*`ZfKCxk+(!OY1_QM?#aOFF=8~$46whxXfwX%ZPfBf>9E%t(TC!d`5G1^%r)-C@Xp50 z$6M-vS@06Ah@!PP?I{a{a-$MRj8f>%eoW)biOZ^j$iVvf*qpm#;X!3d@Kbok-h;TV zF(HQ1nK1sYf|y~zYQ5SzpGX%Ni&~9)G3P1AG$~PiyxkVqDm+H=FIRj5W=lI-jdR=@WDJ0DSEJ|u9>D_k^h<52}Db>IQGZ?{Qw3B=w@?!TMab|5th%hL+VrWHnX6Da+Cp^bH|OF{3imXKG@$1j5;u)|no1E~(+Dx9(swyb*vH)p{Q&w{x;^ zG%*xQwLp@0`(VxTeS>eB>ydErzO1symp^8UQq6Xvdz}9<_ybb1ORM)ZdTF9<`VI}E zG`xAjaSS&aZ9!%X@s3F;wdCDF*Tk%KZ{-6c^1pM^?1Q zA45ApmFVqLV7;Y{o5a&EyVt6g#c_tjrpl#SGmpg2Wk0XEU8ENPB_E#e%3X>pQttny zUST4?I1V>(>HhA;683KU zj2xgi@DUi;pZ5l&S5GhGy`I(=XqO1Meh(0Ic~c@hVxC4KXFm?t@OePBMO#d}-J*VIwYO63EFHQr`a_g0vkXCfm>h5yo{xBG zO&I%}*9 zb4Ku3hDD-M_EAJ^S>dxH^Kk5Jh70--xLr3j{iI<5r>-F?ov2sBXDF!eCSL)5OyuLL z?7${79o{FW9>?Z@(wYstF5E5bw7ZmmGFl)7;&b}?G>y})|JkbYPHrK52c-$0s*S6b z25AoMDD}Jh*Hl+0<)!OgxD=Z!_F?i2ybW;Hi_$Lz`B065D|Q#KZ7d%^Zxny&8>#w` zj~@{rhke31K*bku{p09U5+ZukCAd~Pl{Piwdi3&rA5PZ2`~A-~&sjevinCahXg- z-ctB>uF|LRt}S*~JU%s!*f9OIu$l`)JdQp4wbIl>5c10^h~rES*xdbwq1dNyZ+-Xf zqJLogeRu+U8<)$xT*<;|Ydn^3Nwr-4vsTJ4&s)cJAG_P0&6F-8VvdUiGxzLWkLMsCbQl8_FU$(twDKb(~%FX!VAe zRNDDan6Lt4RoN!ckG30+6rts!%Mh*Mx>*=O;%$>j*mCe2_6j?4UG1@P#6&(ola8es zozM9ER56sErTXpo6Z=tUN?UmVzatrD!}o5cNsKoRP>YdKbjjez`%kapK3||Sl?6`I zufTa)`fobs%>~7KABG$-7nWjN+VDp)11I6ky^z3%$L~%rpo4W2^d%+{T$GZ1f`OD(UCPhmz*{oe3?j|o_MCg;{x0weJNd74#^2-e zxsyYNr#3|8s^<;fJT`{E$5*9UvRgb(Dvnt2&F|Thg5>mMV`-W%2A+s%v1pC!bj!dh zFo-9?K?LK>Fou!pu|1YVZmm7k@8&RFQq+m=M2(sFVIiKI=`O>W#4tbv%_;&FWN7Q!h^pt zL`Av$883a~9;%@v`S$q;W@1Sv;udK)&VZ6^p7m3wCA@B`L0e5Hms+xEt?KLt(suAT zxmCg98OmLm5xJrxhw^QfJ1CpbC)LLLU#UfavIFkPO=-MfM_XW~8?mhJ_?H!9o`fuM zfd{EQC!w&k0b)bMP2y(D`O&1R?1%Cyp!@A7@%^SUcjBNCCdFz}B$xJ+CP%96DYw^K z6~l?ie+*15*VwsJ3EMW(_;4e790;!TkD>WMsANxMq+9tmCB9dW(wf)z-+QN|)hYp} zC&srSQI+s|hik)4dc=(jUg0WtGYsS9{!VkWB?iHXXvJ^eO8JY*w}ODF%7Ja~8Ko_E zS)mXyf9C4&WrE3@iS(!VJ}-xMKARIx9m>@A4f%+Kj>iIeG3Ohw{}^nVg;|myrmu{i ziuCh=+I+z|2IYg_#}jGh6bwh67 z?*N;U@#<#Vj4moD(bP@B-70P*D2|%C0O7Kxyx$OibKEC8eV31nt)~ePi+dmK1+dp- z%0(uUI+4orY#K5i8mtZW$>F7mDfx&p+NIMHX)n>}zFAsFGA}*tmuZ46Nbhyuf9k@k zTk28CkP1KA4B0J>6*yr6P%}8;c+t)z4)=^`8e;a zy~UQJI|rDgAoB(iTTOq^jmdZ0k5tb*&i^j@E^_%0E7I)!;u!J!o>Y6PXfjG$_f@9Y zlkP#%u?1+0Cz-I#0b$c{E<4#^d2XMYO20PgDH8P{CqmB7EdTt7qqDMter>XvOZ-mh zf3m6oy}Zw6vSH&=1?qh>Q&$hqV~#(P`G2ntZr(x+Tuqwwu+zm}v|O;{YEs+s`i{mm zaR1n=7oIJ_=nvi9F49@~$3VTqlO+EN#+GwEdexlzkhG$4Ub952bZft3`J0+&*PY_P z(1>=1xHsU=j%PRKMJ(;B?iCHk!_<}k$g|YjpbUs9V>#-Ym zi_@yBniEWOar#tw!zoI9X2I#adX$R+$OIpDF%LDvnJZ%)ob%aZ(MX-))9c3ZoReFu z61_cK-&}>wEdxa&>=N{6Tt*HGTgfc`imrIN#_gpS8lMBJRg& z=7XT6vLfIut#6hxat(4{ZvAc%Z91#z zRmxnPqMy?Ki}O$E{Q>tS8UBzRyqjc0Q(bLM^s2G+>cfSK!VPusul`EOLAyO${{vOx zWDq0(bl)gXOS(?X*WI2`KiDhF4QOEVLx3OQj$xST?F){nY!^C1r6c=6CtcLB2`!GO zE?ZICcEmzNm&205Y)Vh|d5EwDV;wNr_tFCA{J7?AYthq=EPn!8_ z^y1Hr-z(~Y)QBJd@!<5g{?^1>AqFJ$<5_fHKr_6)X&ae6_|WJ#`*wNg#mkl(uRT5K z{TF{{?Ma(wO4vSJQ=RZ!+U2yFo~cm+m)Ql?AJ~34UOiKCfBx zek1VWHAk_DR+J#p{c|Hv#>2rCm{ywOHpM~~A4dLA)l^YWba{&s-DxV>hJUO*X?kyX zG*Pj@-eh77{+`U*i%FF){vIlu817i)QQ7h>xU*LHkhI$gn5kNXDIsTKRq)=iZ4bX> z&&B{tWSwjRPyjk5T_?u8;wtlYzIUa2Lj6v+<}OU*6214xq_7(Z-o+=8n>CXui(aOX z_VH$2O>N(wOCQT#3Nul(2&iOyLvZg>VOWH;uIsG^Z{bpP^^yaYheUezSGS?{EOQaw zXJ;(TUows^s^~8J}WqJ|r)*Y06ox^5uC?J@oXN zq34Uw;dS^eRhL0nw(XVW?~|KRU}2<~fN%GE@N75>@lc}NVEzic+o?=r*Rp#MCb~Fn zVBq=C&izqbZa2dL*A#SNmw%(KX=v+t7UPliwe6n=WzLd^N|kG&`RPEvqX$~YxsJxJ z|LMe>AwZ%VUP7J`M{hBQ=y?;dH-}4=3=*n_@XS;*Znb!t~-z}zqKJ;g_AeYD4e3-Nd1Z8ZsAt1SN|ZqxSdccXBwI3U;#ZBcF&$li&#Wujd={JzA8rm~v02qSJ`>NQ6^fDjlb?Gt; zFJsO9olvZXrSmtos_Y~b|Liz=@x0lk9m3tjv!sh$vLorgb^MKBS31te(v(QZ=))p4 zNC>)NrfZswSs0)&A32;MWkh)m+Ht10TE0C}S2d@Kd;4=41wiK@oIq^_Ut;?>?^0{! z^7y0E25WdEP~Lao*kqT9zyQX%?se<~!tEaB=WwLWS`|DNbuFA--o}Icet_jN_K0MK zywc>iFfXXtav~~(f8xR`IyqkT6dD*VMi2FE7Rhoh}{3*2Zu`Jill z!kSo#nR!wyY^i&!LxXZ1Q+*2Xr`3t^yJzf5u#Wgf*%%~uliQ*}+KA_KszqK5t2dA$w^J9Bb@{NUL?Ek4dE#XnXQ z4n*_)b(F^wj*+&!z0|c+2~usltPoA9fhyDNz7C>Q)+MaO#S}iZZQ-B{EwS)OVB(#M*@-^rrivY4i?`K8i3S9J;6y9y z2;rFG+I})i^yvudp?Zl$Ed26VVslQ;`Z0<65a_3FK^#r+t9o^#bnXzu1=6j0AEqI) zwIFcJ4h=#Sge_pZb}16O-QJTQ?OK}v-&i0Mi{M(4SvVjm9XZlvqN9;r0cDkbmJ4zt!(=Yl^<$)}Vc2^Pc>I#I# zgVPyhSO=_4Fo_DJ-?U&DY_Et4JOi!r8fu9Cld5i;VHuG)>zbC~!Mg1V%{>&_=5v1D z>tkUd;wpK2-1B-$JC3_F!+r$%s~I=`^^mEBOo@O*g}Wy#&A-mtxED1*_U!cxAluL+ z653VXZ2)i+5!ZY{E}uyI58VVTgJLKuI=`+aQ&)1x$&d1A;+Z+h>_nPa+r4e@pphSv z_%-a_M*{m#h?c!~GR_^&MDgm!prxdz(Z4`@{Rl^z48SPS_FimMzei54v~5*Zi@V=J z!aj3rQK8%`q^w#YLE+%B+;M3$R_0p~AUu5Nkuu%B8b#9b$3 zLbj$PtA&m79>hv1SNz9twQGD2E8rv$f*mzt@3i4AdCHy5 zVd)lq=b&WdsU0mYlAI4Pqm8Zy8<3+0Cpu0TQPSg!L8{3paNyGSn$ryAb6m0tM=*6k zxwQPYhEoox*)y@5THIaZv73Vb&<449fV;?|ZhQgV)Cdxj;QPlA98Zgwps|#$H{d;5 z1hPy2K2Sb}%gBx7I|pW3j+muWNMC6-#NGI=4}>ID58lCZwML&}uP>bI<{Ah_2te4U zP8&_zXE;9EMc}CffP8JSogTyd8j!0o8nV~i9G4$#QYjOs>6?~KMpSXy9m?1Xy7Xz5 z(%*E`#?QKqFDqp$R<9k$kt0%iWv^_is76s|OpXA?;~25#>(e^DPap9*mxvw}loBG= zmI7-S=^Xc)f=AyR`u&!@J)Rga*5`B~MSlpKE1fbi4_vjnwnO%uHS5(4pOW>5o?-U} z=5EYu=Afb?Sdfd@sTgu(t($)eC9fK=^Dckb_v$|dlb~&MrLY$vNkH2+gb*Y(SGWDQ ze|tV1EMYt?(3U@zxrX9LI+?^lE;DYhyL)Z1O9goEcR?42TXgMJ_vXAMi!9fNaRziA z`Y?{8Ni0{p;*)!p{}(+MU$q_*0Ae4c!GSBI2rmi-Ox{D7DOs9rH5z?eVaQ6;V%rC& zSyYLYoFuW(fY%{x&Pu=CVQzlDb1rwWxd=(XSjyeP)iq($9M|weEr0vR{(n!WQ-@ph zo2`eR@jgi&%QJKisP{c;%`%C-?%EsjA`BCfUbK*ME!e@NYU=qzRlJys21c-e;gf@MSuIw(@YkT~PUm#=6;Id5U> z$X530j0Y1kPh(&iTN8{7?}C;3)zor{pSzk|>JKf*8KrA}A-;{b&3?XkqY+abQA=be zuTzGI=^d`#h^)DH0J5>)Y}GUu7#4KvlV&BBOx`pFd=G)0)?DFMx@(>pY7s5gaMAWOU#7d)ULh&h1=?7S7p(BD;j2vjs>-KNO|>H}g+N z=EYouGo2wo@Z(ar0;Qh_CBG%+=U{>JE-Obdj$nDH_w{qPsf{1~y7Q6pC$Xn*kzQes z^Meb<)Hs%YL4Q{+{s(l>&x}!+@G(CsI~|x&l-DMjb)haPm2?i*MkjA{J@|bM+1SqR zj#sjA*l98RZ61cLqt8@@Md%zs>xJ*`c}-=;#)}<4blkzJn}D+Y=PnYxqnbn}RI<1# zT5o)l$_b)N_%f`nF)?g`_gu&-r-d5wl+QnFGC-Vp)f0{u3^T3N`j027C-J9T06TjA z3}r*FvWHR0c2ZkyJM00Nb!!}c)Kx6&xy!}fWTT7OCAi$d$avI zo$DAzwMTT#T^wwZSYl5Bo!MQXAPLephdq{YBjQvw1zz)(DFR3|GSL^jI{++pdc;4U zHA*nEP!|smzI3itOm9jxDq@i=xCmyg%RFd`(`09o=z6AsOC0H z`kHd=*iBukr8ws3saFPx z3}Nw+2Ez7CpWY{`{0_6UpYgRZeun5ST!-6OBYJUy)w`E|5wMpjCKjWLuCJdrzZr(c zFc!@)T@aqRbWEo{$iDn;=07M<@U-&gUzAcm+b{UE{_+FPI<^DK*Ea+-bAeQn$MUj? zO~Eu~!p|kX!t}wsTVohZPCXLzDSoqa-{8u*7{=Prl%uQEF#f!)hq;cC?|a?W4XJlF zpj^{R>4cH^5@l5KwI`Obt!qBJ7rJ{pO>ciWC@b@L#B|<_d!vcJ;)n{pac&}3;;@Edlt;iWrEXs)i~?E_m@=jP3|)H z>>7>f&#A;3N6ruNUW2Kpqw&w4fKYLu}_zffgsXvB5-iLXM6+O=C<&3dq+OL<81z zP1+4~mVxFS_@Z+ENSE?zzPGD`!nprM+O|eu%5?3rxeF;KDjq7b|j-vEEg^dD+slwxw6#FDgRuX~-NEQEw zt@8|O@{7Mbii(0rQF@7rh=PEC^b(OKAR@gJ6p$t$y@f=PA~k{{QX?R}L`vuodJz$% zN{2*xODG|b;y%Cs%2elfei5|6+qo1zwxRU5LL@cW{5_`eAn{eOx5|6|nrpTDz^p>Oz%OC#~kZ_(DA6Rcb6!8tDw_5V!wH7@v_C{f2d5eN>gSTH3a7i1 zPh1%UQ^&RF=#wUN3?i;+eZKl(iFfLysohXJ$4Z}LNT8P6g0gGE7;CB6+Ax3c4DmQ1 zDKj;`NWkVFU4`9oEIMpX10bN6u9zf_m`J) zrbocdRXLU%1NYwe*B#jB-~D^cKFCg=X}@K*rGHllB7Q-qz(*02TqG*LJrXr|U?-Q> zG+&b?p(>h`UG%r~jGgr~pMU34mcUGfh(aVUpYr?YXes_Ui-{6 zF|~>s-iXwrQxzqCTn@XGZzbV)a=O~EIo7RMyth{W*Nvw`EKB%TVugDO8~^i(Ca+$G zWh0W~vM?>NqOeiC?`NwCzn^{o8@HX6&bMziEMAN|B}UA3_E(G$OY&2e1s+e7&wjGc z{;-l*o`+**^Ie$hi)b;gKW#X!SGO`ixjloXG@N&XSbuML6X|tvYo;xO{axUb14W2q zG{GW~?Iv0H>$!RXkNyCKscrE<-37yCyM!jQta@Wxc}2&#vSPRB&FQ?vklqVoe`R7# zQBQqffjX$j1cx!jtGG|mceZ0L(w#d-J^t<0B7ZL=dd&~x;L&-+a%RiCO7=G&m5w9Z zEq{~fr=!;UsHoohB#SfT%X4>0Ps4QAR<7g?^N!_Bow-zmUsJ{?Q?IrV>TD~6Mtyz8 zoWU&zmytk}1itsvY2t7;{IigN$9(w%tWfWL26YJJEwN!h@rN8jGeq{pO-oBv$HvOK z@4)6dk80Uh8%vzJoMNG-LoK(E+Jv&b(2W_XxYKeq!E0%vDy6CXao%Oz>K6Qg#TCZp zaCFs)p}OtPQTt~z&CQ818(o%9ANoXoGU)vZ_*A+tQl%%CIG9k-=H4?U%Uv5TZ8QD& zgzN2|P2`Tf#|koOk+l|e<)OnkO6ZMf3P&c-bqlZl9_?*{;d==6WK_ z^{WYFMv^>Meg}+Y;dOhbGs?Bi>EP*5lqPtDh49j(wvcRlXYF2yg}w9^XFWQUC8TwO z5|kis?^(HI{%R6hi854As#q5*j20YBk1McoMVJwQ1_uPQ^jSx}iT|=-Kn(b&B7}IflTd_SFcCf2Ojcw77ZVFWxsho~ZzVh4oZ z@*_l{*rQb&1GV`|6t@oV<&^NUzmQl6r^wX&A%wXqX+AiRMg$kw=_MDjCd!Mmp$IZK zmg)NS94_Uo*ZWC}fG19W)GJ^s{dRUW?4ZW9#JL%D>uAVIj*E%y@g-YAPY8!cW9!pX ze^Vog61Ycx=YvG+P?-9dFboj*j|v zd{2|o+?GMi>e-LhUR{6JE&@2okMc$E8R}gnhoFcyUd!MvVgeGVJ)+;X4wRQpx{N-x z^f!>}Gt?DPzlEGf-Jzz_IOf-B=ifKKYD!}_5D9b~Koz|FEr>nX&JCvD3#<&BF0EUO z)qQawvm6)!gFLshh^eK9>id6h`bTHldPOSPK0atU***3iCcr00Rm}a;SF(Jjb2R-| zj`$eg@n)fc0PwpUHp2P6S+ugSOcE0=%Ob02dk4QF0iCYxkka`)B#1Bls@I`KK6snusNDjNiPel~Q@eLU2^PSX8Bhh5vXCs$ za8HS5{0wB9YH{wwiaLudfB~Q6i++ThqMEnf@6p4WSDGL8&1(}@k*!r%q&zfEUJ~j% zB;&x0vWoujs~dhAf3sX(uKx++U4qIlFJ5+sT~G2@_3&w4MHC5*Vk(B&T{BqTZ=Zu* zuZjCx^l+USdV?@Vyn+|$Zgx}AU3w1$0#g0nw>|3)W85${bY$Rc0;Q~rMqW*&|<SZ0LDq z3FLi2j$@}TNHW~nRknm)H-J6UBf&wpZ{C~O3MfUfiH3o$*b?DSt+;}bj96s%p*Oog*0pWvi21P(4O}5!R)Wt!!`X>x(? zjMrxX7EmSuo@?9M66d#{#kfxNjcKPrCVS-#wP6`sYPZxLfn*Fp-djv%w#=vjcU#1K-k2I(j`;!O@_4&I(62t{62bD~ZMe>Zt8YEzowo z5yZqz?IOvIka1y>cG3NJ`o(!F(znA&@JHB$XccUGB0cjR!72UDf}%%Ge?5dJptRySYBDX{@f z7sr-i?;eGBQPC2JOR50Vw^Jj}N~0@#0SmeR+%Lxm`v z9_DjDSdP<*OCA?augrMGmywQ*W?wN$ZHAqJJX*(`Rl6H(e{HfwdbC3&*eUsv$tf=e zuR|8mnW(5GAF;6qNq+~5`L_GsNvbHqb9=binvkEafEk;`u3p{uu%#ADLbM@tvBWKTEB+7Ir&Ihrrn-+r2Syj=soQ1hYFwVtI;XD6vhaKd&y zRlf>}TRa05BJtrZ$n=l1B`FV0byZhGLAyzRI8OM=NR0H+{2OGnfXyb5p7@wuYmoyEN(bZ?dHBHpN zuiCz-ip|M_8G&PbZKQH7sW0N_L$7|7J{0QtcQyST&Vel@bmMiJx3w@25D+&&_lKDSwy+QVGj%e2;KQW zMF-*YqCpeaXM?VpVRHYMP=~3e;LHwq^>82e%?b45m=VVZvtR?zxq|WY{1d9pqRp3Mgj)O3N^|ZE8FL4w98T46a=sn82 zd9+OyQzQQRXRuIxfo*_{`-}|dR(q&@!#7Q6y-+dg&J;{A7}Z%1a<87h=g}X08b=hf zDcKPr|2f-A|8y!ZThnj9Rk5t~{feREU>gtZca92PyfeDNkJl&R0C+V>C=0dzLAH%^ z#de``j-ovaR&2>GXn8FPYH;Z@2kXyJX@U-L|3AM0x+3lk)hK~~TND|j>-&*ES)Fy` zmfIygwU|9jjG9ESV^82&RSoBHuMp?n<&63gsO+fh7lXCvmsf1E6MPWvH2*^|0`>s! z{eI!Vput)DRc*)#T$Fl&q)hg9Q+lW4QZY!hI`t9|`)n#K@WQ{h);V9TccRvtEf}^W zd?@@Cvh4LvkXp9I@)lGt(kuB6W}lq98Dli6!&0yrGZcx+t0JRuz)Y@Vd5_Z#>5SP& ziT`~jf0U*%K*p07H ztud2!ns5WR`3IJ&A8SpI6*x|CWL-eH6QSLuNY}xyL6v9A*LM!I@%K~p$kOD{sr+b^ zftv?EN3rRAGxEb}c(qNhbazJGG0%%@uz==@U=w>h`fr9;qla+$Mu#q!r_Y#O?bF|E zIe;^Yc-Z)f&3q#kYIHdmsymwA=!fAh4r;D)klM4dow(U06EbV^=1%OYTX?JJ<7Zsf zGqsUifBkm!a7Onwh~I~9>`0hcF?SwdK+?L?LI5#q6lwWmQ{VLVB^T2ce0 z{mA`af?r|{7e53DxgVrpzF;o=ho_in#-XAsp>Fv4_RzYjZ&nf~5-(z$yz;NVpC+C} zIz(JmKpkGACIVh@$h0>$txJGu8PUZz3%~4ss+AyT9Bh9A>k4}@@>PTD;IX&UrFnzX~;1!UF2ereP@DpioC@h6tZUq~Ji$c+|XU?8^^Yw~U z(!>c6Jp?YF3;Yr_!{Ze&B=X$&lC;P>u((q_qxCs zw`Cbzkf$gRIDBACtpsX6fpLrE_Q)~sd`=G3Ft;Z~Vw<(56tUMUt+NbCo~erua2B(+ z?;h#Tl2%H+j&Ou*#o`PZUFA--CUYxm?@!P64$BNiB{~`nya39*DHhax%u*V>s%WQi znY-<}TOd=xynAx`t0QF&vdlnE;`cZFg84~0de%Mra}7U$K!mbO{l0E{DZ3&_jIyXl zoupR4mtv4;(Dsq+)cY3DE0yQVfA%}|Z_qx)(#Guwv?16XQcRc9=cwYrLif6T8^cS$ z7^zU68<(nZv)1HEVdeALg#jKuaB#zlp^?SZv#N&MvM2mLwUfqOHAUylAT{NqFuYlb zWz3VwjB78;(|7dh2OsK_UsW`Re1%`)M-&)%m#E7LHBiS+<;B7^thuuB+*!L z@7}w(N8il9)vff=@0U-;rq!R#c>cpQGl4f{%%tu62VIXF#;sCMuk#iYwogr}7}|n8 zaFu1u)f@H_QL=nS?N=|S{^L zbc=o--q|Jr^iS(|o9|1cmw?b#Nh&4UM-&-g%U!Jkt_Zd;ioG@BbtTOo{x->RxN1|( zQY%lGV?^>^Wrg46Qd5*yYVI-|M5^iFT6DuxakDwiI`axg0%D~t1l4V}inPRgqnr1j zX-rdszZ7IYibTFT>L2Qh@-p{J_+lFc#QH&CM3?FS_fsGWhR6@~3t;o`4dWs_`=rn%EKS8L$t;IOnO;R?t z3;_0gChtt8?eho>tSjFUc*B1Nk{AT8ElN;%NZi1t*?g)u6!%r=Lsgwxph!B{iKw4$ zw-bYiZZUW%z;IJxdx+19GOHM_>)TrWh`TE;ba1a=KPyxa+*yZNcJ=@t5HUb%Bnk!v zzU|!hOg@}-IbVFn4mNDS_y!7?N~H9d#3myCP5!?^qUf_Pip3?f8E8J$+NrZuNts$z{=8uva8%e1oG4C#Zv-LG8e+8k@nle3E zp+Vu?0^aH(30LH%Bv~Hb@es;E02xGY;*8l7?(#jo!ct!Vi&@8T-fMY zTsv(Y-B4&x$I&sv{g!r>8W|(=y`6Sq0g8H}4f6Ywhg! zzT%daE#j+A$w{dtZ*CvdOrc+FTE89JX=!f{vtY}osE-DyY8yPLn|klGc52Sy(-3N6 zTBVuyc;|*=A#li{{g0P~a#}cD&FkY+%18qV`!a(d9{NeX5ry zZS$xyjKG{6oirrp}iw)WP zKFhDH`-hNwrj;W~A1wl_#kz94ZayF3Eeludk+2ng+T@$sBcG=nh`qsyg~Me*5vVlt!OG*(0n|RA3VNvuD@$P_? zlUMtB25OYdlD0qWS3d%0Bu~|Erz-o2KXsPyjDv7uTFSdV}N0uur}{D!x_+_X&uOE6C(oip_&A#2$Xu+MUF_HN?+w| z0?FsK|LFW}Mm#m)g+e~J89g22q}1xf7dhWHHoVu{rEFetrY_>(W!BpGYBtfRvmAi<1Nd@H++eE(*UYpxdo_L-NURm4O_GTlP|OsExXmr zIB#%w)b~M)AA!%3Ff?mDv&wW$il&Qg3?659nift=FFIPL$%M1|ogI z%fE?=hVLoj_}OlnT7%}O1*`J58;e#Y6K*O(Rx-kRoR|6NhwXq5>uX0Es2J7SVu1>_ zSdvSZEzBz4|Im9;s-ZH}>ku{kr-gyQhr`4dOckj?mH2T0heZKGf@I$d3;9rY){j%Y!v*K&rF=WOn zV-09v9WKOyYqgR7WlD0qOpov&K6&S^RTVu{?t z^c;g5<|(t6BG2V3WwJs|lOIB>jOzu;E#TJ{tgl&p`o57o1LRmv z$K|=z96EYJ(BrJNVU~VL=gr5AwsAI*e6|4`EkF;*SH#yNmW#Gqulqur{Y_CfbTn+4 zhcNLEs76BjluUo{{jH`Z(3#+jjWhKhydS+*e$hNTOTM(L-V$J!U*P{k%1uwAPsvkR z28TchvU{Ld^s&_2?RN9C2`vwOw;*YP%#$~(2njj&NZxL;xh(a@W6s<>w0)F2bz-vO zuu;UL-%I{3ZcqR$x?kb9yx28hhu+&J;abHa)|>UifyDqiXzb~U_yKrk+wPP62%Noe zmRdFY-0S;)-&bDou|F>#MbSbTa7Bv064tlj?(L(`HR@%S++l+*vcO$uZ8D$bopb&m zp)fPY25}{60EmB`#0s>QR+`@A*>7(WzBmaX+7C;gcA|&l4>L>9`c~orns>ur_TGKV z{Yyz6E1Es@v3s>GiTagt)pHpxU_<(-ZC7{tLFzHXlOv90d5DZ3L?^0b=kD!c(ON}k z$Fx3$o&Z5^!j;D5M0Ht%3mZFLLW;&Q0&1zrK8)8F(AH;rPIhb*3d?`CvW$C2E}{+k z5*~m^I&HW9^i_0mvOlC4f||g#p%&$H6d|BX6&tRzR9BiNeKR`kN#hL(q2XhA3s70K zv|GT8NR<2eI_6T2|$=7oRJXG zwx{VuT0etYEh4_5ro+T(Lkq&lPRv!PYCz$_qrm8lo26mUqFm5ZF>;LV(IVq!(V+CS zoo0y1CkGvLXmkY@6OG|j0Y=3vz;k$Cu?|jqk{ozxB_V`>F!TC~aGw~XnOr*st%*y% zJ>q$Vv`K$`HM>=GFH%j7bg(Q~!F(L){Wr>FPb;gpYor8=`Mx+~Lc3_Q4B>ON$N{i( zvhR;m}GXg_4r1$O~T-PN(**{tN$u1YrywT*S8fzr8%|dmo`LS3U4WQO+gl zG~h-H6zX4!1eB`f<54>TWJOA-)C|Q6aLGhM?8qf0(}`7SM*#w2>m?8131G$wPZh)E z0UW_Fx?@3N+vcY-c*bZHgVg8ta%z;*;&I;n=8#|ksELelH$(!mrge0V1Tp%c-b=Qs z#(zH)4gQ9u4fvGSrGWD>kzjOSJZdLy)mv1(BeBi6?GLI{f_97KVLA3BG+xsL6!N|Y z#tzt_zQJ8EGZ9CC80X3&KH>Iu@Q zmWq`ok{f#-g%5?Z6tz77O7AAnJ$+FyP>0|~hgG4`z_3HO7*_HMi3KO}4t=RbcPX|~ zyvT7?DEGR{XdcM}t01{n-8ya~etRzM2sI6g39{!vYQ#-i;+!u^KjiRaZ_eL`cY{Ds z4qUT0NfYid%%m&CRztS`rTS5%*?|tudt!ryYzv1JuUAuasF?x9(C4{`%>KlxAokO1 zTRkvJFpl^VkLYfdcD!|weVOr9J3YsVDh~7KBhCV_3Si|4G3~8SSEmL&Y1#+h$&R@k zTt0uixU7+hQ^c9R7*wXg#MhKbtWZpn*_|%DEczR%$S7jvdSr_38B<@s4M=els>eGT z!(I6T=+z5YmMLQx5x@N$+bTMC)v`C9JRzW- zqSj&_msgciIN9e&R-*m9R0O@BiIeZ$LRE`(KSdU;T^Xj_>l&MByG}|cNO-dbXXu-I zRz2)tm87Es0P_I)x_6j|1mLAj#?T=%lL|(Wuc;k^3mw5dFn@QYt;(IvSv`5zTF_lSiV_H(h;=D|z^yi)*sDdy@U8TY;2 z*0Oz3kL0cXV%g6>E_P`QU^fCwIrD*mhl;tx#@gr zss7J!M;L80NxA#=sc(5vL$BCj&GJ+qk}vpzL6gC9$}H(?+FD{vA#><$%<}Mb4e?!F z3eLV(jeUySWhXG7%7Y;5&P16s{g1SX!q4}X1^J7AU7KiTTyL=`SLY}mtC$TsP7D_S z7_ktbeXz@bHZhWK;HcxJwG(?_iXcYpi(xC=3du%RrX>Oh_c3jkYU!8wiqGGlMDD=; zpk${gELdiCL-fquWUj0tBl|Sw?GI|>mBR*8*w;52=!m7B`O3F_z?Wg*;NkW(#+II1{FhwSdgTRG_Y)KJ1coq=H=_~`*27CcuB^hu7t>&;A1NH9uvw2ft}3>)up zPSq~?Gp-K=BwcKCxB z`BG8hc$YpV0po#}^%=^uz71;(hywr9oVQD0?nm|3R-6`w6FTz%H_ZHMQd7QJQRnb) zK9*R__u-vXc9I`XB9-ecRM%H7HVdBnc#Vweotx@~*J3E_B7F;IVktKG%Ew0)H;q;` zN4^J$&3YsxWhX+nN`hIsmDoCq8)B+)zR6UvS=c$))S@7;PqOKW>1e8r?nnaZkITqr z#jneUFLp-V?OMK&D}D$TQGqN;vIonZ91gQi54kDvOY1fJBq>zNzRJGzd=yj4=bZrm z-nWxkR21E&;##s~$ZpG=hvO@^qiVk6lOb93EY>0@Z2#!`7A}8>b3=I=4Z^fPa3$vX zh;rl*|^2SE5FoC+@>`u?N)112U> zGbthXcU?KWB=s;irw2P~Oz^6+B?ozLy8Q=UTv8o)U|yX1F|Z+@;l~VEi$cEyVFIi} zXv90HkmHr?!i^DOG`g0NT>+awSDHV2@WE%jH zc;eMp`qyHvTj)yIR4?!jZ{HxR(8lWWy!qWgjuTkr3Us(^+W?$CDdD3|G;ZXth5SU> zvww7fsZ;x{ZFVh9ykvK?2W5O}>xYtwNP5pUsg1xHiINiASxe!cGIP_p7ZwONJQqy6 zc%li4-0M>yz$~61IqA~pK59pWi*f6)Dv-vmd8clb-J`~PUlK9CytHMlb*dXaTo z{2Zr`Tgvo8+qb-npKOMGe?L%cjC?NgtfI5+q3E~p--w|K9pY8M15j4nB8A`W-Lk^( zX1fqc-CPithRGHUW$CnWr}HY!%BB~e$4b9}=KAnP%y}5k+rc40N6yvk*@~XIluX(m zFkT%rW3uFw1`{SJ=p@wM+27)K5XW8zGD9RhxcS6lc@nd%>etEpX@({4RpHp4gC~l) z`=z|GGHW~Veb-rxX#*g&OW76=N%Tdni0(`IzYW%i1>LkKr>Q@=U5O;dV;JYw2MT%G$ikCvE3M$NbqtUN z&PA;)_JNow&iFlBWaonN=&pYz*{|-gM84}B+qTYr#mNUNnIF&NjnQ_ewr@Yd2SsS9 ztAclJ8YP&9m~<#*vu)jmzp`HBrd(W!`O)xfGmKU2saK?u>q&mUach}c*t5kTj@?ki zn{t$l+$Y)-5Ia<#jzWE-ppfaoG87K6m**-Z{sDzys?GJX@Ay)2Mm- zRJZh+kHLBl@?dqdf@gCUTEdlG2|lLZ>L0!Cx=?cI&l)egd7i$njhC2n#iHOu^DgN z!4K!Xr6tz8d_ko-ViWhEhD!kB5-E50%13{A^MR%zxz)VQ`D(&W@!#aP^E^Gyck(`} z4Is@39NDGFt5$`mLSj0QVLy(!`Mn5_jB0tT)%Omil8&G>&1LzmSBy;kTHq(8V5WpAy_y*N035GGjA!|%cp$XPavfOAK=@V`3R&wP9n>O~6yK?K z9e=!ww(*VYT1kev{t3H}ZCI@4g(hEl3I8|s|EjqEDYl|!4i`j4@0@+zj=$@cOIP&l zjMlaPd7P`duQ7Bhfu~v&=y;--+~F4RC?0`ol&y1Z6uZf^wp4x4xY2>zu(gFl&ENYa zKLoWQzXNLTVviaffddiM;ubi_?R@Em*A+2xF*W_aQ_y9~OT1adVov1~J+lsmEPvE~ zSXFb%^rU6dq)7+=3rrArDFci*0$-y@6F7>x_BayuMo1ioS^2o9+vB9E@sl*WDzb)pm*^*2Q=Ow(8H`q1& zupm~>K6>_J3cbR4edTK2CO2{3g>GGdL{E z_p?Hfg{R!jaC&yg{pw_ARaKFFt;MSN9|rU@f(>64QT30i$CZWkY_p3F+M$d)0RnQW zk2ta<1e9JhO7ZVX8_|+$*$r&>$%!|f*qds8oC`#WH#T@G+kEaXaUFMDl-;eL5;AZo z@a?S;ZxM1V(eo|>$?#7h5S>b+5ns2*D9Qr0qWdDhYE=;0wT+*bEzfF&Gb}gWia5i^ zZ^Rkv{QMFptA@7L2^gZwezL>gVd`6JhI>rQB8q2ZOuo zPYUzOLxy)W$U4C>J)s|6Kg$c6r3w8=z|l@ogL|5RQ9K?K{Dy3}l5Q|+8VZ7>z!+On+8S*1G!pB!B6x%6GliuTZA40QIdoum0@ ztMbn2B)0U>WP3+^+VW}@Zr)uSVy6x8?n|S5GN`PiMEpvNFunuL!e|pB&szGW^Rh*{ zagkqjmf=UWTO`=zUf(-Yg46Vv1N}ffzZZDf{`Zo1%;GHPBL35p?Yd>3^WVu|uVh&u zT-DCC5bE+dXw4-o_4nMjUd?NTBk&9+b>l5&A`#wWnSne)MH~Znt@yqL^8ub9RAS4c zjKD<4d*&#SPfopaEY{SUq`PE^V2gI3cCdeP>zU7i3Q~T>KFX&wQ{2qcKTK25=FAvR zPndQN-jZW{m(P3fIMiI9gLAuybG^1Qr2f6yd8lm5IM7POEIq&S<;)G&j$dkdfC3T$ zJ+pi%L;UK~h&klR>Ki}1R$`Sznbh-6B8T3XzVuUwr{8$_T0~8wL zMXwzKL`a9uJuVnk3~%`>(|^ zS|j>78Y5RV$t{zCt(ES6Z;LFSnf&HVLyGXbI%JWdU&5{EyFxL; zPMbVQmfFV^Zy}yTOoI>p^R762m!0|YH}bDE9!@n&bzbsjn3nt#$l#1C^e}vGq-J;T zWoeh+Ng*|_1%yL#l6G<32eUuKO;#Rmsxt!pvys)hSWKiY8bSR0@kWWB5TuqKoRJk$Xx+2{-k;= zufDHPZE67&g!h*#@VwhR@uODmGvVO4Dh>REPOS3~bUFB4cbX7Zr@KP!z1^R=5uvptfxrSXVb-)3yKZb(p^Tz4>j`+fUb%x-HPbX}yaK7+1h~ zk32)0fN|GNR%8c?AG)%yN(KaUomEYQHsc3rj4XH)T*7M-oR~Vs^>}>oViWocoy$Z| z_}vQ1wU)-;!*MX)1r4m7K-%k^pSuC}(?-@hDWUq>ELs6oq$#o;yra#Yhmhq9R8Ay& zV+kDJFKDXkV{}MPrbLAlQIXUldeJqze-pdT&y`c=e`8LibkEb`dvf2i!#y zCFr~j5N;jWb*@ci(Ncb`Q;dQT+Dgc3u<;BA$_>C{BNFY<{GqV_$R>boUL!}EpAKHY zUZmiT)|9ZwrNK}8x>(Lq?u}HoC@y`WYz^?q%*%`S;Y+sFQU@ri)KZ!Vl(&H_@iB;4 zb~bR(#b4!*ZK$U#RRcB-!9Vh5cMA0=2fc)bko2xAC8f+lD<1tk`A{92f+m5BzM(=nbgw*Rm~3kz zX^`!pFG$Qhqmyf7{m0;YpSh5&?@?bFik3Cr*>Ld2?jjZukr=dg49c=SZ#kIGQ`KW@ zN4{@?lOT-z=(Ez`aDheD0}txNuw!1L5C=>sMt70r<2WSprbfIX=+7a~sYT5YmL&m6 zU&}<*!ScqwhywqX5TAt_6FT~f=cZLzn%iVu!RJ&sL;*aY-2M55l6g<2I|{HqJn9#i zjWmV*tELL556YhWFIuKFNF^w!S(2(v_s2A6uSh)l^evvLtBrHK z_VMhqM4m5~ovxg>j+;*&3d%h>!6Fhd=V4;Ay342;?}thk@^C*4UUaAw&IO+zzsyUj z$6~g1ahNM^DWtV!t!*`F{Ne%^r$LjG{Va|j5lV{!(i;PC7PUJaL}h&iC#RMAH$IRw zFe}(`ct)A%PIv+gq*#id)Gf!OlNDpr#=IYa86$VTuUtC@bM^GBM0M01B zTGlqA65vrYJXoZyY!p(vfom^1=PoKkOQ&tcafDuNn|xj0l20 zGtgG`SPs~!kAsRSd-0}zydb5mJiz^zRm|+x&(=&EK6ni>MkNRUwl=X`Al(-Kd+=ar zzmM@sdEZQHGx6LVmVeHF9vRv@f+2#7qVoF&$7uVF6 z?i8sdGHwQ=swPhAG8?B@s4k#xZ>X27G}=;%PjO3&y(ocTUh=OIS%!U^y+>>QPgEIm z#5S3Y2Z?UfJfT^fK7^zha6)&M`19rbEVZG}uoH6!DulD#Ne`-56YK5cYd&0fuDp|z$DR29=Ld1iJHJH*|CHQ)uMdzFKO z&z@E%X+ayY9$T^Sfcv5*iP=&lGgqRB<*tv?YBDHRMB4vFa8F1bW@(Sj&9X6)N63C_ zrg?mF@`phUC5vEJnT-`wPGx(n&QY*am0B4x+=WS^$pXgDTYin2DiN#%Jl)ou;4V3MYna)8>k{yVaCjezbIdX11oqP7?L1ZhKTWI(wa(2f?660w~RmK z!Sw|hNWb06ti?8tb%QRbZEP}(wwf-G#JZDjKF#BTUOwfeL19-n)gh+SFb=+ip|*gz*%9*vNwigJ z&O~$gUrMt@K82fF+@=H@=IX>P@1n(9bHOcRUc*SoI|6lbfi!MZx{3gyvd^w2&OmNM z!U6Xt6vnXypJ@{y_^W$OT=y9%GW^l#{B$;3Dh)u4)FEr@w;^PEOUY<3uuQOYEoH-& zUU9j$y+n%TkfSfdM_X(FRR?bfFtv$64Ujlcb##-gz}?|=?z*(0w`0YBeGdCkk(DhB zAqQ$HnuMg+7fcZafX`xhMhq9UjVqznFE3O{AbEg`+^tcK%pb`I6y>}<&k=jI1nOOC z4txnPw$d(aeh2Q;MYSbMY#fh{83S=eKjw8d#ZG0Z06o*o`wga6E&lxR+oOKi?>pya ze=OY>`L4QkQeH(Z0XrgR${s^5!6tI}Vmg(2+W$x{8!8(nBlxNn5r;j&kZuI&c1$nm zT|SNTc-sS8*E;r8I7)o&Kzus%ZG#I)o1@^a?>{=`*}>D<{cG3)0xARCs2uJfr-(Z& zv+j#~0AXIYu)jrM9yd4VykC2K^+$(el0vW5mmIpYnE2u8J{prgSpzmS^Amgp>`rC( zk6>pVx-Kl2B+c?1cxTpUmtuB)lOEcq8{f=7y1`V!egN50=UIy1nuGt=AqUZ7`#%@D z1T__?3)EjEY52DS3MH1*W`nr0RYDXa&Am1=!-GKE2Rrs3m%L{%UFMy`9gE!F_g+B_ zybER*HWwW~8Mt_>{#?)u{DJ$Np2f`G9rPe`*>=8Eq{IvhxJ2!B9hECT{U$HHwueHk!5T<$3vrlyw3r&i8%gQM?; z9Nbs(h0p(2uH}p&i6Xa)$OfCZ<)fqRHU>gBq`3jV_ugbXNcYa>dk7;*HAZd1w|fk^ zyM=O*?4mS#r6p(NDpb0Ar^CQOCmY(?u5FhH z*CiK9jE58<&KHF@Tn@Yg?gTm}LEC`s{QP-5M-Q^Qm3>loduJQA_~1S3AG<|OVVmcH zU+-qze;8nX;wv+5u>+L)y|rf`l{mTrfq>S9oYBGO;qH$NJ?;@|pAz_4nS@wL=`!We?i-0ubDtV3u46iQKIOD@JKMi2l1w1I$ax1pyUTUY(*mW%kj?0ezM?#j^Y6uJS=>`9=@7Vk+nnF* zmFkN@B`QOo;0^pre@5@}lwiz-)_Oxy<}!rKGrpHISAqfLUf=CYyAG{@NQ?1fkqEsQ z_Qq)a zdD0gJpaC305eHxUM>jlpz1s{tu3a6b$O&g(uZMMu`hRe_nnI`I`+K+`w@#W>VW? z4~*wies_CylKdIS>>7oAjmBtK1bsXz?hnx5?5?LjDp{LtKz$gh9(IwpW||K}YP{(o zKQ8s12aScdrJl8CQJ}ADX0l`Ng=abpy@s$*Ai2T4XTn}|8q9Z~d~2%f-Xy-`z&+3B z0k|1^#dH2*Gj4<4tdD!?*2TQQAIu0|QUR8g!Ah;cgY%1iS)ADG4`9$D2OYD=capXy zi{y4y+C+l6mJkPpcA@%^z_Rm~dA9Bc&W>Jd^Ah_rl069-dED@rb)IWco^py_GZg+? zrDUL~x27P^G_mwkNqqFSym%N{2;<($9R3x9xT=&elHu>UCwpSQh(IW~_L`Fxx*@J- zTZ*cW=ze#Rc_~13bBtGj5$;wZu?Xb`fooN~p?`!hC7;+cr2%)Q{J4HzPNi;akjNh(Q2$W@85dfrY1?Z@uzGx-kz z{uT$bCwM1j^KldPjdzu(=7V2$Zu!^QHOh7z3kxZ-@@Ks5JfxbNP9wR=(2jaEHRZ>s zs92RZ_7Xt3i_M-+T3c0zZEyf9dt{Rk)#p1ti~4fpNxXWgcy%q-fV79?P8Qs)$UzR_y8ls8`0*ZapPi^u^%8EPRh z5ATs-ZKw?5x{)xq-H=@vs&z65-vNqOpGzp2?8INUIU{&InBnnPjyDlC7^PjgLi7nZ zd{Q=HaHiTN&L=zU7?Iiq7Q0{@@V`x;{8=x_oY+%X8+~Svh`zhu2TO!igeX zPo_lbf4Vq5^z!CW;Fe=~G4+vKos(7rs*zy%<`Gkr3JW1xscBya*F?1G7O9VEmds52 z{AI1JN;M!U5IF$Z5iQ8=9NFsEx&1|QPMjPo&GR^YPn;R>gPi(zSS@HIlvLV*6kflkJ2oD96GOtbv?HYaA?C0{!;K~ z3wc$a&O*wrd||$E*x@*L9^daXS+Lf0QC`uYnqhD!X3se$dScV`+kVe2kz3+csn^B~ zbfk?}pgBQuIDvyJrb!5{Gjm3x1DdXs?jiQ_NOPs;wJodDmSSa!Su%y=JMzdN#F=lO z3%%-JY*^9_$hW*Ms8W6xY6q^Ihp`bIobEFx$P?lZ1ShhzfyO67U5`}WrNH{bd6B2C7zI{f=L zm$XcHi(cVA>$y5iTmfHP5m+6ZltX{@FG{T-2v2e-f0g5M#TY)HIud_c=Iy1EvDPe6 zIFK73%=;FS>TtdFo^K6bT1HH57NMpq;+dn;VB<-Zn&M%dZGlEeN@-I??c>jYT@Nr4 z!`cPoH9aW4Wp7T1?qq3OWp$gY1!q`Jrlh^QQQ}p+5SRXSgiY>)K~3gJEPJ`~P#5B= zmsO`&S+VoO2)2R*<|CqFd(c+ekpbl=DL0vjVC^fuC#dnBeT%t}6M~KL-!SI8#~f)) zJs)|^A^lAZeU3Un{c{H4qJs}%+RsK5RE#~P(+Y+B zZfCDTQd2L!RsC$v`|w#(GYbs<)7};cI538auD91`j1#O3Z0R4V3*-&p{8xdB6Pq{P zT&dr*t6CS0c~P$}X`sjS-`D&jt0(DrT<%Le1lG`V$j9VZIxqFVS@#t$%6E*vP}9&F z+!QH96(Qi>O8@YvW9DcG5V_iaTD*E=v3# zOHkm&a-U6&wE`0Ay{1qx)v9YmKBgovG03#9(jUW#100?ISXe7-4p32b6{Q!e0H1J{ z&nhe@)JyBruI`iiXz$0Gya_(rkS_Xt612^hZtVMw*G=I&<1Z~?I7D|pO{l!Q0W<(4 znBb&`(AVG!Q1Z91e$f}C&Wh%X?w`YXm4a*LVT0P2s60e5EfJ*iyiFEaFyiX={lZIo z*Ku1{*5ExWF=61q?AH9Jja3>NM&Vact&M6xb3KAgqj-;u?ley}%sE-4%H?Uy`>a*$ z(;@f%_)r}NK*?(|P87fud=?0U+Yy4kIdpqTyMF(AMuXP@)xM=fab{rmFOhItYJ@L> z6w4dAmCwFIe)Ne*J4ua`9q*F5DMZF+s2f2rh5VFt-jwh6e~e$^p`Q)3rmEK*>lZMu zXO&6GQFQ4#mmXI=ZztR@?Pq!+t`b*|3WkSV$L)Wiv*>%w2@J`5C00|_g<43HbHK*= zDKjWj4^K`%_KkY?V$N!15^xe*(6pv(HJM`MnNM~8o6^~WIuVb8-?DOyj~(8cM@MVR zkpy;U1798A6A8|TUN}xj&Leve<~EaPu5JJVen3(jZ8#?J3SYAJQ|gmi;hQ$lQ(5(w zZ~L#I_XtBMXJd^AqjT^PAR&RkSUA27WdqCBi{G!lRX>9 z_zmFH>IZ;4VGz#YZImvY6$L=R?S{-@84@`gz@#`9PR%Lt;PP zRk>5=5lTzO)AGXib6!>P#RMydX%&sB@+qrb?e7+8lj8ZoyAG-8*K`hW>Fx=8vkq{G z*pM$jL%EovO=q=*Mw?bB%U?1ryw_6sgr!Csl8C080t?09JK9=Q-U2)7VOx}(%uuiQ zQ?to?b>RSz)kMMfvN87E0#bu4DD<%WdY8a~T4aU8)47%#bQAJv9`Lu1o{Wqp4%=BV3>fL8=$5Aa#n8^x;p6l@TwC~Wfh|;p+Ln^f0`|X0R0E@``|?-Y zgRHB_@r9z;>f-z8rwgjpR|B0W(ZC+?o#s=2G~2qlZCgFV^wCTxS0+!W7xX>URbkSk z-xg^*h>tkH3yW5zebet*C+}nNJ>2b(ldW$@_5~w|ffH5&KV`V(x#CS$^5`!Z5lrbo z(S9nX^%7VxFzAh6<51rh9nj9VB@g1G{ZpPxQ<3pwRMQi%HK8_ey!JKHN#YFkDPtaQ z*=rV+WzizAuUdejX)yaskDvm6ek4lq`Q{P7O0JxG5vlp7=;xygey|BB*wep0Lz8n+ zfM$Gxy76S4}AMU`SYz^BHcG06W00;_?2WRUm%LSr&&>dv19oWSk~8Z$p- z_}#B#iYHH#pVHwNFR~%govH0wQ;jyVQM_n>k|Tl+SuS>JtLyz*?8`FSG~+Rde=q8h z>lW0tTz8-6zG;p9i`mYC=kMKD5(2>s3h|mefyK^#)Vqg<*a5^l;4;>#%3699_$7XZ z8nVW%SJA0KEoH1Z#dsQk6Z z4hA|6HZ|aDLHNPO7P=3iPjD(oy`%02mDYt{gLfN5G$~N~nuUzyOYDoDy)fYvIJTKL zoiuG%)s`4K=##j!G&vK8mu>ArJ2WMC!MMja^5EG}E37tnI$fc1wKu7?Z98;{Uig!G zm%O(SwK-RUdX7{Y-B$i)y5pJicT^FrA6)~Ege;+>=EOWz?ELEc5rTg61iQoof&I*k z1r&(-`2wP-y=W@zn6U|yUQ?-4caov_LG|Vtd8lWI$zF&Ah|<(=*J}I2E?HqSI`kUh ztCM2Bi2bX5tN^tVkg%%BD{WLEA}}QLA(~b`QrpQA97;X^WZ28UcP>j{*J0iOw3D(ouRvxSfCJuqqA?KyIsBrd406R&-|X^#H~HgOKj!3)2C@qzsq;nxW^0} zfjz?da3ouWNesnk#7kQ&Y$#24?DNB4osLUE<2V1{P_4LJ!~_NyU(!^V!^iSX+4Br* z&K(JTue`^?f)cFQ6}9_}UnoHPgBeNkFiJILK~W0cLXSs^FYZ;W2_HJs1{elLWb}I| z=>c29}M1y^}Fs^LB{UaU1OH#s23A@J?sf?0Xw zJgep_L4fFRuP(Oum$v2Q)ZTmMY%gx2fY>J=BN69?1MN_gzuP)GI;l>Mer@GmeH1!+ z|9$Cn@HyCI!w!*SvoYuX)8o80^TQRhVWEd(#U&+EJ)`q)RRt-CR{?c)2A4W_qkjM4 zN{sS6N{C+oMZ;PBGvFPVbI!&WuBDASxSA)(_uD)E{SJ{3(yc-qq#I7|3Af$bWD&U; zqg`$*xqL~pyD6jYdCC$Mf-P*J4Le{C0D(~9iwji&_`FPj>Dzr)V!cmnjjwVN9;6k>Y6?`(0_o}6EowAkRRVXsi{?)z7IGo>$0 z1|w?F!0b_s@e%4Zcgnz@tkgzH%XZND^=L!YpsOMNk+US^J!yQ^K<3_-omtMG+R~)z zL-#51hut<&_Dq+lt z1*QpGhT>7w%g$7)*TNhKDPM(dS4X#7*42&d>@3@P-8tZ_QvH&@`dy~HTDFWZ@n@0V zfzcKO*!U>Az-8vC={avkvZfi}3BWd=-9;)91v|45!doYB9dCv_K!-e`P?|ET=+Cbybd3-B68X&*$ z?Ne=!4-E#h29%X+bSHAg+s`aUxyLosvMwQiTb}Gy%1qYz0k7EsZ{!_?{_dOVpx?JR zieZ$Z=4Vhm?%n=G^n|aKx__5UWK#aQe(m*Itqvt!_fiyDu zsXbAAGopA2e<;KDgucPB_cuM=q3O&I>x1$$PX7on+h+G1$ySz4!O6gYRk2TrCX}c zlY9IJi?1Lr%NzN)<^_&f4>cH{O)*M!MKdx-7L!)nOK#I4q6C}Eb6&KR0~4sxNEKOT zA6!53Wbf(P@@06tb<^yQg~}PLXU?Wx6rajS2)kcA^@(E&-&$RfUB!|017P^*Lxsua zuhGu%KP~ueee-_$wNnXYv+OOuTlP!J=0B(n7FMt_4{rC~gs=^!9c?*?kDk7>+xK7j zr7G52!pf}4hKXi9;)xjdD4+drpnn7ZU#V8M>#(3w+xoR&%!3%kE3D7+72S2$ZiD`B zd)4;@4YS)P`OR)~smJtl2xQz(x-Isww=E%ge{PciY^*wg^qSv`sXS1>CXEgcWXwD;kIwV#-VrsgZr*E6$CpqHIV}2`phI7LP zX8XzFulc=H)0y6z9j^E1j6&H*GQRNWhludB??zwP)`%Go!P{$|7ms`G(bnLqVRx4z zuXbU2uc}JU-(&Usf4g3P&2(swtty2Ar?w?hr#C*UO(y>WKrb&{tBscC+EOO<%4%>I zEn0qW*;=@ZR{prU#B>@@d@x+eJ!4=(gMK*?|C(jX<}RC_>EuR__dk~0ju4wGyTFli zxmNDi7UUD2y-{7!1+CIbHR+JwqDO+&EF95ZXXZ?E)E}l%qthL@A|o2;c8@5k#u^z0!&oPHkl}pvFccD>C?o~c&pi+F1e$D!1pb*ZNsGUbf5a(jQSCOsKgQjr(6?#xP zN3^Os#(QL+8A0A5LOY=6`Us{Gw));KuK+_ti6wIjZRLmws0@uFWzT<(CQ&X8mz0MT z_U+j_74u3m?hq0v20srpQDhF=QbptUD008LS_3++QZ>c9kBZKRCW- z&zGF=MXDbuMv#|k3pRhWHt_T66>*SJ4;j?H0glSsIxcJ1oo5~WIYFfi84k_@B!<_v zEQf-Ie zW`^2{xl`(b)jJWB>edNpMA)9RV(zTRXtA=@q*}~s-n~Emaz3Jtncc}#F3gUUwZ z!QAUJjKUhUaU^H+LB9OC=hyyvVy?Z=P970`)xZ2{8y;oi{9cxoD3w+6)-J$UqpRGw zkp=JWci}E)`WpwmipS?IVz%EJ$fc(6zj&W`Z$-d%cMPZ!GUfCR7cQdZ|9a=jZ*=hi zVW-&ADdu-g?yH8b4;-BpRA4Jq?txyd3%TxTI&Tqg(goWOcMM(33cW z4|w&r$*)k7-&Lk&%Q0P(avWc`7xyNH=vSyv{l{D3gKAKV1Wy3*kCgCM%$2TWGbxFW zR&a>-QYYjSS-&zXQ0;EI{pjMOCvkpShWq?YH>fEDc;uw+aY^F~F&AAQ6rb8Rfg>}r z#lodVvW2}zck9#2yYC-^{q=%t9*A$$Gm>H|olQ-0$2w4Plh}6S2dzc@YoxDwXBWsS z^TD&#xvRm?CVR^jZ71!rf~0*TtOQbsXujuRr~AD%Er{t~HaCkc%eyW#vjoEyLGjFs zNOMLV(jSPk9ui7JHChD|vR>55aUPusJGgwrHAs~u-E9E? zDKelaO<8aL*P0V$dw*tsj8!S(Pw2aG>Y9DTzh?Q7G?Tar^SBo?&y}Z@c9*v{WAsW& zA@0xjrWe#?nkhA)?OapR{E~zIDl!*-^el?B0Y4phNbE~MI{DVyHwBytj6O4Lwt6x! zkPdijA{(T3?gKk-rJ>d(D|eel-eUtII3B~%a8bnH*ka2ZE9vlan-L6)UB^}7-^x9E zPKOefYo-QDq^0(4>2$%5_KzdvdK{bq?ttX_i`ZCW$;Hl~*@r(D*2e|*ei7yTy{_ob z#;g>J=kb_Q(fiupYZt3me2CsNdI{9}oKS{78lp-_uT%o{1a|6TiA1*49B4=?)pO&-7q zy{nJ9>zJGsHVHm+pl?NHkJUnpH#WS%FOHBZ@vR_H&Ic4qwEDNMT^YlhlFefuC-EUp zzVh+M$}jb&BZY~@E)zLC?q199k0a0JexQrcdnOe{}O-m2PXmn*N6gXp%!JO|WCa#zrBBMccR_ zUdgY!P_F&bX)BKB#0v-&rh$xoazX%_!jG?@fCw6!FVgOWfsB|j(<0*rs0xh2yl5Qbb%$7E1Q z=VER6ay;G%e*!kpk2R_YS~XNz3{rXi2lV-eXBZ+7M*wPCs3*7-Vt8AUZ+e;Igr;@;ouq_{kI666Dk2+{JFI1%Q8L?WdCm$UX)O zRC+}*5~1#WS7$XMs6oVnPM*q^o=<(uE~YrBr z7*_8RKl1m$RBgQf*>PcPcOu@FNvHFXeS!CXW}A)rYU`bA^aWI!nq)*U>;gHD;=qVQ zdD*F@bWYlbv5`G%!IP6OSx@&&^`N&gC9FAa2K{oWOy5j321XOWR~cU%%BI~;ZEaif zmq}p10e_BEq!y526z{TXQb|3su2r$H&*LR>e4&qYE%voIOt@F>;{JK?v;& zf!=r~pZtjnf|HqFkmFf*N78P5i~_&Lwj$!21jdv#R7U*uv^|SIU#=c~Hym9pyPq2k z-CQE0+c_TfZ=(o&WH$fmFUzCWKqu0xW0!g|c>Y?gRA604XXauMtQ1MvY;1bcO!pvv zKZMm(e(mCWOz8;eYg3pUS`cbox$hBz+tRwwIU%ROkGb>mSh@W=3lBh7F`~ zyjLhrEJ;p2drM?NerQ+Vuq2j<<_Y{YZ0G);tQA#c8JC7TfplSHgk6Tm#$Q|m#Dr*) z>WvvE2x=3T6N>(ZG@(|Kilyqvdf5(cUpv*e%n(9xC&4Ao36^WcwyK;E$Omsqo$40%v6trzp zA!4cnc%L-O*8E=X>TlS%unWj%WH5e51h3Whr3iGKZ@^Wc^FIB)#r;Q&eCAoYS;9sO zxWXVx*LrocbiFQMdt2%R$;N=^2-X7^27<*rsCGwD$tyrJSu9FywIJ1z^Pb&!BP$vw znM`3|Ha)%tZs@38c>Vbqtr0@Ny`TGKu*FA$w11?w5wy#EJt@(R=l*LZPbEOPeiWCc z)s28t>M|0F2{YCX3U<|>|7^2G#aLZgHz+F!*Cp%qe2p=yv1PE71Hv2!S#^kc7ChzA za$|N{VH=QLx1_*Tn?h5TWDuP06_#mt%3yF#J=2z3n_JCW-IVz??>a)Ib5M&3N97@h9MfXpgmggc+I%ih99_@hr#$C{bv5vQr?rnN) zeIf>Zxt!P`IPiOfowrSA+^kUoVbF_?Grc+cTM}q}{GVO)qO1`l%Pd9T?-g7a=|KuG zifU(y4@;Ldq<GGu+!?ih>|txW*&W;^JkPW` znj?1;_c6Rp7aap5xPH8>{uA3;qQSam0(6VFU(c4ML)46>M6YH5r{6*v2jGxvjKIe= z3H5>EluwE^4drDFeg_sWu-Th20aANiXwTQ>Qv%tEXGBa-I8EYoo5bmkWY@R;Cx+>= z`Q4A{&!mvWo3Jc_eaT`hvkxapP{oMsOAl5W8iu}T3{gS`?BT*p|=*p)^1X;V7sIj!$ zugA|_b{@U`J-@lRB<(WYq|a3r1>vHau27R& zEDU^dO$L&+6!k(^NJ*65`&k0$Xw$zp=LT@c2fLBk^C(bY5T7*Gh(&>z?6vMsJpIkv zi_z0O_X*pnq3+(AH(qd6Ke9V=$}yuN$nXW|CV|LWK$jo&_LH^x)z8+{`(%|2P60Az{mFX;kvq4Hqw<5O*cszzo$7K z@dMDW&t#u&$m@?eQb%#F1;>bJYTUHedWsZyvKCW24~|BO0H!~$0GcZyvdo__ai<&4 z3vyGdNK~~9-!Cx#WVB@Z`#6>o1pM(*<|wAj1c#W1PC`u@QcU9Cg*{t$R8)61eSoU- zICfy9NnoeW(735(*W{!npN2*vRHtdYhe@k# z0%jlA@)jRt(|)kh2ofK6vd1i}%d3T^V#M;UVq1 zK$E23>W-uxV~%wC27jLQXuvM?S^LjGVeCh!+uS&U@{ECP&^{KzgSaQxl?0}E=pKu+ z%^||e4{qXfeYPAK->K>MM?DvYl_x$1UJ^?j2#0{4LJDG5d7FEJB^O6B@ff!|Yrsss zn+8B2T8#hKPcbOrOSwi)N7|EAaPkBYZ}qXI8Hn)Kk+pfPGKnz+f57aSBiX;8g&MGwi$_uRw1y@#J-#XTV0m_c|L#6{%?gJ3gFh*clA_ivkcLPBf*k(OC)9 z8RRQcab^XVK+GaGC4fDoL;a~Pz6@XKqsd>e5s}g%~m0OkA~~shFb6M{AXj=Iu>wG6t;#T2oQ0A z3ht^9i4oWrL-SA<3@U~_W@ljiFDXu8VaaCXwA_SMpSg_ zX!|D5Gn3DbsfZzF)^9L!p*v^9Vc%xm{uW2rkKVOCN%-`%()R4*Hj2FN_R!J^;J>L# zh_J3H^$RV!uIDiw7~D#z26FZv{2(nqZAO*nrvGYBo!tzh{#@qMm#Zo%tf3SYI9=l^ zCD!fN45@%-Y;h@SfzRfd0_MwqJ73+a=P|o{(&w2{vWmr~sl&UfTlO)96d}qtYv{kh z|F78le@VYrE!WH}3kF|<|A<Ag6AT{-)#3?Q;g^mOjIuSj_bJ|My^(dwD{*u;B8a z>k!ov{NYI!_o)XD!{K#Ml40M}9E^Hlp&lMJD^+)~m3aH2+*}?NM&P=!=5Qvtzus5pt7Egb z8bo7ljrlIM^g^gWM({0=k!UfzhE=A9iBNl2Lw}8&@Ti$j?3eVQv5;J&kkvGJ3Uj%U|K=5Y7@pwBzc7O#Hw zJpUrJv9L&$Z|n4eH^-WdnY*q3Ra^eBK1%JI^h$R-I~`0Yy8+U??UX%>hvDO`!MbcQ z5x7p#*eU1pUtWN0H})HirCVRJE1oL3W#d|_TK}?p3---z>O*# z*_Wi|#8Jp&E*t+=3M53xCz+eBQ{n&ctwI~=mpFcuCzd_Xc6~uV?N)Z0Cic$P z1TBxfJYqG_8U>D4-Fq(TF6Lnf>bVr=BAMCRz)g%SB*bpHFTnc`Zf)-$-hoUz#wXld z@elJrdH`e1kbGP1H}h-HDu$e8vAiX55Hp~6pQgNqsns4IAU!YK+q3 za*uM#4D&Z%%z{Xl&;)KNM2=4Wpz0Br&9`>FXWOG9xoCd_)`0U*P!@5~WI3tSK6|0R zLF|WNA@!~QTYeD{rqn8A&yJXa2MZt_#)MiO&l<$EXPTL)vCi~8G0J%~Wc%*?VsJ`_ z!sHpR5SZi%>WlfHo~6c4AIzagLM4?a+YN>$g29fbiU#h!<4A?~V~mI;`?!cqP}8sK z5xI^f;(5j2M>Jy^@;@>!i zMqevYhM!*hjn?jn?OxR9%jGflA~f2fh6*^dUwg)FX?8czY246k-h4 zDNCEYRr7HPZzb^tTVH{SN;8Pk1R|FEL!;}k$KADeSg>S1``%a9sz#ejNEHI^%-x@M z?w>{7Z6`U0@m#qQWkV|KnQ~-lb-kbKnU~K0&Sr=XXe{Ijggk1Dy8Y(q;03$FBliC6 z??8;5hw*jW-)@n7*q*pN%~mIz{!OB06Qbb@B52wXP|bbfbpK&5r0K%=)PsTIX-(_# zHf=4B{h`)bo_r@#(T{iIi}7l=vDL@@8oVQ1*GcS|}X9$!fXG9`)-%U&NOV9om)nv5(U z17@WmO&JZ_iA$T|72#d`SagytHH-Xf8NRs`STs>r6;vV`_-g5IZ;F+4A`lGZ1Fq9} zSo`r`K;vt>5hTqWf$fJz+B$6IMS3|kiKIN1KTz4lH)AKBAeHj&MEs9Z+6@9>c*7xZSJ|xH|P$9S>;0#7Qw!f zkc`_SeRtI%ZWMa9#Lqu#@BRuzJ&D=yl+&I6@8*_WVcs{Vi`VSXgO!$X8F!<94f2Bj za{n&***m?-Sn=)fNau*x{bpG<`MVn@+=1Y-R{XSWApW0eig3sC;`c6{SMLXkY6=!V z%22$~;o+oob;0=YT@YcvI7D!Uqfwpux5{&LVP(tG^V+3zTZ!70MFAJmYtH)>b=Mr? z_w%M32F2TbY*MbZs4Cccy){rOzMjx}Zyq(v`^FLKO1Qzlyu28c5Y)Bhm@NHJYj~7+ zXXO%ih)Z3!!v))GK{rd(PMXb5N!-9()a-@uf!T6i%<(rt0uh9is5Hv%9hoW6^c36P z4d&Y{zzmzzHpjU?RvlN|Td0V+%1XbrJ-!A{lDtG#tzO`6XjW#MEg4u<6<` z+jVU>H$F6P%tLFtsDuU&;ET5LaYKDNq8jrVN&uM1fjbGBUIc{*rf@A89ar*sba2ME z5B6lhGC;+YIM&g35-C9X(R7uZ(|ns_`!UC1lG@|%I2k`Ljs*~%pN$BmjMhCH(v)~| ziT@0*B?>^n2K8axj~&i`I*-0EYqF_gY)ahOIwW(`lyWC{qGLjAXYu)s|AT+=&$Howa+&F!ol5d#6NffY z+l53m$b%hAt-bu4mwYuD_FX)2yYnhD)hnl^C^xIP-Sw>uQRe1!D5FqnOaI31h+!oQ z{LDD^Lxk%oU#YoIeAcdI#^l>;ud$KxUi%|JZsxy#EQbSKB|_HJWTLT)j*5eRQ2mpy z3%f-QK;-Q*S+6<#?yt1POl@6a<5(NKY_6jb;niMM;GS2I{L5e-qG^qArX7zuV4V-S zJ`04tmppmxJNWS9E&7I}m(f5v&vm@F$Igrtg>|%2nXO&4>;buolS=Oi7EA|DG<_tI zxTdl~#}q9=ts15k!8BTt3MDdrZzud$PvclVE&Az2P1>wIs?Nu_Cd8u)In#{81~bf~ zn5VW#qa6*|TGKHNu3?~6NctFRKBG>iUD7pD<^Go+5+}YqVaNg<@I1RT{U+N}OUXp+ z?=aUg-dz4L6_D9$$QoLb`Fl>Jfn(Vi;mjy>|GHC6YYxd;*~O+^8$2|sQLX#LgHR-y z>rDbt_|crW9Ba+tA8EU{WK15u`dCVR7}={18<^t;?lg0Nr;~?k!e*LLZNk*hz+_%* znwz0RR<5{Q{&;opH|W2*e5b4Z7%dT-Baz`OlcFG(67(pR4f&jEMZOK%+`v%{Dg&Bn zGHqc?eKgPzNoDnLGz}NE9X08sS~4B!=!9BDana3Z2PALeb3*w@eUvl+y>q~4RGa7@ zDib_0Ltp2hUH-{rZ|YVcuGZfb0jK?AQP`!SK>~@&wWT|YVX1lVvpx$}jy;CGiBx1? zyx0K2W6G<;>5tF_fe(t~+fKj6tWS})sMh~j-ZWjOU!r1!9nS4PS5G?Qi8cMbG9bNn zM69P}sF@8c|2MCaj;UK$5<@)*g={>x%CprT1TxNOOrg=kLw5)`b@m$7e*qCSh9N6f z`$dDguPmL>;=~p~U7^$~eR$OCfKyoxKs$&5VQ9BIcOBe_cKx388=G}$xCl{{+16gQ zPA=`-^p2zPPB!oXGHf|Gs=-iXn9jf9D!bCH**^EcucZ1)w;?vQyX(pZN7gZcw6Nd` zIG%0wgh|@j;J2C;!jfH{+)!OQ>@MXnd2{jGW_@U(XZ&v5fr1D4T<f6Ae7$P-i?4;!3>l78aNh@-{DKcLxnEQ`iIwU~F)d z&lnJ3iJlIxTOVQlm4T-3$L3wGX~7y%oykfRb4H;TFoijF@B`4j7Dd!Aw2eZ|qn%P+=h;UXz<| zZFG;jN7eCCS>1n+q~SVw7aQ0B%j5+bKXbgaJA|-$k?Kb5Vm|$uzF)hj^#|3hnYk8Y zKNyLAyl6iDkLBi^2vV9{9pQqgnpA4bl)jS)zaQ2O<t4ePnKK@>aXN}7%bR7wqJnnSkUdO|}1-W4Dqml0DvT?YUa zl82O>RWa!KhAXBjC&_sXtP%_to~gEITmRB%y9{goE=G`^)SaF5GoD2)0t zEII3c!$RN^;`oS%UFUH!v`6JlfK?^3Yw=QKZ1)_$yQ9^VuIaQV%z(5(NdeU8+^t-a zci~05@IBiu+Hd<{eoeDKqaCt&WN<&e6^c$1r^sIX^ZII>tQHN_+e9Q~VhQm9_s*{0 zOEJCKEhym&?>!7hxg%a=@;_4T;pRsp`3`eOjc5){QUNM{evpwOm{hQxCZge+B0g{h^IQ55I z0kiEAu716@lr+#cRQdyR=VtbNh{F52u{TVyRb!ghYE>Gg_!h{$RYf%Zr_5X|#KZb#FQMCEM)m3Xbv^<_kRAt@&rnQl$UH`nIdHD2@~4S6BIjuf1_)$x z9d!eEk{0qF;oYym!WMbO8NH~u$&U#+s5*lj^Z3knxAx19_Q(otO^PING_V&IUP}?R zBHsEd#QSAD42W7LX9czr`$XJnVcnE5vhJuMaIfniRcI@e-R3ub`Bq+sC8Q`@q|lcM zBx*Rq)F0u1JMS~NP>0Hd!xmx2;hma0s$LAhmebdd{;_=NPAUTNkopqlYvSboLWV2m z1d9Tnn@T3{DkNy0B#RLQ>t?~!H$UuG56ixKWpJ%XMrDDGUL_MZ3LkQpU5q;Uq+sgq zKB5^X++aw691LqJg~)%RgJ!<6Z5ja7l*26#BBEI##;5HGP49VhuDkE+#EsyWCz4GM zf5g2ozzQs2PBfjh9j9{vbBDR;+Fya^E&GcWH5`Ksk8&VmPxFaQ=SCU=ep1nvYH9DX z#>H93LGB$Eo8IeozFXK>Xqg(?Q(0EiixjyH77GmgZ|?a}uSrX2Zc2`##zm~j z3jHSL7(+m86`2R_V}(!eOS+Q)Z;nPCz~stROA4QwY*=w5saEFx_VzNsfI}N^A3r1r z7Ri{K1%n3y+OTo6l4R*qs!^{}SRQZoio*tjh2poo4`uod+j#2jU!Fu=ndPGbqb&|c z=_;kVFUP0n(OFm=lQ(QCKy}IQAB*6!KOq2|j9=&(2>{%`tr;Pa(L*hwQrJBiW)zI|{TrW)T2jN>PD0zs5v(mO%@ z`5;%vLE@g*qXn_ts2zn7Un)$Y0MJe}=r;2bCJI0|z^S%{NA^0R&!3h0)rPN_=QyhJ zR9|fua6S*~Jk%B;G-qI=nyGh?Z@uN@I$MpVw54O)RZpHO}m5)N|4}T{ z+>WwjJjH`5M7u|QL4_?KBIS^GpR=B#e8Hy*241?wE&-wyK}IVw5Qb#-^fTq(9Q1-* zU%Vlvah%g%4;FC#EwyHQp(hww`ZXlGui-mQ>4-uG(fJURenxaj2@(P9sE4T0T9kOc z)TtJ2sidCjMBgOwYe0cCMjkiS695HX&U}?N?|`NQsHJaer)4s#1uIy^>(UrM^&lha zM9~+y@ab(+O5x$dvmRaAipUdG6`D?%@PY}=vRJ=i=&!2 z!r6kc`uDxo?VN+!)`9lUSq}si9OLk4gtPJaAEpBeTi*|uMM!@wWLwX8bRxY-c{=1v zsj7*^6CFVw>YqN!eo$A;{Py;S;mN3n%I`GOVz9^Z~M;KZhvq_ zjw^jRd<99+fbDkEY)~!N0Pi4fLUMa~10N;{%vJOwnjByz>>=ijv5&n?sb>WkVf5Qn zHJUD6Vnq957~sbcUTGjXZ8*xyC}Tjg-0>FeN1rvcxQ!Jmvnnckn{7>?MtaYW&?ejZKYvh@?(CMpVp3F75M!rTO}9e^elCbnZNRChSXaA zidl+xm5_|OpNnZleRUX3TRmcXF7T0R-qoUgz*r9(25NFm$IB@#)pcZ{W(WPT)>)Yw z1_4rQav$%;zP44;D!u7Iy(#5J#3k=c?lDg(&>n~Vo{S$kjM3CS-ii9&eXT4fTv?3c zaF~Zo9+Q$h7!fVV780z0UiIfu9kQ~&S6Cai^AsxDrFuM?@`hK)jQ3D_T|2VgU@$k4 zY1u5KiQUlQa>TRR#v2*_Ct{$Y}`^yF&8 z>1TXfF)KI%h%`p0z-fehi!&DghW`Hr@;6B_srmykEVfsETxL^d6+6hacu>1#B5~!v zd+SCC>K|WoUHSh1KFmfx#;`P$6t|Al*cZFpImB@_L{+kcz+14ry{atk8G1v6GF(oOoePo_HE3F?8_wKjgahFCd=3vJCW>!EHfkfG9$|{OaJTh zz4O2IyXo=pnC-f*^E{8^c)gx4Go)oJufpK7m6fP$iXo7v@G9I6=N*l0t8IGPlnNPc zy#|?_>G)DdTEyE)3gx4VeQGA|I2JZtaJV9t)~}Ocu~9>fsHt!Wzr)p*E0R`#Tr>l^JX!* zImmOl*qN2?Na!v(XMXL0^4nR1Yzt4J~?47N7`A$Q>)e;rwc_VJ0Y}*t-&pr9}&?DoDI-}T2fdU zwdTl|eI42Pp-{*tWJ_*0E#PE!2TvBYCfNqV!io2*i>rCeg!zgdBI?$6 zdR7o`=M@|@uKAV!Xl`nI&mh)#$a8pbw*_)0ld8-J{T1b#zWnv1!VeH~>w86}1=VS{Y;AzGp=F@T`h>QpX4&LJxq4=mBjxZ{ zxmIyUGDZtrPO7CnjFj+ngS!`LbF+t@075!9tMM&chvbp0+iLO+H%w0k_aL@snmrxt zgrCJ<+~?I|uJxt=Xswov-D_C7oqZP8op+h!j&(2tU)j2nnK!~Oqj+lSvpMF3_8lr8 z=eN!=tZAKiRy$r*)pRNIeUfyTRY3Wc;-alw%*UJ(Br@~VN&q`z@OOZ37obzzkxb6f z@ey{S9HMI5_y|Ac+2(F6QaoNVl-wDudUo6@yVAh}lbi6LGw-FT${fE}gU|!T+APYa zjhKg8OSB8l;D;Uq|F(o2^zuI=XggX62 zt<>m7V1Pz!N_BnUhv)u$lI<{Uq9-n;94F7A?_PAF=|5U}h#Uoq8@bz1*2mGPJIS+4 z;~?1-y8jJF7uei*?<6-Xd&Ufw4$u1r+TSlRt4^Y z_a{2Xep8xx{0t||ej3Z5(NRGxZ|A=-<6rmR==k^naxBwy0-4N9Z@-XNbHC<(+s#fM z*?VSx#O=NIw{4j>N<};a7@hbe?lz1FKkL84L}z3v<0MlMUaAFxif`zGg9fEy*Hawj!97TzT6u ze3*_qpxR*B=9{eKN6yysGJ$0QMmD!@#bhhiV;jhpN|j&S3Ve|<6ed^3AM>kw`>Mm< zJ)(;9ZC3c@U7=!%Vq<1chv;>@wab8TrOnaFaU}u?s+TDIY@?6$j(z`K4$q466Xbm~ z@a4V6KRRj3kD01~xjpfzqc~S~F_+?WYv+7lnVDf<(!r9U9+E3Qs3sM)ZCe_Tb`S7+ z`7$pg$4@h0t;3H*ByI!8csebKvy=GMD~=lZbvEUvf4n!ex0#M}sHHFvS(D_m6lgf& zi}mq(S-ST@-Stp(`m~DxIu0h~lATB!6lc;OmG7V9)x^Eh)fmWenvg9B){#<@*~~C9 zy)~52+QCDyp|--WfqS9xSB71&i9x4Kiu#MrpBQ_vuN23*9un217$2 zek1ksfWanA8|wwQAXNmJVs|HIe$JhPa62)7)wulBQEavz!Mv%oZ97%L&5`sWjLbZW zwR%zada$(8LjxLnxT)&!!XoBkj7?5TZpB{+kQODgAEyyJC#TGo$@K#0gkWgnv~*7# zrf*?6?kwqx*_gFnAG&D;be!D*Yt4am`h$irnm9J`7nr43MOvH(X6`Z3UMq8=`{a^(@vqJJ%mgD4dI?W0R@5!aL}ALDOn| zE|UxANv?1Ibqdmp9&(?Yv8_u6IM^o48>w+}Rrtqjai7^I+AW8B1XVoBWka55ut^tx za(=efqOMmXQt`*r@>A}DHk*aW6)AgCkU_$nV4sc4_r;%9A~lxJCwJk4*Hhm>LYGox zDeWGs8DD`eukCsO^hwQHXiUCv>?^%ewCloV(7h+$#g8Mk#hL9vh-5ElZWz1XCuEi40(5H3-!bPW1s))h* zpBMYwAt1}7=5kH2m|*>LRd+GMUz`<%E3-G$Jp(vsqc`$6KQx>OD;!@2m!TpkPK4zF zD7*X!GVyS(GRs(L`yjuNP38=pJt0N^lxJO3jsTCv$=x>6gx19T5qKo(aY1-?`VYm` zkvHYK*6)eI;$o5RccB+w>U*4 z=Q?LzDsfT-xe3k9Qz7}_O=4OK-5r%C|J0|NFvbq`P}9a{m!A1O){%2eLIJPrZk$#> zodOz!Ill?7ScVJICh|FH0NSK%eH_jp!*=l@5iKUs{!F&u96{I)D^JJE1% zt+MW3B%YW|S*_;B9)){%+3tu+6ViKrEiW|u`JjyKJVSa=HAzlH69~|v{_D#rr=|BY zV#yZ8a#O@54xzdWEnfYtYbZp{PMD;C3JyYIEH}c&X27YNb)NG&bsqlt+;$Lgqb^ys zYZOhZpN?99d#1`)xe9it`Oz;*_q^VJ$l7n4j?Lm;tqkbf6Jd+D^={=V-<|tD4>w1e zPIN@24yjR-+=0leFws9Lgmji{H}&(Tn_1Y&z`CQ=pf$^mv>@atL_0U zjIiEpt&H1gx9{r2Q9EK{@bWkrWl@_R>_QFMVny5g_!aeQ{r)a^4&QsJ#PhMz*@dsT zJ3(w;B()*MATW3(bAVykr6V*xs9J|9v`{{E>KGj>u{hsr`~aDef6md>w{h^w{VV67ev!>dD3G* zA3yxnW}ZTAzDW}@`Gb@_z6nS!dwR>pldJea066_0-GU$O;x_png@MRAvP;W84$N16 zaEH443E!BhL>ztg3)Oc=O9&S(*6W3>3Y;;#)BkYJN?_n|nAO?SV4>qUq6G2_CW>Op zt#ge)>2aQa{pE|BO<06bDHeJQ1~l0sZQuilhRf__FpfMB11Y}Cd#4JrRu`oGC2b$F zo1*=7>UWT09_TB0RI^W1VAfw>zpWtvSqCI8>u|%z!TdVR9)KUhqOQCAhoYsE33CQYW9xr>f*{NJ7ZL9H(aSt4+uDA(nd}bUnN_U(PXE=c zSfUm@eKa^{CBIJ{W)eU=H@_#(yEAW0ZFnOkUqhH8%ke@#!@J)-teKedHp7wn#M z>ey%f(#6vOylZ5+o2b&$afvx4r8T{ccBXYo1;G%D+MOi$;rxe-DS;CRtjLTIa<^S7 zuu!eC<{zC7`lK}S2+0#1$c0;G^mr27dM}6Zwg2(ujlz;c;UwmGowa%jvhaV5@Gk9+ zD=?35A-i5?GM_OrZ2V4R2^4kG-2d)0(gFee#0$X22lU=v4^@G@sz5#twc1DZAm$|v zrc#|gIDGJz{>_0X4%hd_p_obeMhB|UTP{9hD@6TArRSGo4+^pD3yA(~uLzPi*=$3B zgnxiM2~*`KhFsh{Fg&T1*@njcqpLyg>=RjRCuuSi>Gtf2J9x4YCPuaPTI#BB83Y`n z%?OXBAV`d4dAHgJzyXylhp{)XcE6#alT~Z@#ERi-)C@)vuHhl7NE}T9>@1N?F+BW@ z33N~IC!X8`!gIr_-c)as8yQ;EPamCGvZdm3DvX-^`tD42bO|PFDjx!y+eyEX zU)OxZRK3vmSJh0!hrdRr{KWt}Y&I%IhY>!;+dYi0eqZC^?B(EUY$}XjdLXQJio;w# zM2a#=r|Ha-^R_!h#epoB3PTbaPew@6u80Rt%Gu*g_kpiA7lkFt^Y? zM#>-`59NYD!)Pnzl+}ZB`8SjZvOiIuT-aG1ISB=9oYosRn0 z@&2L0fhnd}|g%|6PEEN=Qg?)Dsy(Im>q3Yk*HKOQ)eI<@*haRvVA9_%J?xHut|k_AZF!?u&Zybr4KL#4EK{mDeo3q`2(C{TRo zVVr~(qDq#9_p)^Gm;GfXYAAOqVGxoOY5`&sma;t-42>)N9CDn+TxuR*8>&^VOIOyu+*oZ zT}e^9dGUwxj&Uu}C95wVLjXvm0VVq;r1281~~1h+t0gA04$%HwhrL#z-v+{Cz5SDf#5 zt?akK(gI6+fQQ_{|2E+tof5$3T&@QMMZomL2bgUFrP`K8N}`K!b4K?cqn~NMR@F}4 z@127js`jCiRy$yzYjhL-6DlZCLjdhUoq%t--m4Uyv`m@URQf)-?@M5yEj>kqw%>G; z#e~-Hp`irujo&MIT(odZCX@u8i9MWd_8f-aqFf<8Af9yn4Q9bh#sU|kXk=#Tj3@o+ z;qP}zg={~HI2|DGC{9?gs2lH+G~r>fXT@fwng~)7HX#0=0f!bIBzp`Q> z0(|)%?<5*$Gl9J?{K{~zpz2C*L*`iJxLQFE+r=G-82Zgrr|m((Eh{Rl;b%AVz}E__K)RYL7Zf=Lo|@Wa2dz5dH?T2D*`vTBG$x?7<}eY0T_)<};;JN`DBnmb7;+Au6T<@Do0h0us1~X2 z_$n#rOOH1WkoyESoJl5XDZ7V(-i;75w)Mx?JMLr^)GpPO<_sxI2$;s%v&ldhZ;U>m z&TLNBNC%#HAfT&oiT}mJ=*$0NI_{he$OU7gcVlDP{9MEn1O265$flL3UE0-Mc>icI zqEg%QG=d5koL5qy4NN;3vl>4%lmTAZ_YA?j4>3lGz!2v-d6RUmFys*a9U^X$X2(s|Xd3p};2bBS9Z8Vyjm1@|4O zB`_a?caI}ye*e5f2AVHcg(aM2C~UpCHVNVJdnu$={LP%6@~M<2j~P+pz{kkg)C!Tg zA^a73iS~+5VOe%f=D#fGq)+AiV&2f9ulwG6z{9VLNCFW|pd?8khc**3OLbs$M_JqOOud|%Vz{9_E>|7+oAS;B2k(>U`%2T`SoCkDZ zd9z8L2>#Ns|D`;hA0sSB+YEl?esZDSSa;h&BDw7MW@f76{O#HJs9V|<>5ht4a;xWb z6XK<@nol^a_8Y(eGF>cq zwqtT-_+d>wNbC;&2SdsD!nQ@5GGIazZ%V%(V;X4 zJ-eH zk;TUV7-<IqS#j%3#@wPM1yh95LKEACFZkkxulBYM-pve>T#c5~8Y{ICL&X&N2#{epo^ zXukP|OzQLzlCLDLktQBHsOE9s^_ekP`tKc7`TUm<@2EqTWlm3@&A>ZX!1S>mCs6K< zGx=D@h3>O}?g4e^()Wc@y(83Zf?9kDVc0>-x!mQqK=7T-Q>y?%Mb`;}v(3rt^29k& zqsd2&>mWDmX>SQqrocenhah*G%BS5KRrdW^>W@<_Td9nVIC#?>^M4p*^%s9f@g zv4B^j|@XyW7+et(H)OTmtsz5L_;Pfd$(O-N%MW5JV4eg5OJG1{>SGYZNd zw+~LPMJBD!>?Y zb%g4Wk|;|v*v;P)@H{G2;%=DI4wuO0E@UfBJR{H)pq(<2pltt_}i zM4VKsM~LR_Zpf76x7FXbS?UQWs}C2~si}5+a9(QQ=NmA)?J2%@ne)&HYNxPqqImI~ z7kjH`t5fMKz(pef-C9zb;@`5mpwAX~6)a7rcT_zV8aA91bxxH0^L-S%X=mQmFQ08_ z-jSmm`DdMy(OHt8{ws8DUBt~XOMDR?=o28*-7ws%?8PN*8PO$_USIOsC zklb4nl|G-lOCJU@d`T$7i5VS?%-J2N6aUlf$>Utm%O!$={2ivvE1rOx`m$mQb@cNI zY8U~08h+LMc#Q(t$Gn_sd)Gu1x5BsN6QJDa<@nH6hozc?3FWw}pa3S)EYjiGgZ7T} z(-RJ&OfRMa>rj5O4{Tb;N^J$EjuT%`FxM-L&F@sq^F_eCh`i=bShxF+j78U==^UYC zD&v!c?{?H%`>cHG_D5o~^Bv>VeE-FJ;fj*!io9^s$&}w#JulSRbe`$CtMD>V6~23y^+hVC>ZxjO z{5+>AU_OmWzY3YUErqG)##%L&_cI!j-pG7T7-ixg(PvEKijXNp*90=4L%6cWN5*-o zWCwXB=jWZcZQXa+4HhE@OO#|j_$iO^Eh-mGRS=^knNC~cP|R*XZgC`jx_M!JK<14zr{+X z!G;ZV=+f>d#!#d7p_(6;a*e+;sk&b_mr8J9i;a{i3fK90eD^PMI00UVMeUR`*febX za?M6JDsAX_I{i?U6R5i3zH=w;Z{tK8rauq3b9+Ukjkx+PIURMcc3DCH{D$-FSGKa* zHt(9;Z)|16p4uH&f6|fmFDhnw*-~Kq$+=1#u1fKwM(n-149LB^HYvYq^1m6wpVROs z#}_bn(BU$@qP)}M<68QjH*$Z#RD7!^6sa<2{&^F)A$te;;a|rnK~g)x6^rOMQYOpy z-QGO@2KG~;)JbTw12HLK=|ote+l%Sh02(AQeh5AdYNCa{`#QYY=Rs2L6GTNHk-a(H zxI(NfKU}!iKJX?yZUt#@3{C;pWhT)sdFVK=EN{L>1ofdgHCZh?qQKf1%-zN``C0y9FDg z>no6aFjd#q`nR?=ua?(REgqU=ghgnIdM?ofJmm_4$tKN<4Vp1zm8dE3BIslW@E#-s zRb?cNJs5y!ir#KYVY!KSn^t0YwE9e%-5%F$r6TE;DxTd6&JzLkW@Dn!H(|Qi?| z8pVb(m~L+MjHRTX%@WvY%mcWbR)LkGLjQ>?*5>_ApxTfefC@|B!MJ-Fn4mQ9 zCbB2hS6;+Vdo>J{-|c9PT^=ZQ2^qlIiae>z*7*Twd9Ri0V;c$by$v zX$c3+&K{5kP8(Y$K=$X>40(gO8z`v2(0|2(HC>Mx8jWRrP8k@>8& z+Z(>Y_~S>!$D%hHSUMb(-z1tgf)?G7O9S6f*#Vc@9<1b=lQkV0vxl}WbjdO+h%JS1V?8oHc zJt&)D*PEl?m3THcfhY4Z_%eKAY&(w+#16mhZkeY_=&eBS2ApRb)OPY4%71Lti>M!< za3&txQ0x4yX2CAIwlYGi7w!{iHCF-5T+BSavG&{b@~1we)DaE7Y0x?uQMzbrml%7G zSGQ}|ZkZKW28>CH)&JYF?@TRLYp%VBccMI#yQP4;UbNp0IpVD_KNPOuX`w#Y^TG5Nm4T!N zTregAQ*WARGV!lIZFp+>cCxpTkd;_LhK|WBkej(N8yx^7=`_Yd`w{F%)@>GF`dA*- zTHoGOD%|_;Y<&tKBCm6;EO%8waNcpV4< zQP>-lGq37ga?|a`<)fE7L18oIB^mvMvfX`NCVniI2Xkiojv)oxJ9N} zNJ%!z6JSaW7@W<_S`iu(E8*YEih?-5^$V6}eJM6jvIPwe^eiv9;@NwK*-2I)-pc31 z{8&ED;Sx{%_H-tGeW5k`z%7}v$(6>P3i9JR2MXORQ>2#e)qiyFd25%Of(~O|_l)@+ z@{qI0_kf*1Q->AytR~a+TcoC4$@$7utTm{o3T_>2N4tdZCj6rtu!5_TddkZPfk{=@ z`vH1=3CQ=_?)KAcvW9@Vp=%8e!;EmwwOfI=2N3y=izf0|)gE{s+9~MWSRFnw{xc}G zyVMMxZ#_;W2K9&K7*@S{*I?0mQQaDEl5PsQONWAi8`OHiZ1zj;f+Ou6XzY8MnY(Rd zNNSvJCv#}|z+)eKz61&i_ji7*Bu`q(2uyzq-#P9~(NA|vM_oMwcYu6)ZQ9W$Ba zaf*Jx3+!7m+Bd;tV@SDSeG}P}p9G9vB}zq6Si94D@<*Ebr=#hk9Gg5nrXon zkMJ8rG1{enba*3WLj*sCbLuLwBPw~JlGGB9SV*k)*#I-2>agAhu1Tb)Oj8nHgjcGy zm-F#}2U+MYaW=k#mGl}Uq8ENEv;23p*K;$|p}eFa#Je-m>5VfhfVk;;oKPcZwX0Sj z8=7(6XZy@7ZjWw5>(`l%c57hih+JOxDpK!YCM-T_O>czvL`E!M;hg6_G9L7;F=6Mh zlo;Dp^>CrzR$x^-6#$C>p9oo|({lhYe?(Z!$6qfEn@3}_uUn+fdY;pKQJ-ieppI@% z{~8?hU`FgmYIGX8rz#|`Vx{WgfRb&hbXY(&&8q3)4;V9s5x>KXu&tlV(a$p}4xa6W zUuid~F&G@@xGSC%t}SzDvht5^SDtu*odp#SGk^8E{^NVSHG_MDcGO7tvCOIzOs;hM zpva5vgDckHs0tC6-CyDUWuA16pT+QE|6n2ca~>wyRb9%>-T1ZS_DMvCcCcsPdurk1 z4hQNNKXi0B6LvqIX6O~Pkk|2#ZUXb4rwO;GEld0>34a$YKOj@Zj-o=Xrs$T&)dQ(3 zp!$bRATXQa#v)i_tvouMEg`sgib^AUN29SA@QdcxuwTL7PaM;Ox;2)OGEis>>GA&V?Hhy zRNpam+ex-)CTp+klWrm+J0#r1x-boxcU~0JXC!l!+t^o!hZ?V&>%lVk=Eoq9l@2*X zm$=S3+N3C7KF)hbAD1I%d_@gi)#?nP{dk|h)D4;BIqh=CnXq}R7^ys*U-i#ps9#^M z;h7;h^*Sn&0&o{Qt_CxhwM?pNzC2aczOKg#NC0Q=3u~N3UMx%6(%pDkAW)&4NPzU_ zlce}4R}%ZLfMVvWOFvs@MH#vE?AjX5Xx6(b{@srD4yC1mUzm{Y)U68To+3)pSrupr&ZE7g!H+uy9I*A3BALB)FB`ny~&id!nvf05&#P2)VwUe+hS1Iz#J-R@a{;#QaNsE% zTw)>(1eH?cs3@2QK=+mqva_C;8ZI?N;(~o$Vwb)D!{|CDYuOo|BpPpfx0d4Gm|V^W z8WhmkRH_j~f@#Da+6#&Wv(sdQjrTkAxl}ym5Dzd$X>-u-w63cskn=tZljx@eaIXm+ z#dNHYXw~E8G3+o|Om5^=E+Vc54qc%a88CHdAut9?UK%h|@vX;=u0!#S6b|3z&OBh9 zhz2UK^Fzd+1q5V2$0{}YwMhNu0901tM{2=d(hVdv)zK6yPm(d&tst|JB7x2n!2%Vd zW@b|nS4DaLxxYjztxl8i=~# zzO~=T#^M9Rvg7+nhToAsE&@PW_B@1kk#i*s%7aXS5=_{6mtmsBzCwW)2}|zax>fDK za`Uy$OFJwBDOzn-Kj*C|JP;ts;D6I`A{`0P{WCryx%=Sc*~_0c5M$m|WIATOG&yvy zqfR_~A7pORGJS1t204Pr1VH5z3h&TGGzdKwMUs#l*q@W^Z&{9~Tq>UTXw1}6z$9XC zy?s`z6OCP)cAArDuE;Hvb9KMcCn&mQ5u_44kx{bHX> z1l%GS;8zG%039$#?wbi6u4T$J2|SSBrQUCt@JjXBZgk&ygsRZ#qYZvda{ulqYwT;Q z5BfL#|Lz0-56L2ZrI%>7`jneq+P`#?t61m%SC-dUZ2G#L3zdqW$p1eIH{XD+`Mo!| zI{eGu7d%$*#dO}b_WU_-R#MW*732#PC=!bPHg+WQAypUc{Fu;dwbJW6+d*L^RRs}d z`;N~deC38&NxSnPen8=Wv7*J?h^UjX%n^OaV? z_W(^yb{h`?Wsu-$XQ9-=pV_0Sc9Pqg|2hgCVtle1zrARem9Gu|0j&whj1>e4Q67L^ zO@Y#v)S)g+pHH?t5Brz;e-52*K;Mmvp!@ucPWrcxL}wwxWWwKl5KMQH!s)rX(C>XD z_4M?1>I&C}^(RKC-&?zf(lnPo9=NTL;pjC6o&<@vxy#b~L z`ouJ|aAm_qmoNQ~QaD1=7BSsz`*^FqQ32SK8Hp(^$3JFFH#5E>g7}4>1&+3dto;`! zXJ*Mzf?z%%@>e4QHAOzYp|sR;x`qTu+gKN5;9gu&G5zi|#!S6QvLkFQbmp-(s=2Q& z4ZU6c-Q@D(4$~%78YW0giCTGdWFqXz0nxNOZ)f4cB)2TE2_%l7dVrAPzqr8aj_Z-r z6MeshNBXVEt>N0+qh)6zWBONMcSu3Fb|>7|?Xw~?dztMQ(i@e zt-Y7K6_*##u57Lq!j<*lJBJQhTW7IzIPt6YH6rI`w@^Fr1SX5w`g)?=JT>3=mWX=S zvv!$D+BtYn9xoxhGCMzMVK)X(v&o}#?3~>giH*cg)~i>xshq#P)q3ABni)7+piCdw za2cyP!4xmNUCGtbEq;hmyKI8?%U(}DV)+g=u@5oo?A(x%ahL=dG#RSr_zXy@rhC2` z@xMlwwwyrRn)%c|*}{=9&9~C1EV!}j|DC&>AtBaa!qsN=RmOQj$0sYVlCEHRF?0R{ zpBTw9RnCY7Icv9ZbwFGwdNe-g4mW7b_)HgEm0nBz#jI$T`0DCAW#;MoBZE~g&M6Fn zpR84@Pro>H9`c~vzfsC3>Obfxcv^RUA1rmb{Gq=x_`LsN6;p%KgFmj`?T=D(Te%W^ z^rn#pv+36__F)FOw55})-p#17%KD}WAujD6R{YU_#!a+3?ND~y?PNa`W7;F#iE!oe0wQQb zM-QorK4oS5b3g3Xzwdt{nmUjStvtD>k1qA3rhbEsoU(Z;>l!KB`cLju5}PbJ4yuJd;y)bHeKX5TWq9nke67~LmKVo~45;)ZY^EN5 zDpNm1#lT#N_(zSzH1`Ij$O>{lLmoJ`^rYt0%BqkC%>76KU#k9e6mn-TZn|7~)O{m1 zh;S9gmqs1CTmJCikHI+DR29r+EjLC;j@>uotVs{)WGJ0&KHPNp8B10UcI|T%C95R? zEPkz?b(J7f*`~#@^RMs;lpzum@uVm8q|eUiR^Y|=@G~dhKyPpuNP+)64r9IpS9U+) zBdF$XWZTuk@0)*@r1L+6^TaBBtQ@K^$&gk#*2455fVOqyJianR<3{m2gG(V#w5&yQ zoLnv==%E6D{wD>_6C7k64+LEnHpZ50=0-KX7}Vz%O(!b`;Vh>ruc*iLcGBh19ig&l zitZDeXP%=l&T1F4E|W!Ts>$rPN}sGip1!4zUK<$Q=bsh~OvXWP=TV&|SAGRg`!AqV z?09cvSX+5;QWr5jNaMVtlg5ZPPR}pa54hd9uc!Q~=IRx_(Krdy^A6D5jkcG~5(ttU zM?~L6Rl~H4`eY3@Fk*BNKr|@InN{ruoT0Y)bg$^iXMb(a_{rborq_{hJ0cQ=B61*+zB5-LKZlAF$hJPzfVVdC{IA7qm4s880J zeco$Y$v=k;s$J`<$|vcoUd$Z+i!60a-@;b7jFo-0EDl&GSf5F`Ht^v}%}j8j{+QXr ztKuA}#k+6bXjmm}^KrCyMJ_}eE6F@OJMb)#SI!u%dd`1>GzmTU`*?^yzR{TZ<}#~p zSJ=-r+3`98`ku=J=q=^^(1n9M?6EEDVpH|(Bf{mz@%Y#58OanV)nla~ro!Q1B7Rc( zePmlMQY2jZZeD0Ui4`Jjc43||6c8cC?#VU2bLiGlUN~2CebV`PTeUX3!H~~Kf43P` zE?a@IbG&HRte16mrVH6Wa$eDuYfU>h89q=3rnfdWhb;DCzhXsXZW(FZC%2I;T}I>9 zn$w0O7#|;_o{~aeS9_s_0EXa)8M+PUyZAc{{5s1+Ercfbsq9$P3hIp0%szEZt7G5b z@0U3r&`_{W`WhfwHB?HZYB9*#%(FkyuGnE!##L8<(w)LRJ>|Y|2wZ0NZQt_db%U}| z?EK_L-@1C*RYo=(lvyB7OuUOjpMI-f)Pbow&y(eRG2|dCMAxpfU-(O(4;I&47Ckyb z-QW(vrKXeByDtgrNnY<7Cd-o>*%mddR@M4Y=S&EExW0x0>>O-SX(#;4EYlywG&Iip z)f-3wYS|E`GCdQK8;aTR~A!8Q4eKKm~;UQ~y00=yE-2oA}q@#P_Vrobx!ezv^M zAZ^Sw$K}-ZcFyvzr0s45`(72r2rtuH`)#5Q@dofUX{~G&QcJ=MAhQz~8Jc8pHTigd z77b1U@u~6=8oN!HGJ=$JdJhH&Wwbl18+1%=I_OcAd7$DC!5JC$!f#K5u^>_qcAEUU zuH5M0fv{-`ci^pGy{1X|<=ZC%JJDpW;7l5`7!J%v8W;$b1uDo-T8e|EJ*I8;$zh_zkee4D~(2*<`+_Yd*(4pAYunb%$`RpD=G9eB9@%W3&+XIu&S^ zQ7z!p%lsX@Zc9A%BtC+{3}ejC8!^W?^=V#3{(*E}IzSu4dB}4LbcYxD`?x`^zKddm zSdswYj|0f8p6od=_A}`6(dI;}9HcXkV_NoMPMLL|II{vr3e)67i?4}o?5#Tts?~1? zQ`bTzyIZ{(w!Z~8VqcbqVMi<{*Aj3)Us?|CghSBqcfo$g%acdDw-&k)4YM(my|3du*u~#qbZ@B@~v=y`;$BwFm9@t$V z&2igXV8dzMZPz-ymXWseK0in7m(>s6#`(r*Rj9<@pEWISYob0%Lk{|aZ@ue~Acf=Z z6C1l7Uaa@n_Cnrdj2w&M>&mdP=Oc8sw;aeA$BxZQ4kSwgX|i0{^21jnxr;BRQt%Y^%w zw+OUxGSEXBny)qS<;EQIR$cE22ay$izX6G)M+4s@gsis6&Wen~&>yuT1W7M-+w@-q zjaV8=EP)Kq_kTOqyHf3nn43$dAVeKUO{^15cJja8s^WHY_ToLAS$-NN z^5s3&Su`SPJOqM!C|pjFBsi6ss4o6w*N&*p=>l<6!U#Fg#9lD_k^W)pASQvgQ>QPK zbH{TJGY_;3{}O{%)ROmhipTeLM%n;Roys7~Y&X%B(s-u~(<@Vaez0jk8$K zM>$1pMkP!eC_U6lkpwjIfRw64RFKG*RtO(IyZLu2TJYz8TFsDGZNRU{K7@IyBTQN$ zCEkbf!bE016OGdvgZ6}3-iaoI?2(TdcA?)pnz3HqcUySRKC%jET~0V&jtFh+7h&%Y zY=L&!RU|sR{zD22TzKbbythENou_Cjt^LB28E_d}j>9JcLm#GV(;Ba#6GnoJ#ZL|J zXS3ZHt`6-jL_H#LHyX9)FZ8@&Y0-oU+zZ5JAdSdSmqpRp%_nI&3h3P7*lDocxdz!tN%-$L7fGoteR z=HR$J19t=dKW8E1Q2MBS==0#$;R{%ByV0|=!&hXKWvexyfWJdjhrt%_U;wA?#V#ma`IPVw@8JXfd$E# z$JaWk!#@~q+5;Z%!X-XShx=sS{3l}9CzXlUy3JIeS zQKQAYI#4e62hsCSyie0xT%Y4Q;-*%qK$+;)DyMdzP*S_ZIPLXRt|T80IrgKc+{Dv| z*C8JZ`sP2psrwohPj;Xnyp#UX@!g`B*1WLnKJ`g1d;9QQneg+6bSObWL&J3)XUGI{ z-f#ulHyX7h-JH`mfB*gF37g~6G5fOZU}?#m)gKBQJ39D6TJ_?B4mZBE(V&=n=G|Y- zE%?^qu)EjwSIWCEQJLM_q*j6^{?CWa<$6#oKJm#_>1|uv zxDwFOgo47ae{|O|{4g;7<3ko-lbQCyhs01Gb8mU$>7?>wjcaD>vtTapGcd12j(jql z8LEpH;ivTM*#Eu`@DKRxQ&#+Tq31nx_^18D`fI1Iy1D4pS4f@bT;MrE^#EiQQCWfn zinkeSWZ=nC_Kz-ww?8~?15(3}LVN4TQus+-SovL(c|$!zfnmm2`$?xuijrjFKROQ4 z<%nVdbd&Fc9k`j>Pc3=3*~gz3TVTyJdPf(8Z>D)$Efwc^Xor*EvBRDEOSJ_MJWU&j z4Us*;*_}}u*LDch708=AqLBdOXP3ox;C-pF3|-Nwzw9SFoEJ*z$Kv z{d$v^UUyJnYf&S0MI_etNHu<3jZH={M8sLg#Um>j37$YxJ@4llRVEd)1N&K?3=sc> zK|8h<>htDSgXar3EX^aC>JZ7WL$gc2l|QTFQzx45v`PA@o-|60rccyj?!{J22i z^;4IdJMa;b5I#1BVjM4;6|ic6_>&atPSFH)cf^jjh55LdX51?d-zGNVcur#47z;K# zI0>I^O6!mTeak@~4+76_Zb$<=D#F8rV`GFh?;Cvl9QF;>XJ;92|Dk93Y04n(%6NWr z1MSvCz`OXPi$Bc!_3h#=LhkH@KL`{RB1$H-3E^(G`K@@Ft@W7IzUnF4{E*|>H0AgA zCU$lqbrwAg1TxN z3#D++cwgBV#ex_3dT!G+CyJqR=x`rfGhWC3PFLHG0V zd^=2demPav$_bl=54CztzP0m;TzZ=a$zL#_Dwvm7a6$2ts`pDCZ;lp7e*I@1Z>zg- z0M^Zek5{m`L{JW5oLW_l7i)N$+_R8sQ+m@Q(L>obd3+C8&_9C-hJb56wpZu;bbtx- z&k1_DzPg-br5%O|3(eHn`ij&A9x4m14@B_8yhA^dgRzhUgG1`=#=6CD+E;F84C#)k zH2nS!YEEi81iXE6|2JdToeEe~2{t_LARLNnjSpR1u$ugm!fL774-TgnwOmM3K+ko9`_Wxt+ zFTpZExXpDG39LIgaHUg`SP+Ul#G9p1XDf33Q zb;Oox1VP1Uu(u_TWt?rBjbcCIifD8pv$A50@tsj8JQm$Y1=zioP`A3FOq%_tIZ{CE z?Ufm}9`OiKjUaRXbtPow-@Is1RUm{i-h!p|ZaTe9v#OA`4A1%4{?4M{h6zdEaEIKv z`iFD%KdS6~O6Lk_`ulo$8Z#`UH(zqpTU7zm$`z?^)cS~;j>=p8)pvfJ6*w$%gBjn_5vcs^{64tw$SdX9 z+BHCsX>(;11_F^98)?=7h7VZxFLxhs&kYeiX^-0tApfOmO=#BG9F6p>T99Hplz_QP z1kr8Rxw$3LiE}GO;Dat)Y5G-$<%U(S(yBC3w9@;A!Jz-9{y&u9?=sB3!fXkOUuge- zeC}Mdj~-P@$BF!hys>9jO2&{+4L_Vy{lDz)|GkD)U1k$1JyF~Q3LAm9{-4VkELjx1u^d)+i3uW>3gK_0F4#l3%S(6 zKL_+{m^Bn~%e=M4ZbAPoesvGAHZ1n}Vb$>KUXU|i^2^RTBw&T^AJ#lr75Ln{-_+dn zIFaApe^(MhZ99p1|4&8ou(Jh@-nJ%`1V!X?;qo*i^`05psM`uD*S=#^+*FzFUIukY z16BNb8B9_Md7OXjsq(99b+z!IBwaOku)O0@kZS~3_xV24EOE&{s|XC zdE?n&ZMq=&$a!mP5Bj0y&sX{HRd)7BX{@UiS1b4QFh+{8VBLoIOuJU)X=l$EY1Tz>2kVL~T;Tz*4 z#Ap)4scT|&fJa|uXd9DvnECVeO4o{TzAPNm?vP;LG=iJ_=)=>)s(GKggLjYnBb!$d zvFe<-Cp9*(us^b%elf?#D8^_6;8SN57aTt>x!*qSqw`)u3ho-el6cHOKhU8t{$0?N zr3Gi9h-nf`F&;xdm7Jb9=Q66D&bprcJeGy~OG^^qA59dny4@*PTVSCsX)-~B)uj?zSdS|fw8mv6KG=x&2c>Kc+u<~k)ygM+Q| z{=8l7`nlI1Xzg5>?hMMy2R}~!^x$#VGc9HFPrdvz@y7=SiN6gV5U@TR6;0#riA$1F zp8DakQ8!F$uG;(k{5nfm3;YhoXG4T4WFK4R-o}b3mFe{u(&2Sqrt6V1o65%KcD1Ae z3ujm=;@?Yq%>$5kUAgtQvn>WA)X@(QYzQFXIVDPF6=6gtmX?N5*gwEs4(#f8dhX~> z$hS-W8cK6m)({rC@G!EZQKucPm$hu@!MWzjw=!4d&c4nT3tMo&-U_Zj2ws-pWPLtt?Vy8^9v;%))Jt$gkk@w6w6l6 zNH3PfssKUVnGgi;im?}quko`Gj#v#qm|FlyQq;pT>gM=`;q8lhvS zl@(HQm_^XP{`-uoTcJWs9!`eIe`(@S=O^Myy2do0S(ziXFHwW!0ZGr!XWGvmMZlc&hEKsEcZ0WgAnueRBo!Lh=I zY4KoyD$_KgwKW|$He}#p7@bR2#rtzX62q!o<-Ftv9=FxU$0cqrTC)0i59;{Kkh)Y} z;G@A)1)*Tx2jDFBkP#xbhZl^ONMI}y*jxToHq7DV@Y~m0kmZM9D$Vbip+4cEwNKc4mn)oDGnr;uv;VvRqiI1|wG{HN$ z+aY~X5!4xf=T-~D(rX8{zE;iUch>)y+rRlWx+<-4r@7Cres9#yuRbpN=CY-*`@*s_qC8T>+J zEw3H;hw9Hr&W4(HG568eVQ}!wUN5FSyM4UmqB%mXe*6S=V}2*(jXYjBBA*&UX*nCh zJ~>$`4if7hEFGW~Twwf2R_qU_=@m1R=wNiUuzq--zKp|rV4pO?hCMl-9tNBmC$v5h zYp~hIOU0m##Esvj)*{yrh9b99{pY`))#Bx*#0>WDcBk9$QAKp*pqk)UcZ`Vzs~4He z*DOsR5|!4eC^@LCHa{Gxzp0xs{Ln@P=C2w*yM7I9MOC>SavE} zCiA5vm*6x284<(o*1(d#nKcI$A4D`D;?^SNSy^>0jGtR0y{)ZZIFx;9sI?a+OeN3o zo0H|8Z^uBZE7?=C>${<5aGhpjcopS<@hqBwK z`PZZOORV96!Cy+6K7{c-d6*$4@$*~r{ESO8a<{=JMlBL7b)ui*MtF2MP<&k3sh6l2 zzocxUZ)czsYBswJQ|WCWjDlPC9sZ8XZOtX)9w)iHlmz~T0DEVLN&(dVNa`D8(IbiY z;ulM=Xb+%fvBQ$YFy23EM!c20K_Yg$Zgx4IIoG-um`rRLD-bue%*8v$3fd?A=byaL zm-#d6m*!x9O_1wvp;`K-@z+5YS%aVl+iMXfzRVTLG{v?SUw^I7K5D_P^SiLEg?OH3 zr`+Q)@E@}XQU0b~IHwQ3T(=T1*71u7T^;h`?XlVU*n@gUdUFHp6o?WhI83le@?UKMvQHiUmaXj7!31b07$qsKdVN+Ov#rXpESg!=S&4${AM)7c7^!^ zLwcVA|ABO=%`T~P&jp9#%OA6`qNd@^JZ|h~xIg`H{CJpGrDja;jvD}n0liRfV#DT= zvu>NG$7{a1&B!@&Q}I=nnIg4g%*%4Ht|G1MG;oXLIh| zZ)D%?X7hv;qgE5oTG8rU)yf7Fr*x9fQnhXI_NT#rdWZg8g+T@?!j`pg-Qz5;TUu3E z!ee-BHo<8ky%;ItHY`HaN2#vjtkV!xJ!)UCsz&p?tnN8PvV#_;yz#icGyg1@ttpFR zB5gIL8i}TYv+bTfD}GX8`6c!`uHZ7+BLR;}X;WH}U|fVX_RG+Fs664%m~5vyg%po< zHu?VsO-k0`hIY$92`{U8r^-u1lzifOWpT^B&>IWD2LdYEi&`=!E^X9QB7U+FQ!t>L zuzA81F)cJGJS-oCvyb6yo(STk*m!^y|1LR>RCC`yc~u5i#BcN==dqaI`MkK_WkoO( z9j&I8){kwi`e30Bw+ERmFQ}KZ6nUfkmn0gKu#rV^-4btbwr(JrW1CF34Zns1yB2$f z$pNLqKae2KGt=P8(o9e60*P@L5x%aRhtLGw1|CPvfpBGGj5=>W?`Q7dU!Hd+LXCU9lj>ilJmh-F?Kk2J zJ_OL##3#i~Ouich(>=r%3!Z>P+-nolkg)X!ZLM9mjz^6^@okrS=(Y(;fgCSX`TC zV4w6Ry~;cK?-*Tijj#Ik>z5OFR4SGs!`_fjrT2M6nZ;VEy?icZ`pb+BC1VL%9$K7pbWkUZ6u&d7a1h_kS=527T6Ex<`fI08RI1j z$WpfGuC#e5g)RgiXVTI$%UkPuE4yv`bv?c|=QPMUKN;@0Y#2Jx!m1hs3M+_2SvOPd~leMkyt+ z9LfIjRWxZu5v^F{ToM-WyF@V1^d2PIi)C1iSIVo_gtBNA{$P|@|Di(FePHiaM684` zco2?Yky$QpId8sfb%>~B98b@C)i#*-h|O3?dqI7NW`!aS6CP~agYNcwH6(N%Zo;Er z;v|eqH`7g2N(Z$^_8o>p-j^wV)9+Gt1Q+>c5=o8S(8km5|ELmK1!MlB@=Rkw(h*-* zC1BJZKo)euUb$+6o`3J;s{5PD%a>*xK3{@quD=qULwlVk@#oj-G{j(K@!sWC$Srr! zjJoLfTlx1ZqQ(Dcw{23@5~uA%muZYEt^7NUwm@PZi7tH{{0l#X`@y6NNM>EgvG%c8 zMFh+AJ=`H|r%Rz>Nffux8yJ?Fg>eQpH3lsHKjGin2GAz$BL~lq472pzFzfa*biLb;rBso~F2h8a#rKMvhG<)|wCGOP3Oa@$ zGo9GuH1X{Rs@DJ)fJzw90j@zAf#4oN>B-i&dG8W$`^)Kmi~n85bshCC%>QBbh+hU2 zwIoI4!ys?C0Y!>WvUtKD=6T1NjOQ77Yizym^d&KaS$p7ulVmVxF7O|%uTtU@)#7yv z5k2eid!`K;+BjnX+d1$b6{v#&s{h3!%ro2VCCQ0_CGp=~l{U}IjthR)AjUw68)Rl8 zO!nfM&OX@$rSVw{e+@p&dfb@ODYns!{en7J#u%31JP4m~UvNfycB}eOX6@P`gXdQ* zgE&m_eNhozVG3--43Y+-`>NoG=RSoG-k;k-9Dj7LFypCTqiH=&KA+NZ8o}L1(M{Ii z+ady3I(Kss^bk3XiOrkFL88*#1J6 zAQ%xV1r$bnY3~=x`1#$?XO+V)`T%i-FUnpj3c*DB*ZB<24Vz*J(M;-x(wzKMH7k@} z3l&!%1(77!LoU(0q$Z%&PEE!S>OTIy^Hz*?gTo;a$S>kq(MSfeHdA~D$Q^*7A3vr0 z<-IPG_Dqp)CfRf_?lAk#1%KM^ayOLwOX$^lEJu+&T~jwGCM@zanaqG$RM!Ww>R*xk zA;$lHMdZr?Gzt)W0WIg-IW$$mvyB#is-+NO;+e?f@>GKx^9cQyxr=YBi?SDa89&|& zy-9|y{9x*4(3D_y{W*WoF?3hq(dZrZwy+>Vf574FXZvvjdJr$!#RETQfy$D z^)q8?9=EEG3Wjnp7L^?}VkLIB@JZe0{B!Tdd129k@lL4r$Z?XY-PSq0V9oyWjjRn1 za7r;;Y2_F-J`A=PsaF#gle_NNCgYW29P8hQxT{p*%v8f5617(8t@q6~f{M#2Po$fc8q9i(3&4?XXyP}l;`G5(R8o7;SR@a?&Ob;-TrJMs5(7Q2PR`T4 zNgh<|r>)Grxhha#p`8=)@*V5x`q`%K$zWTJh?X)$h2DUyG1+WM>k{m#o^Lanb3g`aM7AD> z3r1WY5mmpw8DL*7&Q{+#PQb z#Er7#)dz_H*%Ua(8RT3~P*>bD+@?h|7U`Ke!laKKc7dkPnNC-uXh@yFk1^n$&2!Z& zNb~D$`!S254AAiupv0eS@KCW4i@w2%2hks2r-hxQfcl?HU%s4t{_0y`>utIg5Io|^ zMNED7x6TxG26uLf*l7GxjpaL+C;!*ST*cSKou&arcHF}S*M75AYbf&`RnBA9erhoa zy#Yu@y{iNido3Wvd#SI6_suZu4voeEN_@k&i?47Ra#*Yyh>$YMYBa$XtX=7*!;*^F zsC(hBhFj_~6&@5g)Up939a@D7u2*7v&CRVNwCyLx?@N(B@Es_iMl3dI`i*8xOloTo zJP~|tN&QPao-GS*L^m@vUIG4SmDl4V7TqmmlNkBkSV`oA-;NxFiVqWY|69g=_Snt+ zU1PiOx&IFg%l0oUHOU_-*Z7;p{a_#Cigc%#nXWj7u$8{@F?lk(cqdd27_DKP?7M7(lZuP~qq6Rp z!<{C#Ij%5WxAf@X_Z{6fP7tdo%-AsfyQJzQ%mdCUVnmxM@9o2j=V0$|ZuwUnrD5$k zm@0x=K&e3tm#yFQTMO*{`CKw5>FTq~FMp~N`B5r;Z>-}A4mq%VV`o6fYyXPO3-})K z3ee=(UWy+0pTyi$sq{=4YkEN>0`v7%4Lom2aT+ara_$C*x$Y51LOu9#>axY-0Z1q( zui1cFmpgI*gfm#4Wg`3@^#(5aefh$%y=G5`bjEs3}NdF+(i;(Wkc+d5loeh6i5u%lO&gR zJiw77QkS>bBbh2UZMJwK(s9KUx=_rS=zU!A9^~6RS$D)->Q-(_i?&xqPY(J|(_!U9 z&=OVdC_aSDdO(o9XbcQLhzC_85?<6!$)9CxRsV?IesKKBD6*Jxm5HqmT-gGc-X%ZN zc!*gzlJtmdq-Xy(tfPC06hSgA-u^Sjj3yWl`fuR)MPT>Dw z`;Y2dI|{J|p~%^dR;Ylj$h&?mvgeBkDBpk6|9?&Td#%^^1Hu}nGa)nhv>KZvIlcV&fA1KnTTKg!&?C_-qhQq@N|>d^kVNgRoV*kl1LZRSV_c*-s;?Uqjs^B#3U65jGim^`pl#FER}0NyL%j zTy4BxVr8^-E2tvOFaf8s+MboU>T%tR(2`R40}gE2CpV|D#nN=c&z|HLQp{z!cWeni_qt4KBA|F-iSJw5(==rZ zxU86=9#YHJa;=&Q-~DP6S9}8Bh}Go8{pq`zg4SEEs%lagz!Dw)oSdjKgn{%%4z+o#Jj#yW_xudywZ*;8kLEYfKbkbNDK&}2ZW@m+8kD_fZPtoh6d5{#O&Sse7^ zoX?LM_x3#S`Zn{H78+%w*P(Upsu#@vMz^J@UYLGQy`C+lXuTet?I-v#mDum*X5Khr z9U4L-v4GOUFDFAdj*)Tlf<{&I0p)s-5JuQTM8btyjE@mhaC!Z`-^;{tJX9B`Urlg% zon`RcY1V`0%D(C~hfQS?rW2?Da4Yu$0=M;B(NK4~A?`7WoXe14HLZi5w*ROGAT0F& zV=t`JAJjA@?IXh4$V_hK0XuVY*37 zG95eN@RT!8h229Ohv#dVo-DpO#3rYH-chkaKC8-fP8Is?|6rv@la?jgl7JtoXb63jg_!g1Ec5n_#4fJ8Qd9tBbzOn!G8W;Yg zveYt(+cH_~6q3X=52Da!Vh*kmOONb`DxO-HvA;$7hdMHUUN|0Bo!G0j)2x>_SZ5`` z=Yp$!1S)}@?1%#ViM+`kt4$etO0V>-C?icj@MK!4=b?}+ZZs(h<^dNzBkW*rR? z89p7dtqDdiv^HlV{Ws=19U%@r(dt(&^1drCeGT#o$ArpfPl(?W-5R*m%|*Rlskm=+ zm|zyWm3pgGBG?$>q|O;3F{vtrn>MP#7YG#nHm}xd@RLM5kbPFd={^69`h1!VL%;>6 z`?Q?8qVIZMp>(d=KB&e{PZi&Fmk3qKV+*M6JjehD`Pe0?@qKa5N_K_D=jvE?LG^<5 z{?L$GuMeY3F&I;S1(-JRLBWp^)FYVT49jK~lmUsRwU@i!;i(_IapH$8g2EfBRP ze!pwH>y;et1GM#a@mWM#U9VY?(18C!?ln`YJAyevzqZVZe7`X1*?0uD4*i^exMxq@ z{8*n){h}y65)3k%55Bg1aC{}Q?%R=Sie|U0N4h*Zt=UXbWeUjbP z7;qamtPD<_eERcx_sX55>K}#i&|v>b_v^XL;vfBQ=b48PV)8=LyOp+m!6wE{4J`3% z?}T}-xlTXysqO|sDlJ7z+H`HyW}+3^x%n^GoLtzv0)IX50XV%XQ%j#+IGrT^swQq3 z2soK1Sb-eKss?;|#PEMl*f(sc!ZQC>Q_Rv@s& z5tyml^q1!JUJ(hweCi4Wt1`^7x5n++g3kB?xykR(=#0ho;In*ICi_y`4_`iKHg4t~ zjqjg4SP9e*GW+Jg=#;VZZQH+e^S#GMZTS#@ROW`Q`n# z8qDS0mtK~*CG4N9f|w(q4l4%S+hN(m{2wjv?eD5Y(ge(P08=ASO3j4?2`^RS3K z(%I*OPpika>M>X4YduNp|srj zgKE0FkRrr$91XScPf5yH>GQr{;eQ)X9oLsTi_(4{?Gj*HhmZ91otK9xNnbx=nK)eG zH%7cw4KN;=b=9?YzkfgEC(x;6%pmm0AYujH1Owe$$Lj#?rl;qiI8n3pkoQL7bM&sE zSr`|^OLF%5yiFXtB8__t_zeuxUN&A6^cG21;8+B%xnJ;7EMe#uAC8^hxOq*HgEz_| zR~PY}I0Yg)BWGK%!?--@0$5rfOL6vt#sHA1$)x8?2NICZ&5HZS_XX9Nw^wMCM@}s^ zofhr5y9wZQhL>v=tfDR7>qR(4a& zxG59isz$>#>|egGeBZmzFGQwyyGb(E)|PYb$`!eDTkP&ve7|%x1#$dQkvIZWIEh`n zDDz#N392tQsA$}CH2b}cbHT8ebcs~Ys4hoQp#+HOcvekfXt6e;6fHU|JkM=>Q&>4fOdb7Bl`w62Sef$*u3pS62+_sOTU#C$dw=_FP-b1rZbF9$d<5{1 zwYhvZ0NgD!NTL?@)9EYBC0;+IIp1a6rH=bJg&qclZa9BCUOV-`TlGn$PI=YBk`5$r zZ(v1*N-fM{2(U}b z6H!?8CRsaX<*kNYbHxwT{9gjayKZ3XO^{uF*47DoA+#FC%zrS4i2O`of3$JRjdxFXuY94vAT&_9h#BvD?bs>uVr{h;LU_&GP%y3Pf!V z<-&Y~F6p1d4q1}PvuMSwxpu9ANWC6QM6G~n zI+e!17k4*A6nR=A-8<4MwvEE9pGv+uD-h)84nouLE?yIXSIw<&KJ}G}d15XcAmCiA z?Sbm%bqoS}%W!*rl>05Q^UsrGEn@60mpCgrrSUK`no>t8hehDAC&YQYTHx@VC0U)l zH~m-h#N&T&Hov;Nrxf9{_1N*?_8zelO6Pmw`yZ9?-JrifOAZ!DyKRO{{f=HJ zWBTOl8EvGde0jXc7CfLvTk#rYty)u33`Bj&0>F`_-=_agEYoVMP6)Vxfz5z371gv9 z`Hl%FLtOE|XV3t3omcR~s$nF56E!9yRZD5$nlj)>)wX(Ltgld&c(~y;!iDwMgZQDR z=h^<5M9ARA=k>%zF+dX)nJDy(QBg#Tra{HLGSGQ!K(@*%i^s~qP%S@hs*{fFh(W}@ z(*}%wxqs4N=m-&Ve^rzX%|7eA+Cx7jdLNCmvQ;WnLI zs_3i=e&My$w!xuFM>xzIGlkZcfO`)iQdr96R0mxw&Xn7aKCr?~;s`3nH0kpedb`tA zV%Eo!GDTFWSkNzSJl~;(gkDLyn2M<>3{QS%PUr+P{kE7GEi)|c)P#ndf&Mn9nk01%n+%NlMT%FP$`D-o zfpeAYuref6Ka_)+kWEU@k>k3}I$s_~quua+UXikd}Jp znFS6?h`IOyacY133wL0u+z`FIQwKzZY+BAaJ}F|yI-kPX;giUkxIMKyTe*EnxbXq$ zl~C!U??!?ou_<)~wOw`Jyj~eO?cT`F10@zKAhN2 z@70yTdqOlJIRf^ttKva6Dt{AzQr!>f#8%yBP6~H45@$Inrw^GR9tk z7$o`}mN*S)FtWGbrHj=K&6Fn{=BosiOEAHZcDR9WKs&L7Ky7o<{x4Rq`*r7a&~|{h zlTKGKr$f1-E=C!Kuu?#>1i0*IXMMRCM|mT2pE*ow&N7EaQ*8iz^vQBOFh})~SmoPd z2SqdQg1OF#3`}+$#D!_DW=c(lkcz|1ZO_w5#2=g4gs=ETIhoPbq}fg%O_PEU-E;u+ zz~&F}f+X!03}HULkTwwB8nGK1hf=ja5XulYdR^tB(fV<)Ob*_o4B=wb$44 z&$wP7tG?37{fj-Dr0*KdRUMVP|1Mfd19UWXFG`!#c{&VFi%_5P2?BFOIO19@8t~9K z?DOaPTE91KDv2%8G~OMPa}Kq}5)Re({D#kLAfE^ktb_*ndsEP6lsYKUroV)(q{2tw}9t^}7A!WFsF$ z00K~S8qo#qZv!(T#E(*j!e(^jDg-XcQ{D{62cO!$p7l5$?^(J(4bp_(f-#)whRk{Uf_)xrS*^ekmmCAgvd~ZtFr3I6e61_Ixm$zb*6y z0VJ+`RqosYpFyA|df~y36oSD7rEi=!H6<;^Q#Y#Eh$Q?F0Zc5$!O!)R#MHxwg&%^2 zID@dCyAXPX1>>3m;Z_w)YQ#>GBH5A1qKrzbD^C4mLi)4`3HK2E&TmhR#x*diSiGCr zK4ou`Y6QFs763`>2&>%<4#EA+Yx(zDa7aC+Oakd`@s*D2fjXr1^)^HJ?|iz8bcFl$ ztfSSKO`;o?cECn=!WQ8?EVXxQky*jvb?R$(lurWZA#_z;V=TgRD+UQXtjhCB>Pl8C zKr;P-y3qfqTtF$vt0y^uNS;lTuEyurD=P0p<>y|hUDD|kR$l0{rx?`ANowkkAbs6QoVi+kZVvxjQcyMu zDu6I|I;8kV-tplfKWK^T&%H)pG?n*Ar;M_ov~Yd@n%@E)krj&LZ7ytZi6^)5D!**+ zU3D4o;f_$87`MjRm&hZ1-+~q!e-d|hPA$VF?{{pCcc_zTiFoF*4#;Nj<4NwTc0PJf zulbyNfr^_LwxUEri*5pya&-_M_^>SZ^A`#?;cVxx@!Ev@la6ChK=!V@k2jCjfE~zU z?r3lTp89vudYKflqK4D{2-Z^KZL{&5toW+E#5tL#1EzC|6}Nl-R&M*Dujr8&{|SCE z-m#zvyU!KJ2f2xVM<~2tsl0aUelk@-72Axp9zc*2D|ET}v$KtFe5^^oxu9t=eJZtZ zHw|O}H6E*c_&PC@!aCO8*$v`pd$M{_{Y#!myjW;Mt%N16ppKiwnf`JpRvI4k9z9f; zkLVq%7!dui5Dd{%y$7uP&Sz5TOVs;)eIEuKX6!D4HyLCGbbfp{Y2w8K{UAwqOmk~E z8LI9p-VXfKu?gBJ3Tup}$asWZ4yBc6MhpqVUH_xHJn4}SgQFjVPa#gSowtU$XuoCy zj+TW@b=CLdrf)274!cBphCB&(=CfHC(AKQE$3V2$fWCz*Z06QH@q!Nw1aETulR5Uz zJg|02?D=A)#AM(c~cFTn~rL;?ku92?RB@U#s2i}J*=yoYz;~G%7^ZaPCYV9^uzM2 z@M1)_T7+Y8ZUw-7sd!0xyz2;KF_pnB>>Zq0h)Z)%d`@kI0Sc5yW%No=*+wemi7!C5oM3*T$tX#iow+Lv3j3%@_=Wxl_raIX4P zS{f360bUo66ns=j3?M034Q4IRJ+|{;mHrfd)wZ+}CIPL6(-1vC(XtR>Zkz3!z)DVZ z#Mf^m68xFSgR{o=q=RmHfd6d={P2-HetKejjDk+JX$L$g%wmbUmD1X^56)(_AN;wL z0?xZS=Wf9onYGf?JYaeJlLv2eZvRH`Lxtjk_MtFi2YxNu+&|S?&3DEfLi=N0-M)d?xL;0RFuT7hx^pA@p|e#+zj)x zJpbG~;N;5o7TKFP**U>i%is+ssDzpAL3E-qQW0p_3;EPZ)@+4VcGJkol3!aP)77^0 z?FMxFRJ_U-#csd2-rsg(R55#8n_x?8k=V{H)%}1tu(S|ZWQY`h`zM5*^7r&AOY#dw zKV!Fd-Gy%RT9~MezYi*B=X7t4>B<~fDKqKS9}2UuZQ0RsG#I$Ptq#i#Ju z#;qgI_n$RAHI@3v0Y5DJk~r1)+>T+7Gx@l?*w)rYQ@)uKI)>tZO4gK~n1|Bpi%rj+ za6T&GIGj#YdUc!#YH1YS%r@D6UY+{n9_&oT!g^FDo#m`yyyEy%q`GuV#7ot@ybx4I zhddb`*PEme({_hc&wlkmUgS(2opRoZ3(q+ZsQ)01J^xyw4+Amuy_Z!i zEq6I5j}gf}x>MyOMemT{c0#}Pd%BU+>)_~SB;tHxCLeRY`n}0(Ox|v>8QUzkp}bVW ze_lBKOz)eD;SH9<748>}U(okAL&>ziz)zR(UhQMn=$`{G}iMw%}e1YZ-J7d?Llh zaZOhqOEbb)+<2)8!7@A7AMl_toU6?S{xYv7e-`bI^=m-48!T z#Zw)*ZC*8^T?tJ3h{7rnSJC-2Z@;N_U_Ci#Fiqn;Qdn8H(lz4_xrX3>2FlMDsF_ez zE03%s+_tgkIg_H&>K$ODv4>okt>xbzD?h1_4_A}^r((`HhNl0T{H}oCMNkZF2tM9M z(7Hldc0bPYG0Lwi=PUaSK#c8aum{m|iB@NRGY2tTTnSV^R3i#47_521cq>7iY);1z z)xfkn+sZc*d`fAbk3GAk6BibZ+)dS9HLM|~gct8<|NQ8EH(MoNVg8-6AKJ)z68{%) zV%>C|%c*65UsN(A8mGbxvJF2yMqQhzo0TQKeR@-yvB>vUnvwg__7VEgTpWLW!l``< ze#lq{t6vi0YZ1;(?m#|;+m&r0u14tLQ1L7B)+1(lQOvc@HflAH&+vyfJ0Xed;_=k5 z=htNR?ASlzpKXq0+>bp}e9nbDl(HNsKj)ql(sn#?F>W8<-FT;Y>D%#sWM`)SgQC?v z6C;VBn@!x_59yhD>)}>RbC+$Kn@n|&s>5zL(H$?+W8p>4FB^?t8p53p&}UnAA8aBS zl}2?z;g15GFM-{xQCUiJ6(iveWrmPs-UqnkOKMsya_=~#(WL$bNIMtkLwE@)x*jwK z?g?S8fhGR#8s{MI*E{nAeTb@AdS>|+HHaWSXy}j=R<+=EWVu@Z z?)7PiJRZFc{ZZXoKNhp0o=L7o%bo@NHY!D&iKxdhaXo)hnEdO(gF)<6O?^^X)0e<8 zzx?FLtHHN#M(wbTMRWxuV(Vv|vt|q4oWdQ}63>Kw+0I@6Cmvx@JY(G@_bCT!u5KM2 zyp~!Y(xoicksczjp3?n(?j`i-w31gMZe)3TC$d!fcXi3LLKhU=qZ&teQvnO!*1yZH zF9?!Qyle5eS;>ptd(7C4w-a{%{b)XmZLjSYr$+-?e$VIK8xncPEanH4D!s6ESz4_j zXUyC$&Zi}W-#R*$hEF}Ftrh9Lo2G{cmZKN4H*7R`_m#t^f6RyaZb=PP%74RV%vdBv zJZ#9_cW}N`I4rSX8-OnIib)PVvF4HedRxpGdy}j&aUjQbGnW4L?>^k6=_1%C4Era6 za^V}%p~2T?@{UeWw_8W{1=?WFS2@V#onx(#8!)lU!kJ+f)z3fQ`q0V~WXhJST>w)U zu+AJS48iZFeL=gQswqFtG()rfo70NBp=kC@%08{I3$NbCd5~5VYSwJqKH)pj_-Mx{ z)UM1hJU6ZO+7?)eI4q9;iIY{h2voe$-27ZVA z#q58?{YUi>X*szp4vUdF(p&t%^e^0E*;%Q!Z>%V*G{om|R>fAa5r4Km2%OkMQhwj3 z;4_khiA!u#z-{Pd*i8Krm}b^wqgB*mQcmdSFe1sUx2%V0o6H~k?PE1{R%R>?&AJt(O)2v zg;QR2?r6KjbPE?o9oeNM>=3>!8j#^lf4zT_!QXf0nIYa0aEgFW9(wGNaig$GrA- z)e!fr07uBK0Z+_Z<=9TohL)g$7gtw)n^XZ~+XRK}oWxds$GOJR`aYWbg-B^K6^(DELJ${&PF96VrY=!wmLC4;@-0pxZ6Vp4uu-QA~+PqH6Pu_m{ zc4fizS7V3YMCw8?s-aFttnPx#5I0#=)>B>TC3S0PgLAQUKl(aat-9ZzmmSxi7Ay81 zZ6w{M><`X-&h8^h-kk?J8N1mJ3hShc7rj9Gl9y~=BnNGwy_fHgTx!$*AlEXImn= zbE4{mcGnJ{Y0##(_4mC`|LeJ)<}hO6N3z+>y_?b7AwPZqsk@R+j?p=P+8zKuG^&9LIQ~lmK^G*F zJF8S>62vB@eGM%_?)s%_!Yyj?3Di_8oYGRiSai z`D=>vdDQkhFGil!j7P6B?hW-IQSK#M`vqaf0d4Ekhd0M2U%T#heKevcUns-PVnSrx z#&e#u$oN`o%x~GGg!8n0e>nK;eK{fy`TeTvuOHT*1k9YWly0r%o2yH`UKZRjp*AfZ zY)~Z4?tg#eC!P6f-~dA%2ESG2nXl9MF2;&#EmdbL-6~si`wvv+>5-Y!*@slq*VEUK zMcZ2&1%GUsoE(xbB0b=y&O^m58bPaNu7^I{366@qUzna+KrT{T`44x|pE$c24z}#3 za~$6=hA_-7=3PBTmz+=Yl|XH_YCh@?Nw1sXAj(aBz~tt_gV7g00n0dh4qpaktLIb( zJj6{EGvM0u&i*iqeBge^R3D6AZ1~Y(>u#B9^8zA7=(me|JRw%`+d;rv6W1BAW#Lkt zY2?zn@b4t4onB`wh*JU?*u%-cux!jc{a|6U2t}RLB_DWEMu=aVrEDLG=lr zZfyv!7kIhi6whpwKrz8BOGzPRLSwt-d^7JwQ~#+-Vc+R)sKSGW8AR;Vx61cKi=6ps zQl-X2_|kK?dohAqVj8ox{{qT0XmU-zAcc)nCb$ev=srsU;mQG^f59vZ3Vn&PccyLS6{sPzE$;U`i;FOs&CR(Nu8PD zGv4bs6lzx1)(z)kB(x(jJ5l&mTk+=^;v+93a z+1NeioC_*XY?|K1O`;Y4h+us2`5$% z2OO+=F!t%5CEm#ZOzRWZwQ=vk!tv6J@oS0^;_A@${_wa&_?<1~!oPe2Qz1VPQ~ig^ zRj8tG%Oam4L4-~Cy|i8txneEyFBQSXK#z%wGCaE1hm)+@@)&Pc@USMiSDB2{>-e#A zSaoi05!>d6LDIMjB7IWqqRg zdTGTOZ$JS#7BpZK3##ed^YL0EN8l#HZpT|XsOo>!x}S%%TII017IsW7J3GIqN36Wg z-A^jqhO(%0SkA+GwbSX*C|BxS9r^ z8(v65JSB~=C@(LSwDZvN9#75-X_kwrOC9cbnrnz&-Y~IeRylot9=hEIXZ*yAU7?Oq-&CQg`F zhf_z@y)~Ynft+_!v5mgqv#Dv6K&$ahf7)MUk4mB&9*ME&7PD3hSskcQ&-T6k%5q-n zSL$ER!79T2f>}t7Ynen{+O|GWh+UyzHQ`tn8h!Eckxui*Z@$+6qYG4m`hI7lS6#G* zB#xppOqTRlu~=#B`3sPqd!Pv+c(-Lx;Oo^$GYWJMDCSsQ{{AZI%8yg7PHJdj{%!p2 z_~Q#k2rKb9fr$+UU-6DJ9{=?Fszv3xZF(cI`g1!T2qrhTS$ix30S6?`2owWh6I+{7 zmAgHIGSYrShhQ0*2Gn%4#LN)(cK6UPYkl_4@salrqMV}CEXd$)T)tG6-7}sYo&@>wf9zPRFR@+DX}-P z_g*Pldv8I_kQfm{`p^6G`#%0RB6o5}9=WdTb)M(*JPvKy-t-?QuN4e;AJC+I0r!xz z?LAz&6gt?(Kc{(i-rZ7#R*$Ra7q3W#3wm7$hN!2Bq3s%T)&%T{Vc0frydEyI&9_)) zKkZj6D6yLko%^wiHW4~R%*|MKa%a$_z0u)_<&X4&!fVg$-k#Wk4Uj(2uym6&?{;2O7jJ1?Uf^dJGu zb^-kV3iN(T5J4CMNrjoUAKk{@7 zKfg(3k6FIeA8_G8mr}h%tL_FIv1-p^Av3dQk}InEmOh4W{LjV89rR?{*cfe80)jnK zwqm~k%zwQqSBim;`Qp;h$;hLyH4Ff8%>$sd)D=pjtuHK`d}1}Mb(fr%$Mb}z*B%}l z`W7~yyN&&V(H@Nl-CN=oS|5$<46bw(Xh>TeT2$HR|49ql*`8?XT!0T`vFmWV0udkG|pKJ6N0CS>4y z=mRbiOWqZPejy7xgU3q)C>)LA)cy>$BXGvKK?1ntjn2r+bjOuj)3%eV;X6eV8vnae zE@Sa7fG?}0AcgQw=FTqZgMaXE#4>(VI%nUDY&yg5Rup&#5b2X{C^fR=ko$Q^T=Z0W=wmekv zJDNQ8s*wD+yp#S{OF{1-FJ)&>UN(0aT()E4k*v!nn*cqw!g_MS2GBJciU{U3Hd*To zt=1m1A)q><+W819mS2dPK(^4MHKLm0s-nK5t;)I7XQMO=mf{fFqw0H2L=+t*zEK2Q ztH$`u{TnCT*vZYx%TgtgB>uoA;^;I8X-1{ohME!}+<6j>FMHUiQsc;s1Lj zlJq}VOqjaC%=CI~?AGc3eN@TJzF$+`FyK3VEQvSM5AG8C|C-PwCHL{jy#M}$JxTzs z5X%bKXJev)L})kU!Kbf@nq`7=7ZiWn=n|$q79b*V?y=eyChk7I>R1Q7}hsMnj1_5L5(tPbiP6 zJwdeTc+I0UDVc~$!W*9};1z;A#tD45_Y1+)dD(^|llu_9@8f>HPWHiSl%*A%c{{6j zoo409K6$d`3!4be4gBRykf)s{tB+fC#D943SL%CDVpqkexP|WKvWyAzn{~ZqwfErh z>^C`Nn;`U!{nLiolQ+qf2g%vDjNjPZbjCmC`jyQfoI3yaGl#51N*Pv+W;RVaKX#2F z&iMHHfypENN$tg`lbksN%puV1S9`-0&9Uqc(n9LcWvkt@3N2CjEjy4WwOtswV>3^( zyrY%}|Mj4`q;-Vuw=Zd;RDUR6meeLFSL{1f9I`D^UuD}MOWJnd=46KG{G{78o*{jJ ze-8X;#wYp4%TMg3?ci_qhw2{;*0xarn(l$ZS%Z=RHCX6sBS zCpUua=Feo|ujLU1>b8vZ-!Jv4s~>-_{^RieK7}p4%r9#p#jD)Xm3?uC$Xi`k+oYuJ zAC&p^WM-_ghJYn64hkcj`;@;?9}62SrL`Q;YS@oQ?aP(o{dogt z6kMN>XwBa7k%f5x89+h=Uf;U%y3Q_|v28*_Kf17ghi9yWaMr3~F4?LMa54hU#A!_jV0?%1jNn!hX*M(`<4SM^?tqs!wG@%hWIqToRaiSJc5(#O>Gc5B{%fbi=14#{CQNI~Y(s zOQNmRLfga|KV=lBEqMp&ZG^`NSQAV4nMZiy6vYs%Zb)VvZt+8nLeYbGA5NUy5DWM6 zk@Xn%e=KLiwy>jo)Jh5ngEM#sA3DZe#xc|ROl?$SjiA0Ii<*jxXn{5(;MV=VEK@6g z1&#gWmDZFfg++G0*r)5Wv_8b?=S^_&A3x(By3ZgQTo~-#>l77B0WK$|koO5dAue7mNmGC4A7rHFvY zJ1W2Wz4XIBBP@&R%s4vkAQnX3rC#H+6frl7jI}h!l?hxSNb< zS$5fo^>$skefWL9akkzM9t*bHZZ8r=X97L|Y|@FwBg1;Pc+_sX78TPU zfjAFS0HwLZiok^HW|En2JRd+L=IKQ{8*#ci0cJQv{S@i7`Fz5>+sf?*oP$eK1smEl zQQbXO6Z}%S#ZAb&Fpei0Z?Utryyj}+)Y^$r=BI#{1jCv%(i8dJ zjByPF>Pd&IJ)i5P&H6S>@@ToC;n^8N6PLkF^_<=e!NAdctNcB^<`3>jb1nO+?*g_8 zO{Kn2*FrJ9uXjufbuPJ#6Y8C`DUpIvVyU)#?f~yr<1D}NdKw`WhxgB0A?Gv@kQe)B-)k=?zOPibIQGcD*bk8jq@CDS^b z{Pjwtz2}1t@6#@G-e{gwAzPLiUyYyjJ}TpPzlx^V3-2&SAHvFTsly{Jt6Y`7DZ6n# zrxngBeUQ09r+i$9xrETj##ePV>2XDY<>aXegZib@2RXIV^(B?40fR=a0C$U#jKYUu znZOvdjq81vkhyE8`EqACXL(0OX(RXjFF0$6^0oFG3@ol5#MV2h4{`mV@g?Z3!?lT0PD-|_yedDt^ z=@w~S&^%(-Gwpd$tM942Go4BsdNZw)qJzT%=;V1XqAwlQ3Kabr%ro!@fyqk#O^2r% z(6~0ejt{jq^JhUAB`xB?sx>6xoexr?(QqA!^9(dI(G*hEcmOH3{KI{A0I@+|f9~xN zfB)4xJ~(vIWm*38x*bi(TYn?%kw-z1MpT+9apHF((mR35ed-R=wo}kZ<6_Bvl5Z;j0tVgkGNx_Wk z*WdCE&{x02UU~E5H>!v(;=NJQZo73DU9a_ej1Q-gz2W97s-S%4bE>KLZHEYBG~!DL z*Mc9X>d(dAw?6%pasKpnRE3?onNLNXB2b zz3-)M$BQ}Fp}j$3k%Q0_4RlKGbd*kGW*FO{yw}(P12K5O9VTx z>yizL@r#ws(74`=tce>hV~z_YA>U-%aG-*1S(1Bmz2Ep<jK2w^GLHdZ&ezN{&HCM;kh84mVtu$UWUyTEZP0lkQmEEs>DtneIWm2x*=09M%T zxiv&;+jZb}1N@S%rv)*+1^es3ju!V!AlWtWBC;|-Nv0r}r3 zMh#);;l|faF&!M9ey_9;QC6G_c9_ zeh=!R+$&Ob5XaUvdzo7P zv3-QpePE0=hg->suHTXlXSKF(awvuD#~>BuOOr>`T(i*qBB8tWTvn5`Rky5{&yHEL#)zQo{`teH#>B9NA^n?uc;v~QE`#Q?QwDE9e^%a55ZKohNh96%7x@}4%S^}*2A=45!#Ig3d%Z#vTsqcx()pf@(U@-C#VZEQxQ{k0sO{+ZyL8Q=d`tYt{4va)l zNkjQW&B3+2*{^GJoC-ep$^)lTq`n4?oq6Sl@6kH0W7X&|(1D3P%cqTe0XjQ(jo`n| z0y|hJ;El0)+0?mG(QG?>j84RZa z)!B{Kze2Guc(P|_IE11I)IlMW1I_ryxFq(&MY2Pu6_<#Z?abN@Yks3Re`)7cew z-xUE158-H<_UdU_jTB_7!`%J~ns)51N_pj?a<)BZd};`{mp_V<_V)rXENB;5n*CIi zy<_c;qo1f(WW}T*VAMx8p;Rm18j`vD}JK5|3I z_KtC1x=r)3MXoG+|45n=@$7tH;El!m?V}s;h#D&_9)^XNh7$oGof*R9TFR$hhFLE&bKhr z9-Uvc-~OYYl(%$9#EPnF5%(FuOws`Q1;60{rx4n38tg0!&EFgd(vDOP)HDP&#lc?T zLaop$>*;>)QSXxbTZZ!&ehw|hr?kXXlqL?G%s^BkQI=NgbPL2Me|Nj;BM|2v+_Cbn zhVvYS=qA>FaqH^oEg$LR=;L&}w#ev0Xjq5PctRQrLJzox1w^ky?ECYH`o`2=jaKF6 zSy5gt;;n4Cg-#jr6-Enof_@BT`eYi~*QEYY`tLSgZOo@lD9zh%+#ATpIb%=GE(13*Uh8V5U#F>@@j|ba#QuuMM$wt;4&7ZGb!9FB)|85~v zb{=2&$>-8f$c+wxzMcA1Q%JRM*daZGv~k*RL6oZ|<~Xz70M$D9hd7yn$^Mzwy<0v) zV;D$^DYn(U#EB82!Kd~2iSwU9>2r7_m^cn%#sPah{P>V5KY8TQr+uo|nWaIa95>?n zACP_f*fp}I@GePrei-Y1U3kyg4r_pUxMO)ggLtg{q-z=csnHKrwC#&$UnY#37B46y ziv_NOI3~_!!f5(1mL<8kW|CJNZNo@bVro1FcBk#XmP_8y>^d1qB|5!hD;F*dfUAipd{Ogu35_q5uMy_@N z62vpX$e9#CWWB#cyqS*v=8M(S1+k2XSvetaUv4KyxuUuhNxO(5yB?W{FTLILXOvB(BjZ1}Gd~u-Y)crAfg$>&*3|2>Wz1A)$ zaaP@d4u6UHk&zHw8WH~e7rc`oNerQY`i_zmE9HQ@rakrpubfmy4n-;sL)7;TI;$$cgVN3JHy~++58+8G^)(XU ze-nh8&lny$ev%+%mj3@c&j0H?d*yyT&Au}iG4bPnIcO%5*B`UU1eD~*NT=Su%T;m= z2s_KZRl*#`R#-gPPc7~yeDi4+MyFW+$Y6>@PcM}hnV)_0;Uj-QO2$v^>yOASGoBQ@GO}k=d@Ov%l?Ok@!ixx8LBVKjwl(@3 zs^=AWc*~P9Ljzh1XbPQo?MlC()-jg1JoWy3Iz=>EB#pz2=ITD)d`a|Hr^IJQrGHv@ z&PVNQrS?QRp50A<$ZGUufYDd~wy{sMnbxJ-wsrxvD(wbafkr=#wI0xuC|?>&6c*0( zr0;H|0>$~Q5_fmCDKc2lt}$_*3-Jyvw=*_PtpdD|ZLI?<;0TraV`p<$t1JcMQDfqk z@kpyzrY#KGQg+e$<*!H1)mb`Z)3Z&FZ49G-MQYk{P3=L^kl-P`>i-A zWvwAZYSOh$+K#0boFWobt=~&Ohc{Ad#{jw?zv)O%6Q!;ZOBMW*6kXk2A$CXfP$e#7 z9Pp~4(W=-CKA*FYcZ_V|7VS0La#Je%8xQPnF^YF^uBnHUK}M9#O-={V|RTziv- zWX!TsrpLE3yz? zaWg>ov;ZaKTEGs7R2`bfx+8 zm3jfr_yzubrze+QU$dkv>CFxjTt8;?6)w%gD_+REGLe0qZsj`^>FTNsQI5<(aVL!( zm+2-tehB)2yAIya7m8AR_{X{6!NQ@O1Li7up~I3s>x;2s`yqWm776olU1wd$XXH_w zG2k|VRUYz>#Ht&c>8q#G!s#Pq<5Qj3tBFeLb1TjF-aTlfWIA*okSr5moWD|r69+XY zt)N5ek)`1ADQe80SK4Mq(^GLG4g=E$dLPgF^&@nhwoi&KA|3Y{FYP{^zBvn4*^0e$ z4%i_-hSN%Vy5x`ArKmpi8b5)f^kgEe(5?8#hPIE*m!6LovA>TyycP!@4`^}g>$Ryj z)ctt(Q^w=D-LPI(W-wUu536u_iEEW z_G)e8E6G`Z$Y{tYjI&K?;|~jG88Grq?)@h@%O`mO09O?V?3_TSmDe%-j@Cd@#Hk0% z4%#HSP3HHhk+|1qh17Cozy)^j2=U!d_WKD*(YKAi9?pSoFJeBcfq26u^Gi8&vZoP zm7c_Wol&&JP!rHnY$M)r*mRP=?q%8GFYiOAA(47I34w8M*1knGy5lnaj!%JnkCAo= zQI+nDdw3z3lk8B!ZBrQ(B8WD?3(nqfU5Yw0ds0_l&(P-fSdERIP#)B;j^}G2Cw34f z6*XV5pl znDfWXkwtPh{yIModDbP4rXu2?u8yd-mtMm!BDcQZ zjaphG{Te;yvpCCLmfTiX73k0RNO9msg1ij*3SeEwHRqMF9Ds38(j?KXG>QozhdodS z7x<~ih*j3daOzm>|BWw0=(+)CrQs44`OV#lRv(x0>`qhg=H^=;ps8B`x9@ohq_ z{*la7QqIKw{cu{`5i^Q&U1JpJGYuI0q~E-Zd)U$39~^Z#le~_4;uXx^lW#46;LK=N z0{%X9sRKv1Xxh(6<}Kg4?VyXs?k|$Zo&n~|)-BFaWE=E1+@usFX}0Dz73eN(vWjM_ zm`|Q0I8b_Rnda5Y<#ojKIPf=uor(6KbefI~wC5|_M~}RaX1CAOnJa9@N5NDQq`Q#p z1_GDASnh1amfV60p9h#u^Mtr#MOi)G#%x#M@bx-%X!@3$nj9xyJZgt#7SD>MuI+YFBRc9U2|9L?QF+8Ghp4ce6)lQI9AD2s_JXYesTh zOx4m%_v`lO9}%JrIn4hE)+v<1Sp%Fl4RQ}F+f^+mq`s6zpB#FO_zSiIUCFxn_w=4I?f}Q7P$Xty zk?KX73k3o0IOf=Iv(IKDE1tigRg%CNm(M+njMS;mJ5FO>7UlDPZao_iTn&x@F%Gs(wVPNs`ygSNuw<@ zIy6qvy3B9ZM7}V0P8BH}y&NA*ekScYK#-hjb1l8@IG5-J-0hLS|BnP(LP+T|A9Qwb z7Fb^T!X%#K>-)AiCXYYKjo5QP0A98dd&&-*b*Yb8lKVA+vO81!&;QrBPr}fKirBQr zvpQ;Fi29y+<|Furx_yl^Q5}0!Ma9^#NKIx52k+o}r7&ik7~U^rX|_YKgSgM;fbNb7 zlM$Wv;4wle6022JwUC{QnB_PxNMk2zlxE49I1pOzi_%gg>=KIuXc75Vy7I!k8R1cL z7nhz+;X3EXCSzynk{VbTyW`Vgkwf25TVk{7csvR`EB=qfsIv!TO@zFNo~jQHX0k*( z2d(>O_BV&+f?;yV#y7dU2URd9W!BCU+3Wap)4a6gdowBYs8y;(Tu1 zAaOaXBmfO3uI3`Js5d~TD?b}m=MODS7iB< zd4DpxW3+haxXnlqVpvo^lesu9uzcaAw2$;f85a_!IwaD1JJ?_#w#luprUU#2Cqfap ze6lE1SA3~NoSQ%V;dHrI{PXKdlX;it%>q@7!`vi$q2|U3SLD~3ihSs)&<{G?F5fQR zKlyb+^^@U%$$EE3-gtV3D!#IE4 z0!PNYWv&0MD@mtNc#`d$6aflkZh1J?#+7H^*gz`;MbF?4X?Da8G=SMwv(QAA>2562wYc93};sGD$K3e3b|Cyf{ zNNP%0biZLtWVE02zHcs__8mKQ!#&TzsasaF#UPIN-?}370O7A-@itd{#Zu>H5RJ5i0O<80;7AocZnTp zn#HGf?_BgF|=OG3$bfvV?VmKA1YuUDn2%3t8_lKIxc-S7VQ$#KwZ zzK{@W5}gAVOFeiaH% z$}SP_!W@?Bqnp!QowleyLMuMdfY!p-pYnW308=j4Pb?q-Qg1hkE7T_srCK9@=#W0T zO>PJ#lt%NFyw+e_u*}Aev)OHV9%FfDTkJuPCKXkz08xq-&NNsUe~8{|1Ib5zH>%j) zhN;mN_#qv9jx~Q7CYq(#vR{EmHOI#*4yCSDMJuMXw!LVkL$iw=ydE6a5A%_#PVgX% zTdJBq=cbd^Q{c0$<8A^c^DsO)g?s7m^OR~eXY@3g*q_~|zF-%Mm|DAm;Q47VRVlq6 zc-N+u8s^U9xkQvhiGdJkWHTKzWT7(B*WY%V?ezM`-Fr4C0&);FnZC?Bv{|p% znNJ$KDqa)l$Q%}&ijS?rrJpiAVAI=$sF#Po@Q07v%MNlLS>xbbB?$W0Qij?2sNG;N zsZ=o}2W9DBinAV&F}<#=MOSXVfeZ>xbhw8}xMY~zWJ>-xI=uh!Pk{Py`TQ^8cO%o- ziXu*;*&fgMpYbhiAYEkqisQOkk(lqGpq%Tg{tmasg9==EWR5g$4DOyM#sUu0FVD4c z#^{X=m$)pE>mT0J)9HH6bwI3%xA1jxt~OLTidbcJa9)>P3FH!y#tG2=(7Ta5Lg?k6 ztTK(iNJ8sXAM#@Y%S*YKcgHXvA*rF11O+FS8J`Y;@%*eQ|BT$`&*)!QTsEP;5eX^7 zT3M8**o9>Zme1<7k-0z!H;nx6xv22FrldA zSF@W&&MNP{K=tLxY?B`hc=2}j;}vm~9G=;abmD~EB21fdfy5`RA-s;&0O0FJZdnQv z5%(qT;hk*bi3R}m6)7f`xGE2rH|s;=N?0V1cWIx?*H;a^kO-cC%8eAd9=Fizdlqsq z9k->awKjFE8waQnJFvxDywPPzWCBxMsroz?ksNSoQwS8x8}OClY^~azi!(m4!hZJmeZ+FGRlGZjrmnogB!)Kbyj;ib5VNU?X0iTbf)r>C~_3| zqYM{@dc*I?-ERl4cOpV#vTioOdQ&HQL)RDKINXFZTmJw|?&$mK$1`(|B4vchmjxjV zpbOgXENeU#BsF_E=wR3J;l~RBk7a*^>4hNGlltJNsxGa>bmv6pSjL9X0VAFvWR`wu zDM-%AH8Sf#%hJSdda6qhIPe}}Kg^N=QFxe{-ucsY0Z*VH!)Re$X5YY1*@Hygwh9}Y zl2kw`eh~ye^GAm2&t$ngX9}f7^TvL>Cg@*ar-ObSTC(4!yvhSj@2`@9?T_LBBxeSl zIR7$58T|RMt=OoOSg}3c#yzDsfCJmff#V3ix#JhQAr$?0vdFg2_l6sr50SQ6b=vO- zfQB$OM!nTGR%=N>4wyS%ba-FJ&Zdj00Obxsx`F`L6&;j89vTH0MV0l#R@#ZlzOLQL zG&zLaLSP(fql^Pj8_Mc8xn9MvzxSx_q^>{o!^_mea>;T|E*q~UvQT0uv9{`1@|o-w z#F{|NLUVU0Ts=xSF&~33Y;?CgrSi23P~6wx31ML}+In*#WW`0AKIBzVDii4vckPKQ z!b4VZ&${aXw>692Orx^+zXtF7t%h%mLsd`V4{%m!gepcmoM>!&niaB&bkX>F2A*is z3)06+1O?B~7TI|di^G$F#~nS9?=b5GTbn+DYDqJA~X5OtKu1`%3Mt^Mu`&pHjw^npz%JqYm{fFvwHR5sNA7J zro$H+*>i z)^Y5@9ON-=LKRv$ zxXO`Wc>?2S-X;}J_?kP8Cnb}4gp+ike$!}d^haSZiU`Dt!9}8BdaPMw(}lJ13APO* zJN*Ah0%R_9w_pMIn~O-HgJJyBcZNM~-BU;QvJ1P+P6^l5RP zqhDyJdWi|6WT{GdG=M2v!kZyy4>FOkJG;kzNyNF3D675SnHK5aMM<8>>Y)IxIJ+F_ z6S^y2qs|iyVA~Rg&clAeMWYK*kAmn*2C0?L+w-ol$39QuFuyi{QHX#`CjwCc)=LPk z>%c)3<2S)5kJd}}nMG}L^jHh%uG<37EwoD|O_-W~9uz&)7 zQIhAY0fogBgC+m*wS{j!7m^luR&=|7TuJzbqaV3ocFEeF{ml`Hi7;|2)fdIe*|*d2#SWKMCYs}$Qz&J(J30Q0IG^mtp0ep zAe$#GS6>JLFGwhxKiDZ|>Y zOY|1534}WiErpA78#iCLjkkC|1GlL25g8}6Ef%BV+TF|uvb;oPK1cAk2 z_T7f&tW-*SP0(jcTZI}WHKHok1d={dP>o8?r|sm{;?S$ zr$2T*)$Q{Zu7JT9Dr9tTCM&Z4YE(o#rM+=(N$Rb&3oAe4ZbHhL8tfXjF}JX&TIZy7 z!5FZ78}!m9jy|h1GQ=}vC<_dU;Ngs3AF1{Ok8RNN%Zli!oeYJooMB~3iXDA?#3A;+ z1EaswzK1OsgI1xhyAHGi82_~z+8TY5k5K;JZ>6gmxu5mw8MS6>ZZtV9P%mjHN zoy<-N3r&B1TX~FMPa+)Aa7tWnABQb*l2?=poKB8f9#kZxtm5|eby=$|pXRcqOUqeL zE}|<|xM1`UEts{HWnx;wz+sUz=-5R|KTS!#0% z)BV()vYotao}P{(!J%qNO~dxN&UPYJZ~eX8f<#c&bcGU~H7*|G`UaO0(t6@t)K4g! zu?B;f!)@=ACXXDH5gPmvNre2Cvnt82!xOx2ZAkFLS zD+jTISv+8PE87)lfq6~3xiFyqhlyGuRY_7@NA8}-^y&G*2_EPNTmyVs?O9=XX1&Bi zAXv4Sm4%6G(NCRG=8@2*oXQ^H@gP6)HP(n6bmJ$HypN3A{K88XEwvI`-`8xRLc3uM zCwdvWEz{cb%iIOM&T;PNE!GV$`u&><1e1TPMs8^rpGsEp)=!8mXe7`pb!NjOJ4CN6 zzYO?>&dKZ7%q`yiEt9f(|CoZjiklq1ZVXZk3YA}ipyP288Sh}LQlWk*RASfZK16bt z%706GYT`nx(JNb6^y8SzlT8BiI?|oh{OjE1Z@AdBVR8MA%Rdrp>5<$=hw0K^X$D73 zWS(G0r4?)EO>{S3;A=eVgEh{MKE@U}6~7SuN7NE(@jSbK`0Mg)bk`KyAk!!#%2Ai- zvKi5(XK656TENg3kdXhiu0=6QSBhV&xdx{nM$SfJAV2ei<}5$aFN*E(J@5VdVK0M- zWR=sh8F|C-e(pUjw#D#7T8dM-evc5oPpgME$=k|d)&uGrU4RtN_K^Zkz5)0e&mD|3 z^mE-fR&8=i-T0NH0pN$mE$lD&Jb(&!yx-SxH}>y*|8Is;+2JYo#f45T>1zK{6AinP zLicli;BlO<^qK2{Cx*xWv_z4;><{3_+W{_@(O;n5tEMa9{E(BW4qeH~k3$x;A;TOV z^tV^LEgI#+@U<^9R6WFtv_fPwJ|9bO8 zH&urAr+-W1a2n=wjZB-3GaeJL-kMaD)h24+xjB3!w7&PEe(1g$?yd|)y0qu}ov0_R z%=F)HC3Oqh_VgZ*f4jlFVax-ZbiMNQwq;=35~AkJZ{iNApIdcE&Fy!U(tyIgDmHN4 zILj%^-NkJ}n7UKZE^Uc0FaREEcVFTcg%d#c>Ir~GtRJcREOLUEl6OiS@Mtdwe+YFIr%AzyO$VR?Oc`&k}#J1OmTKY(YTM<-MDumWZQBj=G=ZPUh2m|Zy$Y6_=*5!9&w*4iRRs`e_$R| z^LP%9T2nY|ww}D5s_jcl9ZVKPRp>l8&qWs>a>I#u|2*@B&^x+X=h;NuqDf2f+=r%} z!M$+7L}LcAwD?2g1-b2jSwLBIxI7j@BC;GuB9MMAd2II-j6GWZv>E+je0q=}4r%UeY&hhmlteMca*0icdVVHL$iuabZkEqr7W$9Z@&Cs(IS3 z+htT+-PC9SpS@Xoy3C-(P9jv~xCf8J#$IdN8f|4bF+Y{tH4|Srrj@HO)3UJ|b^KyD z<+V(uldM0rb+`BKpipNOPe*uD4#jkOq_oeeMhFF8mZzT1>v29lv#HZgx?Omwqic=u zKN8765(Stej=0O(@nGu8qBx@!mc&f#_&*11>^W%-gow3a3iGffwan#qcYCkL_k#a; zaNfB^y99enWP}^~xvp<|1ULbp@tD`9Vi(v}u_NDL84K47V+MD|{?4-X!@Vue+FtsyuV@^=ixsw)7mC@MhM9Hrug2Ds zo%H^RP8bl{mHM(gdQ}639s$02UOMBs34M%LmQ(B0{Mv#Ppz)GD7hf&KIAvffK#G{Z zOMrG6L$3sJPT2@1U)giX2>(aBwuW@>zOu_gPq>oAY2`>s-gUZw7_Mxm6-nLm{B%S? z!z$=H_~!HPZ*DyU+1R-at&dPFZ8-eu+PAbdU!~D3s_|%R82=EQK;FBw&lpY^iwC}T z_b_(w6u2r2ZkU8L;cI(%Xwb97QNe>#mJX%?n!BSPq^tOq5tmi)$ZINvJKl3Hs>i&@w$m!aD4|ZjhXW zN*#U18jBmnZe5pF2D$b2@Z4!lR0J0+sVW_?^lRw0yOyiVMEP_Prd0)(_@Oa`4R4fP!gJ3_%T8QO zjIA1i?G$h@5 z4t)6FMKWb|87zD`ve@Zgsmr zV7G()!b)K6H^LxPr&)G3L2;LEn4ODk;I@)DumL?Rn7;DC$~cZl^-cCG$U-mL`RT(} zf>HI2EG3-ESYamF;vF?I5A#b6-~i&?LeytUPW*m+ZT#vzIsF9UXz7fjyLr0e%;{UQ zmfCZ`gZBi!xs7_y!arAVdD{~i@5NuF*;r<00|dV{^G%|0KB+< zdhRaf6xV-=jwD;ckXZNp6h^$<_6hiCPOYz1`q*-hLCUS1gxP-m6coV z`t<-a`ogQ5DuR3gr6BLcyXu~^2o zNHdIGe2A-nw^JyPEpW_+UT~ku|?+5JC;9P z2uDp=XHLlU2bV{unNObo+NqVwhyRL*qrSwP!u8Q-e-87coH#Bder0zY-=w|mcX~RA zf=FJ;P9A!d5&hV&>JV@F+?K!kfxhPp zHlUpKa$NV++eTHnKIFZKj#2+hE%~DxB8Ez(M`@GKKup2>2)b=Zw zJyW0(q>8HC)|%i?oZT8A{Mgl*^fHz#9g>uTJEDuC2EPv6^^h$0BTjRswjVn(8ThKR zvPRC|XJ8?1kT6T_po{R$;B#x<6L0DdY>)nQczEBb%s^a9S%c@zTz(a)Xq?bSnBxHt z< z+EP%Q3s30Z)UdY*jW6kVD)qBCpKj58UDZhQCmo&lvl=-cD>5~p(Fc&BEW=5sP_$7k>tFxc}!E^`41Eudy125 zs#rU3Vdo^%{9q>E*s68jkawOyRBCC1C=lMeFOz*%TZTpEq zlK%tT?QH>Ga@or2Pwb|<%t%RJii0M66sRoK|ynj?9%8LDRRBkp7?Gi zVblwRUaZdyUd6_eo;Iw7+*E>n`A`Yp{DGtqsjOfNeWbI}-{R4>mWM8>8()MCb2^FH zsPC*B+^J3RQ-{@RPvvDlGlrn{eOEeeXdrIYRc2aB({-&QNz6+1afP9rHTORD>}4B_ z!qCY$MXbpN!WRHfQjs52KYXuolcAyP@xxQbhT!bEp~3zKFb(dLOIP2Su613*NCM8s z`PByep|9hHYeZHLl(vrSo%uv0&j>+H;zSD9mg=G2iHreTv8$h*r=(DA-ZX><*o>t4 zdJLqB_$pouj)Q?ZamEl?MVmL$b&PBD$%(>5SR0D-JMC4;Yo8M^pdM!=d?C%E0dw@d z><_9!C9QN3ll{&i$T}XyzIAPY1jRu=@%@N4^H|fYY{{viKozVurg<>o_tgPZ8-G5Y zBpRJ?7ov|1L0j~@JNOf$F2mZCY!KbJ**Q|(SY&X|#E%r9%uiEO`V}sA8;*fD-9~2x z#x13uC|f6iQzmME3}}arM_+)5AP{Wmx*4SfkprLe1{JwRs9;pt&z73x-HJ=9yw?TH zSAZWHtpkd%U~rxT(pe3*;fqHndM9&z{?7}Tzp{ROIe{F>hPFvS>n+sHaAdBJe4TJ_ zu=mRs0shh2yOWV3pi*Y$_qC}pXr*7}3lYyB&;D|rqankwlmWv-$SKC)g6#Hthr=In zM~kmM4J!n*+kX?*p$}nocC~}cN&q&@|IR>>`YBEiHx~qW-qC8X=v89bSMEhC_G#wS zE$=@?kLlcQ#ZCT4Fjc=1mjzu#j%VVNYZ9e2zP}fG^J!o+!0JzNmqi^Y>I_Z>3x`n) z2tdr%wu!vsDtMc|d5k5k2GPWak@hJTOtq=&#!RZ9!PaQ~gLkw4E5qrN6LfxlKCi+$ ziHmGTdLOrZ@ESzY)njN3(MO5vG8Pf+cN8+|BT+nA6UTyD6z)9DyP;U!I~`TvMIZcq z_ZxBiLOYWqXd$$Z0Yp{*6Ytd#F1C<%m^3JQV!X`yOn;X%+452hw-Yo2M!y3dMQee} zV_#(*^T844hN=kSeMV%n-ZAZ}+s-P%#b6=Rnawc$JkDiSaI3xPA*=;l5y*m$xai0B zpp`7g4+YbV#;GC2ll814E>=fTIPG|>Zlm0XIoLb&Sf4!A9Ox}Jtaz-jMj|Z(`9uQA zNs@<5jGy@j?9e14oh_xqV-2($57OJwAoZ0Ahko{x`CEH{*V?nD(s%LASLOroFk$GW z#5zk~3fS{W_$~pT(O}$4;zoN-H8urF6)_k34s)Jm!Y4j>B-(OcGsQ}B;|0OWlOc>FxC_5(*X1Jl9W__k@kq)B-u-NR_ z{_0V5f)P8*ES4(Yf&a~?RzHDMNx6HI*dtocwRTyY1~#h5d=V#u;ioa|-?wQ_mWt~> z``~_3h0JF4EN?b~OD|2`{t}Z}O<0^MN<&}I9R|^kOPK7@KNk;+bo=yCas@kdz0pny zxRiR}u@;n-Yjd9UO|ixuIczf-HmMBe-{^V-<;u=3*Ph~2xO0~vREY|O%3*NzhT&~>ou+XN<kNs=YB@UREhOZ32_ z8r*$tFB;Knb4(0F38qsiAE!TRK?qn}UDwZ*%a0lGeb+^`zche@F5k2f8=99#oLA3o zo`0wD9h%+n;)&Z8p5Xa(M|!;0mLzmRah)UzYzItI#w6Wr#HIqcbvTcC>V^C1%YrjP zT}htUHG^YUoRseR@h#;fbk!Xi76ir#V%QsRBz-^cbgnRcU+{(ZYNvZS=AG2nw}X+0 zY9>G|O>qEbdLN7mKHT;{f{qH{6Rq#j8njm-M_>Hk^#6)-{trF$e*{t9VM5tS%6h>q zAfrIwTQrsa)wej+^yB-xAO3F_&a?gJDtnr20MP;XS$@S+Ac#PC_>=OvbjtOI55N*- zdIzQBS&|+)8O)^r{EzDIS3mDRD_O0V6z-%Y*iVuE@Zmji^p|SQlt&G|FRuwYNdCQ; z@%us8_Zgs+DvRF*evzjh(C@yB&=Q;4YXn;M!Nh)IWwu`k`v{=MtLWD-MAL3#=$dP6 zC3;t(QBZ3Pq#%Nt?5GPAbab}WiVx6Vs`-5@VzF6^w%+($i-vE|ry&2_`}{TF{=`W! z(|cuLKeHDFq#zD%3q7e8N!BV;DvUL2)W?eqS#bzumFn_?vy8b2AAR|W4s?4?09A?) zV~-CJ>)g!&*7cN<#g0FrfzxOYA$#>Yk~HoxEHO!JJ}IOus4tZ zn#pE_v*3>69JIb}JL~Oq-W6nCEm}KwfK8=(18A~$L`NuK#LB;$b+Pd3epi-40eD~# z1{Nx-g1-1vPu);a@TQ3#)r*F^KG*E@)+cuFCoxW~t4j^-a3;$X>UU+|t4YP&)xSQp zW?^>flD3{m-U1#KZ_k8+n44ZCRn4Q1H z%k@HN>&KpH-&iyr3>9xMvox<=ozUOaN)K4C$zfTP3?VMsPaFloI4y-8AhOWb89U9K+ z=Xoz^y@|a21Njz`RZneioLBaWh>oR(Kw4VeEhVV*b5^~AV8ZA#bh9@D1@3`8pykf& zAL#ht+L5`!z$!53(6g24Tu$Ayp}s+pYf8O~GhU2|zZXqFmsfcpO%?W%5B>=I>x2!g zSsgp#?%~$O8At2Ig%2&2rF7h|tD~X|S~UJ<91-~uri?Wry6;@q-1Ew#q6U_GTiGFqCL=;!9mX_6XOE>vpui!#6v`A=UxFzWh(cS7q);o( zCb8fEJ>b7Yx^yx}xpX4xY%V^C!%m)bimSanO)&ee(o*j&agZs*2n*j3ijwE}Q(K>P zX#8m+R)ZEI(jEMzClps}sukVN?VB}_9-(V%?i(wyA~$sR{FX!=slP~?M_;SnReNgt zlReuYH2t&u)KdJw+?EtW&Fz-+4Sioutb1W)PCciN8c%#qEuQx3<=RIR6ScdX-k;dp zX9TM2Mf-}3$^>z#9@Edx%D(hnKoEQ5ck(f70drBjAS2CalZQi2zxSUK+#6Wt3_xD% znUn~&gF zLt$B;wOzxev)(Rf`|%u&PnpF-@*Cm%vXv@Z--(sVSIM$oc{yd-=p!-=eqJzDe%8oB zmD@b;Y9aS7UvOq{MNW!EER1vvvEH}d3d&%j z8*x}1w!d^Jl55qJ)cOH;nv3%V;8c4WTo+D=pB!E!*>s!KXW&B5byudMcyfo>(x%!= zaIb`TJ12UysFjA|UkD3Z90YLor0LXteMd)iWQZhb;9aeS^=45;vQtOXe)rSuW(R%d z48eMqKwjmWj+AGKnYKQZPq}!xJZZag5?5%wmcVAJ!4Ol_9(?tQYpKASlkQ9s1M<{T@9Kg+5$2dKA@iJ%>*FS2l*IKVa=6wbx<4C&A#j(l1ueu;5TVUA)++*)E9C0A^ zCy#ZDZ^jkvXB?;EVoROZ*HkX&n2b~($A0#6t-%YoCcbvt>S}cDTGq-C?PBNB zY5>KF$MG$3VDs8>m@!pErpzqB$++tW<^b&BaAB-vrLk|k!AySCno~%9zl=A$NMPw@ zkhu?!_Bs4-?OSKR(NfXXVl^cr&gLm+{IFFjrd`K946ZZf6$2AC30q8GoBM9Rd-=Rz z)4IXKyrW1tmH|O`f(=_jB(ljW8&PgJ8n4j1hfbC)BCg2|G#se_G62t$Q|~9WRY@eL z0#Q1FeEveHl*Qj8FN9=lm@3FrrS%WpiY?>$7&yiMg(~;`eQu*raKXm+)8vC@%BX!g z^)wCEz+CcufW83h$!Cv+c*p0?f;>-h{3>GyLwpQ4{Wdh7Si1-El|Mx61av6S{}y&W z>2qMcm>!%Jd&k)$o>1a&QZC$oA8?cKLm%@YwU~X!bTGh`Bl{}+pZH&4+u~O5Wn2Uu zI1XasfgH6MvpTS*x=v4El9Q-9my1A0Zw38ZczzoMNp2g(GR%UyJ9dD% zXA`zocNApboBbN|;<|wek8<7lkAR?n5cHUd2e2qf+_(oV%7f5BAs%a}J6iH9+Y&KE zcq@u2wvLGp=mES5=m#Wtr|GsH=(ePy^d;+|!3OlyR`fn%5-2*!j%ZI*o(&f_b#7Gb zog_L<+$ts<4dPf>FvG?eff(9(6>p0J{ReJ@)U2!-?uo=5*9GnW1N{&s_aqGk`IlZSw^!=v?4C|X7(kJ-xS1uh4hn%1dYIIh2o(O7LmHfD*WLjc3ZRc z;qR*ZO9%`%%wy^x)o&z~4pur!z+tNoWU0^wT>u{hEthFLzbM2=1c?tiZN_t3PtUwE zGVLqt;6$%pjNE49pWg(+cDIGgW8e)etZ9(ty8O%I%~qaEE`i^>H{O-Nwqj z+k)a5A($CPpkg#FawFbQjEt}ORrK7b4l`G1Ebm=wkB2I6Sq;wqc)p1?>U{8XGdCTu zp5Z+@*77o;VeP^Yn;NZOtw=``U@xxRs(+5{RcDAW9AiN=uUThay*&`j8+rcE zv<4b=(-CfJmZ4rDTRY!UXH`+WU~-Ho%#A4X)iSJ}IlFO|Kbd=%<|*?rEG5r6h5bWU zTX%#VSR#|)0FS;3HN)ybxm!-L;r(jm$5uCbZHU)(k*u7&1Is;Bt>e^FbUX5-z!1mK zeMx^^pB7WV9E_?BmI>~JTb@ZFIU2=;$Z^{8y!DwZF^;3+_Q@!vA+^`nzhjS>kOX+T z6H0OKcc6*sLLPR9AB&oc!1bK_Fad(@k8$@YQuF!ott+^!Y7G-FEHaSSXT8!@-u0zS zc>7^MEP-U_0Nu!{i9?)b(Xhpo9-Q4E$;!V!Z^%?ZV(_~W{33*N}5{Hn21z)_7w>aJB<31<$ zZe@>d@%idSI)26At?b+WI$pM!g>9i#jr@ARuBG{Nru z3x&NYxk0#O!;F_?dCaPMY&-%M&loSodOB9&xVU^ZEC1!kd+`#z_b$3)E&F7LT6p^~hV`MUKZx)-;yw6t~pP%+3@-oTU zLN67uMUQbGuaxixD^9WTMa&$#2wLGs>nJ(3;7H2F8=^1#C0%Q)#`vZd z5_GZWbp5ocCczUzX>}n@|p5(&7)8x*$(8DN-wn#v zZ_il&eDkuUy?oTMpg$~jC)};Th8Unt&LFUS3ZcFJSoBePnOsHUX{9+5B4X|R;A_6* zamZY@)&Ea&&2j8C9-WE*xnjImqv48A_@Vru*Ycj$$iSV3;&(&4ZdRD_e3tWe zD3xa_vHimOD#PktBBE`BPKQ2{)RhdgtpYFpl8|L1uXOXP*9fN;TXHyhB~Kq5>FtBB zX9RZrUp{-5!=7WgJ;L6Vog3`UK67^56G&k0+#F7yn8w59)&x-1Q?^!^$HV##MM|E( zAR{M7xzwKnlYa-Lk+I3Xt_ejW4Z;35w{DTnMQQeQInjYBz;@-RuP!2o1f-*{X9Cc7 z$!Zw4NWOlh{2|6i1&Lc<^MNxvvE7{X?(Kf&-1?#+F@`ejg6Zo&%m^!{?vx&MI!+nE zib+giWi`+%FmEfaU%kb2)QVLT4Wlk?(M_sSd20&lO-(9dp#@7Yrj&YY(h* zSKfEnDZKi~q#i58%Id>Z^uv5-bh<+Sk&BGC*rEI*EQ=y~V&>&YE$7PJ+i81_Pg@Tv z$s?Q!Re9{Y<0Z$S*+TLx_ABg;t%k{m^o<{z5K-b)Lrj!p!td|7$GTh_OyZwl;+EVI z<|bE1yD?1PDG+qN1R=OLHEh*{{n&;0sbRYa@6$A>Zoe=oo6H>sI=WAk znRKaBeVWF@m$DxH*99*)_o5yt_sbP1i`=aJmY~^H5j**WqhE3C6=wBe#5qo#vmB)7 zd8fq2Zlza)SKtp@qR)?tTACPRDTys9j4nU$K6k4g)^Ockk*z){8)r{RM8YVrBietN zO?OlGu&;b{|Sy_X17^PjNvDrSw(}8q} zA&MH8FFiH6ci>Y@70u-|-&mam|0INYvg8W+qgk`E4-uifJzD*h;0vHayy1+yB)K77>Wd7jwN z)gUnXSVw^?+hcG{*TN`h2vmI$hqmsS@=fmZt{wtHBag=ZD3bY(@jkjGvGBh*Y5QZ9 z+wycC*Ap*UdvhmNE=ku*p5;sa~nAT3S+bHIysG^9G9_(bakEhOkjq#ax5G2QBAyW(-QZewQe-8&y-R)s{R7 zMA)VYBmXjXj@OUTu+TcKyV#(y z?iI#h<9Wk3RXvZ+9(4S@&fJClDOTLsXvrEkMl2u-bAFNjigbGODwyDqpfo>PrXd^acnc!j(ZQu?aunisDEg&ghawU;R}VZ?KbR zNy^JU>$TN~xJMZC5SL3lY1WNH&)pb5BPD?UnGNhw5r$CTS*q@iIPk+DN^Ap|4)=0a z2qINEU&YmJzGB09Gt$+yH)YR0bD47<`|e4~P<_tz2*a|Kw5w*sndGk07jqFTbPdX2 zUuutZmXa8NaU8w)kS}4DRLRbNNwUQmn;A1Kt!8ICG%F2K#ggG4aH3<75S9PQ^b+BV zBl_RN68hRfah6%`NU!CRUJr7lhu~}{U?aq)t?9Sr(t)Vk>G!>JWzZpn9rZ6ipF*-P znnI&Ka<-oj{0VxBO#sBEHbL$tbXx~PT;!CkHYFazFb+zU`cu8t=p^zRcCTFy_W>Zs z&R~4HZi6m+jQ@13o~T6y<2T2vnAl4WqGxj~w)~lfL4VtkXSN^athBext3LcwaC^mw z-`l&I`-Qc*XrJ>0lf%2m5>ma*!Dz>$le!9HPB*KB;-jaaKOt4V&lQk)d!`)`0F?;5 zfV}uNl?A-L`o@T?ZvU-SAYyReBHR*)etz*4@c9USfu@t6NdZ8b@_?!>L?P4*C0aJ} zJ?_%wlr{eMbz^%qu`@$ujLbc$vgDN{Q-u)Iu8#hU)0h7U=)r55(3<}UP_~5{?l=4n z7qg8{xi&$h7}F3I*xJ-_(1$U}S0{-}Vo+IF9SBDQN0M|3F_>YYO7-pI=ck{EgH_Wi1DPKJWtPZ zyo$88B58zH_yrDp6Kh>&b?|pC8)b%N!)3iUI+{R0(G2=6IC!-_HH16whW9}K8fVvv zMB7F$2-N$MT=KiB-RA{$lPHK(&7G#)DB=h40=G$CCVO6l(7YHXIv3A4tA+K}SdoE2 zRm5W)ZTv>Ituy(SM)g%A8rfOBu@Z~&nP z0AlULdDp?w^oiIG#jHDaLc+FYZ7yo0U}01fO*|o8KTaFN_8&nM>8#7fjX+WnKMLsP zmF!xh%uXJX8xuQy68=tkD;Jz)SK;g-6k&Z1?{tTVVE7?(_T*}q;WLf zzOYgw+e_M)X@a`|i!cG(?)E3rT5+hIs@6A#6M_sb=ODlDXOA1*`$a5*H}LzjC>PYz z0KYu9!Cm#5#HqVqNa4-$EI|l7x0#$Q8)myt*7DaEy{(NShP^))aRWC4Ti%^l*mB^V z7dXKFu+w4_EXxplD$nF3D+Hk3xsNb`t+z$OHIn`3bXT=&8Yt2Jiye#GtooU06w++c z?(!t^XuWgjJ*p8;)T|&!Bmy@?7o!N>iyb%>FtVcdb5&@x8E@Rcn*XdO_g?-8UJBnW z&pMmr?5Y@PJT~Ij&`B_GTd)Td$MST+?s~x%WV#qSKoG?;v4(n2^PZkr?>+#&nrg4& zJ5@CG9carfYJ@W5gzSr;_Kp&{@3IZ;vzlJb5nUPd!KmwRju#UKL>(J3S!~Y<1U$?1 zZW=DWqdU+%H@(-%D6Nq^L6K#4+10wnE_fWhMnTZ5@vBhXPbQx*+_AD{Khqaw=R2Ld zRBrh1pdaen;vNJ0D$w<_c3{};;Q5c>dzr^TM|E5SyeO;esofHJK^6J=x#syg6y0kM z1jAt?FdVU%JxActz<)FUuj-!0{)6C$wz3?6Hcca$ObUMR>=*D=I8aFNR{DYCSuthG zeJ-FBsE}1hTY*X-G5QnT^Ae6=x^|SJuEcVQ9YLw0%|?D#Dd~e3&j>;F!d|qGw2Vo# z6=)`ahN1tZ8USaU&J&&r0wXnVK@t7@VV_ak>8~nd#so`Ru*HIt{enjx#)-eHXbImT zuY5~>&?YsIt164=6*QVH&=Tgkx()y8D6zl-LB5MF3u0`yMJWDB3m+}Ko!Dbw+~8y( zmYZ)C2QH)PT0-HC#zhe?EJqvGCN7UZ%j{m#!&~Y znG>{<`SL}joe!gkzYhq}#COn_y`TqsK@YKEU-;o$deJfwk6&?G$3uf*LEx02)!uG9 z9lJrdo-Mw?fkuao{AZ(Pq7%KkDLlgy|4ccmNU=mkj!|F=vO9M|U>p!o>53t*DV(wI6LkHvu(SKT(9r4e`E39LVK`5EeI0Ci%Jo9(#E~?F5 z=_W5Q*Koa8F-?obnXOwytT41;2I7u?O}#85#SInzSff(Xp2VPZI~+gf>M zU-lkU3q^e41}~Y_a?%2dzrfXg`++HRj?3~$5*`fbF0t)0*ij@O6&p@nL_JeYsL-DJ zi3y8TN4Tp%)G0TsWr0AfJCIAkoc*lx&uN0RSEv+t%J(MpEFPmcD^0LB$al4N_=nZL zM|EIwvydcTk(GUQNk5ijMF-zkBRAKpz@cyFO782Zl`Hqecw5OzBzvp&Ywsmb$Swha z8v%&Q0ZiNGR+f7`d#$!F%Qw&=T`OYB2`Cr`AbYTmcv(0kPdCT<(ukp(`Tbp6MXE zj87hZT;z|mDpU8rV7Sn?c8&k+(bjCcKOt2w}jX& zUYNx&ki_4=HlX<=So%aJa&uh;KCS1Nl@(!4W0SJk4t}dIsPe|z8lJn(3jJ00@!0^= z5Yd&MM)R!Ueb>~-t*^|B#lk*UDSo!0+E&$+cE(X?2SV~$G~77u-=f9ww3Wunl=TI) z!EHXQ9NA^;#r1wa=9mi}-NtViPP}LJ8k3Wr^Vp!!Jh5Z`02Oy7I2s9QBCbY?Air9rR`H7>%`mBLGhOF;??A$bHHdN&sWnJ zK2iB3u=i5V5jhr3=xk%AyyV=~8w?He8ecjU$+8Pfz9EOGM6(1WQ2F)zoBG@dI{T+- zaDK1DqTaVI07gEBB>mEutgu_+e~)aoBOSU>Ht=qpD&Bj7NgS~sdiYm$P25N2IyOk` z_TE7d*B{tAF4g}ty-|6ghzMJ^E#tK({AZk|v4Om@=X4oYkj{b5@`3bW^nhp5uu6j( z{5RyOt@FKl%20>=p{h6zC#avwUnq_Fee{%K)_Y7?M|Q@;j`XewkWfV+&lRb6lf5?N$yon*siLVDVv~_O!Dt)==4f>BhjL0GC+q*hz<(s*oH|DLqssMd2i@ zor~Y3&JNOASHaCDlQ-|Qb*OhH0_@+omjwBA>sCyq%l5uxA?Bs|O^?0z%(aVJO8WfS z)7`7b_eY6U(jMjp^0jPrpKT?{G;Rx>yTbH`WvTd;I?Q^{o|v1fayGSQuggsK4^895zk@xslVyKLevgPgS5qu-SIr9ZsVp>!~q;Yg7-+ zP|XTf)0H1xGt^iYdkwH-YNnv?ko@g`2yjC&Dy0}lHh$Dbw{}y_-obC!+#DBiP3}@0 zc5rjSxG1?WcO1C7^La;irk&gI0ywLptx^LFvWBP5&Lxx9@x{S1t&PKFLvRESALyZ{ z`_ew_Aw;2F9@7hTIL#g?vR7YupZ;^rw!rO;RO0n>v=m4C&}|0rygrTs)xmo$4xK5t zf}c=Umml`hV|d6|HTtKudqwVm+X9e67tXg?ZGksx5Jh=nbPPv{Auw~CyHnqY<@{!U z5>Y(d(V212fCUi(NCUd7#6b^y$<{!@&OsoYOfCIv6~(0L{EIQ(8{pejIzBWcnE0)= zvms)#x8p$&oiEQ?!dC$Z#hQs8us~({ckjvu?5{@JZ1HUv#%o>`M1W z2S2lD8@Dr!CxCFZHR-1mOr5i-BShln36bGW4+(}05?^i7>QJig^lj{|@7<^;C@u_2 zLYcBQQmKepMvl~cf>hG+%dmQtNLfc~FLca>KZcf`epV-JR?O~)DgP+T`r&v!B&MG$ z2-qKIVnMyk#Zh8>0(Vw5$h<;4Od%V0~`n>id(|7T_)$ zGfzJ=*npm-e{7iCS4{aQk}CgCresBTLbr>%IgvLS{H3Sris6&!dc8oSG=yGkuDe? zLD5&3ec_H+?Y8El5k^!OlAV&Dh7PeKUyeQG4z=`hg%7x2oM+r=CG{4iJ~@*OT)8I$ z)DyQPYRgZ?sR%r3Jip|aEiJN}w5ZI#AOFTv-svN)o_03*@xLQUauS#NQTuC`grE}b z*Z|$XA2v%rwh1kpH6xSmGj5(H&fD{-1R9|3tawszsul(>-jX^GcP7W8Ne2{s`y5Ra z3sxMp#p7I<&S`!m$UY`#Xp&q~8(RKqz-m?baw+v|iupUH9eZoBR`J~{R>-pC^+bU0 z158>0iR# z;*@=9&mZ8LmrreD$^AVmKgT!C#HUFG3Y)J$q7kuv&dpaMvDr@~QHZ@ggO6lxhU#=y zR)g*#vCvvXJ%E!{$~SD;kV)}~?c7XR`|>`_#b3~kYD;sf4#8NrCwT5=9huWJDsMiv zKDc}^#ra*KUqp}gZ9#>HIR50{w2Di%^|l-A@rSi=;;Nq^Q>)P!p$7-XMX(Lx`qqZr zjp40}Ces$Wp-pd!_}uh8^Qbb!yxB@cWcJ(dyh+C zlShsL@3^HlPu)XjA5*>(IQ=3+)aQlBT8`TOP|7TFQO}lmz%GnUUh9bPPrLB0m&kI% zL}fmc4{S!kcHIIDQhZ?--zJVY(zMXBr}O+1tX$RQv77OfhhIY__mb*QMvrpJ0Af0`vUm5~9~fafulrP&AccU7Ru#WgCqCNBS)eZ(2V zXFPQl@=0txS2^C$n=+9L#%r4EN8pJvw=Y851i&-0-2$IKZ9FZ0(Lr!(CZ1Bnl0ih( zl#X-sV(hbXKmAbg?9`LrKGWeZ-lpeEdX_?S<)nmxbU|M$6W|6_Gf70W5O!~pYr5|i`fRc4NDA71R< zQ4e3-RN<^qVoJ<>qX2Cq)zS0)H!itpKvv?YU!y!bE5N5(?9P945=eVa6b}n?%S?!J znVma4c$~1av_wD1CMCxw_3kjLXU#9?(9uzzGkSE^S)MoVxzuU=np{UqP!q0f^r=nr zvS{|vX6QBZ=e)|*{Zr09@Nl{sQS1>>EYYolRAC$^br>@5KLYS%t=zoMhuMKAWvze9 z(XbyjREmfbxJZEAk?t_!F!*u@j@&a(s5HB8EF+l=E4mY1R;*}U=)n0tjU~@)_>bgX z@*W^#%vMIbkGs6vwU|J;$jUg4|AxIJZzggbjVEDT^Q_eEYv+#8Q*I2Svn3*t(q>_O z^wr2={G0)W5SmLG#F3rrj2?D-&P~Z!8W^pnJMfUaR5c>P@ivS!zM(hF@7Y zXVg?IY{_M>{ z>$L$$IHUaTM94{94j2?z-A&Lm@Rr;jpfd$P15f~d7bN7z3tPnfECxhbr2-ue^5QWWTp?%n&3fTJje zX>rL0$j~;f|Bv;TMXpj8R`Qd-4BiuG=jAz{`9N@u8$JBLcoFVK zcZQoq$YxNqW6>mDHfU_I=lLSGd-9Pl$f2=07`pn7e&T8l`3XJFQ41BYIgKBWMH0Ht z5f$o9al~;Ld)Jjq(lfX%No$OmX=;39jGw{QuG3&+28-|8? z_QbI^AsmUb4m_}XVE^T{T;UKOAhR{eUQsrOBvc_?aU>u`Q+Q+$8x}l}KpBH%0oAT3 z{YSulc{eQa74Wwf6?_5S%Q#qeH&2HGi}+vPy@6;x-@2U|Gpr`SjcWk&6UpPN70pc{ zIa}^mcBBO6xjM|w@!fvgxrE5YO;L9qCb9k)$Rn5es3-IgCx;EhSjNLD7qbG5mn?m# zxVIQs58W@PZYPKKMO9?tljjp`a;wMB*gzM#w+R62N=hjW^)ycjjp+Z^MH2PWP-AzF z9~!+q5#%enyzx{;r(O$5a#(>8U2pid!rLOg!ig^I6h$~@zFc*zN|lBDRR8wPRqtaQ zel#cu3rJj+*o?Id+nF|Yka>*~e|1>ja0&k5A3%z>Oi)Nkya5Nz0^wQU`ds0#alEcy zFCO(0`RO7{+OL6uv&D2_Shj&AZ}^CPW=gXVPj4;oGE+WewZzxcWM7}+OWW@lUq-^!J!S3lw3aeogGfSJbPIKcQqtic@q0zTk_5 zTfnnwZz56%F`cr;Dk5e2=e^kEn2|k=IwJqfDL>T>9ff`Ox5~nlU6^3uX#ZIXYrUF; zUU`;gvsvl>6b>`{)BITc{x)kY5^Km8pkJ2y8sVIfgMR_-V#D8ucuyS5YTP{9ulCK? zA{fTg&R`tNyVLiamblgf7^BYS`L1@cSvU8557s53>dAyCj8{~C8{25)?F;)EIM$sI zI_F__VG_Q-k6aftuOmz9>=!3HP@sTDl}C!WkS+2w%)3b-U$;MY4%K@H5&c3UxlkI z4G2J{U)^$1*N*tnb`)d}eBYdP$Cbb2hU6lTLZChCcA#Pcav zesX@)Xasdl6^Xv#8IRJL+}XNFD|3`*Xa@LqM_V?L6gRBf*wz8JbyARFsxy6Uj59y|#U-^cA!DAC&-f*>Km>Xf@wSGFJu9yl4GLi#_fOsK^+cJ4(+`F#=WEWWz zB#V|TkLO;AfSQtKk>onYL*{ z(QQ&pixY1n1cT^?q0=G!)11?JApQxe z)D1vaP?6=?iJAkscb+pbR<6IYNb>5J9B{w~am~tV2UILVyl%7W!G}~dDg=riG?p$T zpq7@1TeIGvNs1ni=lg)`vri1d}&s4S)GUQPJ^Y7H8xidR!(ae;Xw^FlWJM|D;2^Bt-Q+i{n?}bRTRxf zK|jk*5YR_`<+@>k0;7@ZUgekO9MjY`TY8e8UBF>}qt4RRUt6V0Cw{E*>lc?Cw78^N zex4>(9UPGC3;Keeyr$|7SbiQrVJ>5!7NmSC(s&9kE`$_fv@Xst=CviGpVn9`StUpu zM%_=H*+<4f{@4I>Ohd=UvfeSyBtYOMf?rv)Gce+nIXj=!*UOKUf$}5~4z23>H8N^6 z7t&e})rx>9-MjT1Bh(3v-qab^sd8(8@kb}85qP!J6>i)+-DZpSQOlf>XY?geoqSy5 zj0q1`-@I5d;R@MxjM26X_%g>K5SjCW*ZJ3!Z1*HwU_f)`^t`&B>0A=>>zlOcKTJe| zmn-N1hFIZEUs-m3P#7^B-A1g}`Jil^n1IkI(_}%>E1Q_4w9m7waPRvQN2_94%6pDd zF6c2gBGi>e&y_#*_ocb`2| zo`fbZ{)Rf3G8T}RrO6R1wXFWxAH&Cwm_nfFlL;7 z%sSPfIWboT(b&Xt%N7ZG54r)<*e`rlGq%&RV`pmQ_Sp&kCvcjT{YQ(3O`n`><;qk! znG+O~X4~Ey-|N|=Y+Fm{*|U(r!YoY6{5VrCIjP5-3{*BoUp@X7hnT})FXT!)h#^a@kkoZ<@H&%`>}GO)6cblH3+Zchzv;Wf#YGf$MtgjB-Wcmmap7l> z@w&=YZ%wDK@MrochwJxC9Sb^2uUKLYh5pgl2+yLy6IjL43t(AmC}tA-#Z`IZ=RYY_MjnYT3G(0w0e-kAUQ zX)ws)M3R3edd9_Ki=?sM#eT7HF=%_n#I=%dw1VIW`7nI)%E)=TA@`6{nyxo=rT`BPzY0utF*+Rnd zy9@JJachUW&6~|QpcE3>u6OrB6zbp+i@$5XGu-x`P@S0lmbbj@q-;7xmXR<-AiI7rogSqQ$w zIyoDx)q2^C5VIltBV%wJ#Oezv{qlL?ha76J>kP%+3b?KI+AkV++SGa(+NsFT7PZb) z3<$48qOU*~Ew>dfT^Eo&I(GhS-5?rM#Xmj-`or9y14a-Md@|~epFq;VZ{Fdj{7DXx zYp22@?>5MqE{$LDEQOI66|#q78JhP+y72b60OYd8##m$}#0P0q=*Y|Sf`c`qve0^f zO^qiQ3^EN`4-lXV29-H5&-GRq;fyx8npzrDk>LZrT49F8|BM7n<-1dSCN*+!0^4NOt+k)!{-1a8h23tAf;{Rzy*dkqa` z6t|mDaC%e3yXwIqN@*$l3t@|+coNVIsM8`CM%7^}6fuh@K(n>e1qEcKf&Mh7pZkEX zNq6%@`kpR@hheGFp!MU!E%XIq{W$pJ$$HFAuFtk(RKiM<@{yKH=SNCn|D_}eN_mvz zSIV8^z92BL4t?FR_HzQlp8~dB4FQ@;u~UZbcBf@QM|9<=_R}PCpLo8 z{sJh)RZ3rF_k0Hylt4xKullc!IbWq%*<)gJ{%Fwm-#_Sg&{}}6--Drjx?6gUbIw{4 zBE=d^D)jg5hg6!o9&jSYxkX}go8fNgv8W@#bFv)KVJc>qd+Thbo*rmU=l+s>4b^=_G=iw@*z^zR>eCmA8!<>dDm$JiwQ@%_Zm zo%v7oIlEbZ>ut@~ZEowNB_7c1!Mb+>Hqc!2caW*bh2GC*1Ek_5;NS;x!Ml8MH-9hO z&hi--pAVS17A&zOv;R)+p}$LBmNd>vF2lXrvJ@=2X)SiVNEu;?K7%@H5hN{+az=4y zJ-n)nNcz(bIYN?9k^S57npZ>w3UFt!}G>l%H~4w52>bbJYBf!8t#{0dl;4&Ob-CVwdKaUhvHo(&RWnnP(bv{G8yZBHC1Ge#oNH>rStS{ws_lMwvuUisGMiwd4DDOb-LVrj8Kox z!tmP$kwWex8hZIg;ROzzH+Xc72&HGYW$*%CeV6NF2bmkHQi@qY?##GExA2dG_k|Bz z#aEF2LmI)Ig10>GA7yQ*j&T@(~v3SJpYU`HDJE1 zpW)@YPsD7Zq=VYB0#Bm;qQU7Mw4KM7Y0lST6Hcb;;F%|Il9%!&PLB0Uod$t{eHH{& zlF@>*VH|>xGDJ$`m38tgSP)Lp7KtlBdetOCIran)p50|QDos-#iduW@ zybVS#K*QQ|HrPK8Mj!&oVa?KMe=6Uo{*!t)F-{PCQNF(spDb`8wbACp!j`xm=ll(gflFi!q;}6=BBJ}B&z!`F2ykU4;t5VSVd1|F z((SkR>;r~))5qqjcsafs#bvDJNFQfhzPq@M?Tj5KS%&Q(!ByX=t%j#M>`8VQGuL?F zHEwDi^HR;rEen&{Bj%2BbTq|-}hpv70MYNLS~Q7W}5%JHFK+9J=bF1e0E3#UO!hzht0)c2hXAWRB_V`lH&&Aix&-hRiG z%lbC^%7XNVO1&v;d;7YtOs8R55A2iC?hj1V@EYV+9QDIF&}E&p1AkrQ!%`ZK%cOW# z`$MtN25p5Tp?aCL%wA-!4zQS>!`xIgYrbT1OR0^{f9r3I&f?sc6wTC}ENawEV1!cY z*1Pt(j8QSR(cz%q6b@G-s=e1XR-bRx(b3ni*+x?8yg&3G5uNqY~k^SXkeH_ zFr%I%{bzvH`uYD5e#*?or zhU(@5BZPr@F5?|sC)*x^NYL*PjWup1ba{yNmO2&$whrx6?9kQrt*M9tiOi@6;#FhyPv;yo3K5rvVSBL)ucLl! zk@+7fS1M7jP}PAtuzjZIbGsb~R*!C1O?_QrPkQW;Dm})G=L$8?+tz)KGI+GAeDiur zLxr&YdWCNeP7HHj+v;38)aZ=ayQH<*`7qD)EtJm`kqM)KT zCtQ{8TKlsl0`E8qZJywK?sI&bJ|&mZMn|;|_1%lf<%;!B7A;bPmKlI6=7IxP z_P`sM@@})=IcOc)S148O;g|dw#y<72RePE4-_bPOnUd@j2W^KU^q;23L1>TNu$&O) zN(GBcqqr-~NF<_%fSYLqMW+)$S8VUP2w#q38}IK0KUpkWr&usbul+<#_}*PDi2F%Q zRK!l3XT5FN%3*Ge2BxFeCT*5$5ztpcYc_gwSC&IBObtis9t$rsrRG+2(3T+Y4S|+70R`WdZyV1d&-%o^j{qOfv;So@7B@~Zm(0^r2j=h z4W@5@WrVpUbZM7E&wSvM^;5q)y#F)c)D zt0Y+p!9A1Dx9>?@C*$O;aO9gA{(foP2~7Z)c@RdKJZ9D8PxwOrh#R!~_|yla;U_qf zO@1Q%-iKgx|C2jmv+&KOtvVS+oQ(pXTkUlN1Hc|xlpknV7b@)AED>Bmfw6lAmmJvp zdLN5HBlQFvcSVS^4yP#cP7?xTuyWesr6M5~Iak0(nQ$hOE@Cw$I;iqfbTiA=85IQI zpccUsa_VR=AWX@pf;B?IX8fGV7;-cf@xY;pygUDVQ&kfz^H_(Xd9|~lNsP>qC0epY zoIh(mB$aoAfi)SPD4vIq|CmI}Sp<sS1|5+Y9}Vbh!w+j`UL z*1;vLnRHi#;-MVfzI`n3hmCg6jy-_>1B~m;pFl?pDzXI={A0Da^}$?7zAHXUC2cjmwEa_p7F$EJ!z<*##M{ zvY%`I-l2))t>L1qSdpeLS(hRn+b|(o zM4S0xT%j%pCgaKNQa`P`Gb!rghnKY}&=RrjtWm++Hd!ev7>q(6yW3k}xjE=L;U{o> z+J=#`@5BmFt_yi)0gh8wfTACSSd#AzAwa`$C#o#9)PM|-DWt!6t;Qs#dhMqP$=6wN zlGaT|>}EEOcLe*=ckzwufx&b1GZ$+=nw6gCS1 z*shnWAGWxW&CS~W=3COtn$^Bsd1vtjiaO=N)`_xFN`T;1IR)cP2tTa8x;nK(iAmbW zyy!6wvgBH!O^bKXkp{dDj>$}nbny|hO?zgRnd{0#>R(O z`B&TE$ov$`-F&4KRxAT2nmNh%-z$G9Kk2ur(*>>D^kK zCLgyG>44%7l1w}_W@{mLOPx`}6+zie`oHk~FnxXWQ$QiA+luR<_lYvOsBiDO%tbg) zHa=ZkPiCet8|H$PS-(2Z3eB#rQ(yVG#=n&eeX1k7aq8HrGw?L^P)rwoLL8(a^8bC0 zqvKZ#$g7q!_hg|PYw0dYx63ktNGN=UPQckOVlhZIbYaWd5@0drHlS{Gy!I zCG_yH6led*G2G7wEa9@AG|CK%Nu{r;$~9@U8Sckd|IIBn$NzmR^@GG$4hg?k^$8n+ z2QkAOI}OEyYJEj77F_2?n$Ppf8ZMT$E&2r3kS5A19IVG=GRtdTKD*6-AM}!L&Q6|x z3Xx*ZnmUK;oRh=Jx)EJcYvQb<&qe(cj*5!C6VR<-a$%i|A(P}G6V5NRT`?G-nOTX( zlkO@dFJ5&&deENi$o|Gl+tW;zjxHC#O4z7k#dLL2(5mlpVY|I~^Wu#4i^J+itj(?QJ4XrsT_yR9wB-1FK*=q&Q>3x> z+b`kJpFUtShn&7V!y&BLllU`!heOrRO_^Ry7uf^$n{_u4Plv?QOB0FNb*~jW#tO;M zSwzVV56koP4xoQ95M`VCNFJKhPeI43$ML|${hnB&%J8MUci5rVMT z91Hgsf+XE;5=J{(C3PxAI=WpN4$NabF0Zr4;K-Pm`}en1!lNZQA@{>WmX(K&Sq)(f z9r=r~Fg{tnzn`2znz!;UQk7@NLtBKfR-S*=6KG*!(~Rx;=t}af{`Wec@J9Al-T?mU z*Ej-?7EfbYRcC?!0jvq1b*;DI@?<#A$6ee=iyx)B3|Q<-g?MAgf>|s+uNQQvO1NY4s}0afYuINd6xsY$a;?Pnn5j9*B(+Zjlf@a} zQ}m?04t$OtL4!>YFSAIVm{r6Vqqjb+T8^r`M#Z11`bR??4^B$Arnu-B^k|6K-&o^M zC2qT)EVgn84QJT+u=_CHhjnFQg+7bu-DEtIXd6St9#NB56zngWO}gBCeQ&?`*2=Uj zWIeG%Jf;gy)&-$aiW4hIb@=)p0K^qnmc{B4LnEdx`(l(AW}&;-Gg?%j{$bY9x0=rX z>*q|!*eBLLpr8Vd@<^CILg*$zC+U`LCt%gtR8IyLvXyMs>ni=wZGkDU?ISP)Ea^>; zvKqVc#EA+&PHhS8avq)YN|~dka|)KesaVTFwqd z1D$X_lJeIjWyllDit}8n{}AQSqZn-nEQ5b7pIQz z1CoTh+nxL8MTQ76MAzM^;A)fNJcUQ(mqoFZA4^NQLU-JPgkD*ZGb#OTAU_UQ0bZLAcpoUS8D4XhX$ zAL;>7F^?FSTRRQEuh+h-RO(oYioy1M-bPYGNW5lj+xWSU=_a5(Uql_@5MBGKJ)^5| zujpi|js&+vfhF-rmS|ks#L1M?@FVrV858Zvx3gubn%w4u@xPYT*`*Kq`{;UnqY>$w z0;93T;@&RAV{R0208SXL@B!=$Bc?^TECHG4Qi9~`HIgVhQAVZby(L@D_AP1J*~;!j zH)t5MBgmV|4hh6foOYR07VwtxXI=qmoZqWJ3DlK$q8$E|vRxP}UZ^bPK#T8)dv6+k z!p{sV%t7Mop8FCN27;2$>VB|J-wscH)E7zp2dm50f(Rc({Ml4B>DGU>h$Al~)UU4) z&-C`!vjDnnNG<(q{{~Vlu%k~cao{m-$C2xeUmX6=AA!kHLf`wG?B`18*r6OJX+8fUY|IEM z2KpHgQx#_#N;%9s<95vR74NnN+S>=vlBmOr1>N$1 z`HlP&EwNPG9P#f;qUirrjL`?xY@hGUJQY+<@(si$F<$j}3VIOT70#ZQP9*YB*E9I$ zMrUJ=9!_Q8C6N{JM_1J}>G_Vdub(h@@6{a`WyGZK=7j!o7gG>kS`xl3OY!C9P9^?y z6vFCxa#VkHd$8KnLfExxNEurjl)VKYhTAqr2J{Xc+!>z?>V2m7>1!kbQE4 z5{A1?nahgrcQO3JHuUJDD6nv;ucFY zifE1#!jHhj%PQgB!Lf7arMg>6-{s@ILqD^Fng%ya;5fCS+I9i2@x$f%yJ)SQaSF(A zvAFlcS8;5kC%Se9*nZ!W-C)+pOwqjk+N7)fNButl*tVxWNq}=Z07v3Ih;Bbsxry{H znEnUaH+l_bO#-*RYw}|sh~w}D@_EtN+r=!Y>n`_4^T{1ru z7fU=4dY@byQB^O2r^*P3w1uL(b|pZ7y#fY{JQ+K|R00@In7cctMW9uNIqW&uL&7Aq zy&VHmOwa#61AFMXfr_7NG?dLK$+^KN)b~Vtv8@|&ZAjmmR#=M`UD@YC*d5D2mZ7%B zgj2o}e`PkMNSE;;)&mi5`XN*7Y9Pj4O2%(n=WwBaF-S#{PZw_tdM-Z|E8l4fF=C-s z#7OSvn&tBgddrkwyfCCvvYeKU(T|mOo#y9abQQX2sD7I1VO zm}&nUKNlc_PSpne^k4E=t0D}@J;&)P!08oj{>r+86N#(qx^_JtBnaD}y$5%$Rg%a1 zJ!m|}ceLRb)K4Z&SvKw#W=VD@KiZhf)}`!8>L)p+GnpyiOo0fmDVal~x33zIUBfmA zf_G)Uhjq;CF3skptw*OWKv6}cW!l^)y+z37|U zR=-Jy?Q0EzDyF#p{+lvW@%i=|M5nEMT`A}XF~h$IgXuTB*j~h$!^>se~25gLsj03 z$xj!$plJx1D(IqXY7_$TJRILJ)#6iWgDGeZ^RnDKajr8o`^0swiwDQnpCjq!hP6B3 z&krjhqNX^|tuv1C%+~GY9rf>|F079pK@vD*Cpz?;GbrpohWnXx^%?@KIO*?}#bv^V zBm6Ut63)S$gcL|b=8;@=>`BfWeWPY#2*7(zU{J|j9KrO?2o<*skm_s=fi7HbVjvyY zh}@67oVpd6k`^b)({#tLm$h1e@su@XY zU;Oi#h-WlS7)d2GJJ^k@U+k+4n#-TUClq@-|z7*PtfKdESW2JedD%|B)C4dy*Kq!kGSUgQHKSmfs)>HAqn`g+RN#?$9J zhrVIZWT~qNJLd1CrQa_v1C2vytp(#mY&GgSeZOyHEd+Nh-bCEz`ZFr!nx#yT(71|S zo|(KsE_Ecg63=Zk{wRU~i#q+S znxTWIH-~erZz^E@)z!o0r3yPH^1yDpVs;h3D5EmVD0LzvW{*F0Mb-3oWG#NpD@ve< z_Wvk%jeaFpLK`5o1%Ctc+$e#3LwT|Sq=@CzUs3@2Cv_qkRd*=C;h#pf8NHsFm|z``{}QNQUZGO z6e~5jnqF@Gb^iyzK|*`#!1R3Q z0^N3kDk@(sE=JMlRuKVrI32dccgfasSf5jH8z~dH_aHh)fU)_G!OIgnT{+!Two+*1 zxgRYla1}0tcn;fyV%h#$wnZpea{$Io^#5_Nd$1p&3Gp#Ij5Yq+hl@@ylu~|&XVLt! zs6p>Xhn@Ak^B9cEl0Iv zVkz|=k|z`XEGyl5;_8HpZw~7OOD-)Q>t!B4i#+Yad6Cf%0=^u`8Y)Cs@K;s7d#laE zZcMCFsl8n3z$OXB#ithbtUzC%^QO(#zv8_4=z|Q`DRKc-JMmp%$>BrRN^OHM+BAiy zNHI8gw4ReAlvFObWZKMpoc#FMs@puK_^e^SW9 zM(|eejYy?oNNb&>RTXe!$dvMOf5U72YisIb2Gj`(GpQQ=h34toK`D^j`wm%MX@`%M z-NDlx^`~|A?%(w64o65fGjeOs@*s;7;`xSN=lRHaPvVcRppAN zlebaK^1Lm);7d)K97<3>9D_g|pZ4&_)hiQ0Uo)fxdVd#~cAWVFnf_m#T}}4}C5SldED5PVJSEK@uq}~Io3>O(@5f<8{RgP@sFezBtwz$I zDHB-8k@KuBmJN7RP7Rvu6G|B`jlktEWqis&^E*szmz~W`+m6$Y^iL|%l9f0pd@&J> zo4_t0sXks69TbBh+$@4B9EM*&yR_t{!2f;yQ(K z%L#uzZ#~Aq8A+2RXY$@x0gWu=SZ|Bq@ufoWgPO;$S@Cx9Q(~S!9rpO2eIokhrZtf_ z`uS!W{Yp?nfo3IX3{kpB_D1XO5$e$|afHO*qO@*C7{5DR6>5Hr6&wehJ?{~}3vGKV zwGk<0({^_`({|4E9{{2K;B)S?CDg<7gL)H^vr%5+nXhgy%sa4CMX>cyT7Rq{QF6qC z^n#tB#RoZ~+sA7P^bIT@Ecf5|{}auf#k{r7J`A#A^n_F9PiVaXpb_4mBQmQO0qky* zfAG|mJ!nV)gJ`4)nSvU-=^yVeO1H(f0E?@_qLDe$MF1N}Q%PAg0+9LFzlKX;P-!!O z_B9^m$$=Ug8V-2>d-DHd$H~~;^9%Ho{y!T9%}4p|PKK=}iU;tn_#6ucDgOau=8KMF z@d5v)Fq2+=uMY{HkY{bt^+T0IxJG-?_T5eLiI6w|I>a735T zAwW}^G>C57pjXm2?ZV{|7dwVj4wUb&;3-qu^MScE(sU*H6DW|lYjAy{WKFtIM8zS$ z?WBXC@yoW3P2F_!lsJf3aYkY9|E7@aE2 znC9LNI(Z{2NTlOvMQ@OT%Z=?a=N%RI_l1JsZLxx@ZJ*lhdOyLaT}WGP4GjEXXkj1S zA(@zSlZp>dyj48X6XMUROv2*l+0(AbD3APlJLrt*GQNHfmU8phr`KMeRcuQb`$w1% zpH*gdD4H9wD|FVkqdjOntUXiXIZD;)_}d-MH0QXLP!XTBFMU&jH67(|pUwAtnB^zPigZ+GHLNI z^nG+-6Wx1JR{G#;IPPO|>&vzj%wWn<=VejNu=69NzKq z&Kro+dq05lP;%$HSq{*bUIg@<&#n6J>~Olg-_BENf3{9gn4x>lG108xaBbjY_w#qZ zMrAEXM0mGZzXs|B$*>j8*;~qb$!YBd%Q~pGSgieE@VUUSj}42pUHAxQd!_Wj^`xdT z%eKlcyVY5zSU0kFcq(a$elB-qKe-WahycIDa|=HPW3#!cC+lUi*p>*d#NQl?x6*>| z>AQ78Q<-k6cf+J;@Q0kv?cl17$gy_v4tZ<8%#w{gU8)JLU-5N8Zy$ ztp~A|OQ4eWYvIUD)UUCke8-}B?9y0vlGvMP<>`Su6cN)-AavS`xSQI%?fF)IR&Wjt zgnE7-7MhE|eGuN-j*24u5IpdNaTWy$i$a-gj#1jxqN&7^**CMZV(A(UhJu~}iD7lw>pj9UQZ^;%(>rq7a2ma79-oA)jToyDx8OPWQAQI&w8~SLIteq0@aurQn)|b z+&ek`Y`$pRPDsbG`m{%`!J=mo}6IP0pfGq|5AWZ`#h-!-YJo@@YqMYexr35+RK zg&MR~5gzxg!xzHZ*)}q;9_w`KH5Nnv9)+{Sk9)tp(k`xu_52^Z!uQn!Kk|h*%SIvv zbx1GMG*7B@oEP*jT%Inh#pgsVCkni?h9Wh1!h4bLKiEns2|zkf&OE)boj9qkj`r>N z79%yMh*O|#{D>^jvL)L6I&UbTd`-sA+ngombF+x7$eltDLU62Y{Pk!9ILy)HX8 z_lc{wy+(9sASmXVOj?cd7h( z+~4_{66E{$g~K4=YMV{6FV^T8#JbyqS}@1jisF>XzKd0?xu9co5{!Nb6s>Q~k!^oY z108bz)ifetK2r@j_7vSFbfP9)BYFd3IKkYeqzN}x07w)+{r+(Dz>;a1F!;VBRy@&g z`x)*^om4=ta2e#o13KRJaat+m@5}8_t7CP*)pdUvt9!^Kmw8hR3tuQC<}x@qCLoF! zgNoZFxGqmyc?J7#O#0Jv%w^l8Sk4!uRX^ur+;+ucFEi$81FJ1+Mcs7Vvyh749X?5v zRaCJqsW04Cq4t)8@5S^DAJhB#yDvrqr2(XZg?fXVW>no{oX@F_!0cP;t;N^FHgy?N>@j%KsbEio}! z5%X5Ct6y&zpbucJt46neUxWoTd>bHoA| z6nr}S?y_I(zC>+agX{8F#DCH#o%h64i@zxXQ6yXbOu1}xAFC#3in$w8OI$RP%SD3a zRUqQYwP;#fJAWOPWg-<v%&0l8e287=`6tU+*n$P8V%qkri1n96i_bx7L8Zto zjr0AAhvykPoD6}qzl!)U=xP6TEd36=Ktx^YwHU=7y!#vF=lCGkl*V%s`Sa9ah7>8i-`?fVK;S)zss9-&jR$FZqf#8g~=3L4)Xfshr~Q7_T^ zKvcRz^qlvtpxcFezhvT`i4|g`2Sxogk)4S`&xTULRWSCK^flZ9UF33nTK%=T1zL^U zyKjUm+Zr|w%cgw^mOI~4CDQMLZd&ZA^RdO5Wo$S<@7d;Q^`fg#Uv%x0qcU63|J|6k z{cYc$I>z8j&@K_~7yLwvOKUjL3!G*6$aels!*GP;n7ma9;LyeX_BTL6$XYz&)U>v` zw#9BIiptXHXHMa&bU$WHhMjZ!KZn{naY}R}tspP{)O<|hXmLg&n`;x33s)}Ft>6(! z38!VzqIaC+Z;a0evr1G%9A0tHMKmPOZo}$ve!IaOSNyXqTvL)5{<#WP?MO4b1VJ&P zzGxa6^5eoS5)z~(&3<{DgY0SX3?HK!INfRmLp#Yr1^Y!d{9D^56H7;W8KV*}`f0uQ zRvSMk?R+L?kH2lzY`;Na%R2n9vSM^$1Z@84pX6ohp-AedEte5f^J8BY;(KwUDEPB) zICI;hkX|gs5zA+8sDIlX;Kw$_J8vwA>9nkUaDe>scJWO&QHgRy&q6p>=1ToZgw^H< z*1AfKQRsQYp@d78@?MP;`kzjr!Jd&87S?c?Iy%vB6zUE+SBv%*Xd-&M$`e-^!og_| z8q}bd{2YWsRc7&N&~ZqP{=smn+m-2-!|Aowr%N0>zXbLQ$(7cif4&NiSHkW`XP9y~ zv2ND8s?6;#BNT_oKKX`7g;Y(|nENWX!TQo_YUTp}T)fG+OWQYPT}nd9V7XlFw(zT; z$H2?x7i}}&wkAO<`*uPic}s-Eg~~S$`f40__9|}M`@FadRZZ2 zO3u;$GiwK<*WqsqN-B8e0AH)(gKfp^A`j!;Qn#9jZt;JBg?ato)tiq8TuVk21t#?f zXc#?X(~LiZ%3q}8NU~C(q+uK}YtCSF@SlP9>k+A&qQ`s`FZ#xQw$&}#RbtCS`9O=; zYKC2}!|wco>2>7zFQvOLb9v|?H~swK#u{lM6)7%jT2oUS-a6#eZF}4Tp(AojV(dUZArY{7GKsoe6X)^1c}QInrJ(cf%-YXU59!J zBy~;;kze=lXp9OlaC6(v`95C&zfI5j&)k0J8Uum1qdbR)h_ zE>Np=DZSsN~2?&}XA=D^pB31Tr_YYeZtPGe?z$a83_6dE#O4}q1Q$Epaz&6<(A zZWFpizhwDPc{8VXItC$hiY78=Q8)0VH9TJlq{D_E;{!`)qa_b@$O4r{pyKrvyh5C4 zo#X3d*@;D63ObifnXh%_+HxH}5 zQ#GPXD9rV(Dm$R3J?9B=yoQuF54_{7_zNE_n_^ejkW565J%Ax$`e9uZw{sM|iuCU; z$r?rJd0}ev@WV4bB*})6fx~V|UWfQVXBC6f5F;5<4=p4KJ%;*=p*~GlcR!|I=?YMt ziZ{KezWx|QxhXcb$mBX0z=J1)vou+mdnPb!BXGJ81?tZ>xO^{E@+2;WiZC#=)84?3 zRi1D?H+hKA^l0oNWL1y;NN-&U)Lx{%K9Lq=8}T!CDU6+QbmmeJG%h^KZw!7Qf_5~& zM?$-6*ND*|XfBZ2+@CNKnT;0S9l>(s*!`F8kHf`~i!BTT+6oi2#`;ML9}QZ=Z}%s! zJx}f`qwbTR_%N{}V{L}lg9SMx(*Q_}>7VsX7qAx6rg*zX?tayQufu9b=*yV93yYhy z1k4%j(bL+IQ7vp^2qL?x&@)XC+UQi7-lE2ODy?XKS8*(+Lth2&DOQ_Z)Y`GY*+t*e z@yu#GnX_W?@jARHgQERK$7IfA-EUi1ne*tV?l^v_rgQczk?3X9%sN;#v z*9@jIjV>VLSlE?u+regWVJ4vR3y=y^zCi<#fdm46c6#)j2+1R1O;Q7cl0-g^m)nVq zFh_KL^wMURq(ocxw>#*|Kg16w#x3eS>Lk!7Q!%q8Kvtf-OO89AfFEziKY7VNRaf_vHvMt34$qwrn zk&;JvBqQuVX?ktGACAVuf6V9bww{i37`ih~(LSo-6Tun9qvV0ru=q}WNLc6VHDr## z@79x8y;t=s^8peq@?RHMjwTLNU9b#=fwm0ev zZM`aJ)hqmF2x6`%Kqgqm6o&DFO5M?r6=g>aQ<%sVADb1fmLO+ijC&H`t)HgO-%cLsi znY?a0ftGF%?+AWW3ES1%CNqH#mNPN8pOAI=cI1^v=;pgD(%LERSu5CRkj%0;h{>NO zY_gfL4_W~aLh0GBav~`TL8cca;vQvh;s(Z?;g=u$Q|=O>jqKeMDQXbR@vD42TcLaH ziMEMy%gTBQV@|V{R7Li#g}|KHK7Ged7cC?#7-hE(r9{1--kL-?zJLvrr0glARM(F(bE|CB=JS#cI?Dff(EJZ@e9;9sl4UqLS z>8Y@wjiR_QGD%vZ4yMi!CY{*w+O(?o?uOq3uL?q+aQ#QJYW&jbjy*TqI1E!&>Vlg; zGS<#8#?zKn1@Ej7-tWq7J?Y)hn4BM={hAz4AvRKDsON3u;wl?iR44(2{sFzn)ZP@g zxdWn+EyftQ&;o=0u-g?&{WtOdlIsM#b^`-=I%6R`JgwpW2WKM8tTtuHAyhn~v2-xF#!I5LLYy05 z0aHGdkZD_dCw(*|P_N;@W#Zp#!4J1w|m8IguNIv;=^Uix#OYtD0QtXdN3JTd?n0Ct0YS;|M zb}6k^qY>vJBS8ke$XWJ|NlfO778#AW)O}#RSfa6e14kdwkRCVv=zAqceM$X)bGptU z)?0O?lgtBjSo#45vwAV$8hJneiq4T={Dz&j&S&$@upJHCISez&t%%&Hlj*XX?wrsF9#{M5iykZH z@d!g984aV+%3rDLZasK!h9!4z)cN=B{CeE#JW15HpYN6#OKyrBOqNKrn$zHns($-x zrse&cL+*XbO^-La8xfr(4DO}p3p&!0o)z30Hs9LeZUFeVh%8PJa0D59x%xJ8=9GT*({&xXPjISh92XeIam9Yp5B1;EOPX1 zz^{7%Vy}bh+v!^*cqbe28@_T!^?$@j7N(OvnyUF`u-|I{_ITBTGbrJL2Jsyjd4w+d z$NuT^aWC$*S`ME{sY64X_q4k8Gt)CS!2&`CiD>Z`oNwRA(%7a~e~1@7+M+Ahu-Afe9P3 z?WWd%j^WUlbdu+L7luRcC^>I$(QUXzfJaQ1z@{;JMHTc5iT_|c@S&e2X=z|&r2Cs2 zY1Ycc=nKq-M~=o!byyhfc z9!-YvN7_B%39QTWmgR=Kz#2!WjN3H-102E)5ELhjZkr-zGvY*!?@~2c+=1VvG2s$p z-kX`~(FGq0@Tp~+na`{J!;x}b9*f3d*gGeU>G^jt8_6azeZWK?r$G1Z1b*c|zvweK zdi}|47n391AK2hq^p+JliC;K^Kq5F(d46s=U3RvN+V+AT8yNjpl<=s&=>&tQI<~;N zh<6Loe;}CX)@p+;$6pEoLF@4-6ZJehL!_NfU9{6j*B*R;62oiIzau^UbUJpax}vog zbQrWSxq}`rN1kHF40rzTFP?_J-m=)eYPj>7$8p**NtB{R$(N#(5mBqy=U3?tn^FhK z_~=@`b$0E6bW%EJo!oR{06A1QBE1iUneV9IUx3L-W6-v3I5UOCo2BCO5%|tP`89*@+K+ zvy!9}sL9wcN&PU;`*#-Pk4}=8yLbd{d{g?Hq&~NG1-@>n2e(G zofMQ;%5_oPiZsSMDqY2#7^BBcJmJ)v*n%Uf~#nq%oJ@VHk9iL6FFhM3H#|i+EBD)0x9Wk%2qk%9i;Y0B3ff*pfQo*=q2Ai|q8e#y`={tzR8^Sdc|6^H>6W>nQ==cn`r{ zz*kP<%mrmlvK1q*R|F?Ilj&P7$CQz%J>IA`kJQIE8wJ?YIjoy`^x&0}x+nT@-w2__ zvg@|+o!P5buiWd)EpTVglNw|};^FxJ#NC4bM_tXLJrYg$KLAxhs=n5Gd|D)_F}n<& zOR}#!KrPAnat1ggS4XGmT2+d)I)Q&R^U1(YK)~ZXeqNRBIyRl9*g+Z7;Yel!Zec5p zt?S=7#%tx}HQ4>pVO=F?$=LcUNbruIs>dd$deMEMQZqDhHt`YK0aMO$F@i`3J*$(} z^eaTvZfq^mPN|c(nAq(gMkXo`A0zUkrUH_GEb)#7M>=mqfnn=MSxZjE)$mfv7V~ON6yNLOS+`(@8&7bD z9E=QfTw^tQ`s=}Zeab<0_XH}OmqiE$c-UeK>IL{0K132Q7)g0=dBWv?H`)K?NZ7Rt%l(!cGQEHEJR!rjzTOLpt>9q@P z$2@eeq;&mmbkTm(dpNirTJL*+Az_lIDVb~qTfH=coSDM{w(g^PE=ekye zFUk9|gl`zhEzdv)2OgEGZ)1Ee8B>ol?0ofk1?}FWJeq~{F}y?rBcg?6KE!Rn#s+ci z&1S>q>$duxt@N#FYik+Ao1@$ta;ezZIShL*7_QduL%O^Y$8Lf+FC0TE%9$G!ZpKw| zr_&`z>0G|Kpjcep*}(*8o?3ynFiIz&2PB+#KDD<^4N`60>|gMQhv1JwfX{HucDCh= zZWvN>bCb^9y)XrOEw+iKOEO(rNRY=7BWgxTZaSC8C!8q$UHI(>OKV%W;JKGjJF~&yp4jWfMA@WcW0%xdP`J`%(Ba?qIp$o=F%K>Q zsS-`yTtMqk50GpLktfv^aI#+`c-JGJVC7=F61@Y#u`7mqv)d<{Bu^K7UkTI z{{URGvYX1YK6B@33Oxrr@lY*~i+oE1Np3uwWF*K(3NhHX^cB}>x^|DI%woNeTuy~h z$8Na@4_(|~V!a}LG96CB2rhRIvms?z_*Ff{lDIi2h(`pDpGvu^_^RIPO}Lv;xrph|sl+eDfzS>@x4F8G8wkjW z)PWFOsVmcH7(EZ+Uo-ftRPip0YbW+Lm1(S5NT4d*G=<|~(6I~)WB&lHT0%V9T$_C)xR`%-*m{@?M$vT5EPvywZT$t{0$ ztnvNGVaFijss0uB#eYP)4~KNm3FvTmdqRO?w6#EpiaG(%SJ`EG9WQdc^YUH?efYV?B0#`k85;q@I)5cQ`lL-XQM`t#|GHrQiBT``wU}_nCVB1JY%7JKJcyA zjC^4?lVfN(z#E;&`D0J^n;donlfbUKQz7GuWn~$fd8FY{7peY$cQ^vS zoe$ds_LS7WYHuE0_!~#LZ6CoJ7JsxS55hy782I4LHv#dSaTw^?`pQwIAA(#+P3c{Bl`*38GvGENr6rMx%8f`nY5Dje3^;ES4iBRDQF5!auTJso{I3 z{iD7$TZZvws}NfYMjs}bXVDQCqPE-&e+c8LHQ~RoSL~MG5w$HhJg&q%W7@ zeHqu~2)O?MNsu0;h|YVTU@P|qIMse!Ry@vL^AuvDtW+?9yjPZdR``vld@}IcYEywV z?_ecS>jy}!G0gO@g}<R%YJ}W{Yv&97I=+yJyuB~(@dURaH$KnCF*(O zIO=6x;FjXt+I_ZjE(uiw<&Ahy0wS^ePHW58;(MJ|>CokMpNBpl zj?YW*c8>!~sOw9(!TippL{!}_*=(b8@bgK3uP@A9TnMd zybsF00r(rK_}|B#DcAlnYPR!zn(2I-7_s}f?cvE)Q`twL#!hSWo8UK!yfg7ZZ1isl zx--Q(BN-U_m_iqu@ije&5Z*EMGU07}y$y|&kH zqCq9Ql#S0&AFtm)+{b$xzJ-0F&kM&-Xpd#>0IZChsNs*Uo*m1R@0OB z8sm^YyGqs$i>>@yTiCUK66m32i{yFWka;=ds<+LLrFk`Ru3qJu`5kbfWqk}CD)+_r z+V-7o;uf84COc6B`3gIY+u-rY5di==!v_1;>f8!sAwJ!*yYi{#K@r^v1Lr%|7u1c8in-O{^%ia!h^@t%h)S}nMM+Wd&Iub1+! zI}R)K+CL0y5xCPdDC1C^6bsySA6n?NuLfz-`HQOAjLHCD>yDM`!xW{fINci`&1Ke; zd81DHHmZdtgy4EG9mlm<(l6)m=AjpeEn$vlw{MV2HzwjZ$R3@m_RHg^?17^AzeBm! zd_QPpyHzG77#M7DGsXsM;ID>%vlokeHRHQR__=2kT68QQYRkXOeu^-?f#;g+=Z&w> z!Ny70`4Z~)Ue`6L^^I2gONnOr}w~N@d<|0!DMOdBTBSG4Z>_8lQ+hBi!qnbfKWrq2CKf zjvVmJ2<|wpU&22VykBK|4dkC(C;uc6h$u#9sh3Ho4KH zZCg$onIUo)d6TL)Y#ja-=6?hJCTZI4J|Ek_Y;=uTq`T0PGKpJK=YQY*ss+5ze7&acqBr zADiy|Sn;H9`LXbbd;9PeNXsAE0YD!;jia!2ba8I@6<*%K{sVb{Uf1QO^(8 z=IX3451{Bg-~Jfm;9va#rU>q9?~nLiKL&s3I{yIp48P%Ab5$Vi2cdCsYg7Ny{sHKsiU260iU2B9=}}av z(tsaKew6xS^q>f+qKW`0qKW`=ed?N_PrXx61Z6R${p; z!}R?t$C31qXIS#^41SfXr0TLpz$wjfaa~0r+Zd%wnJr|^VI*vwk6F0Y3Btx|?VhC= z0I}!Vyt3a?Yl(huO6V)9OURnofx#8YY8s+TZW=Szu&y!Ec6wFeW^nt2?q^aoLan*PTP%AiI=vq-9@TAcOQH1@GD;S{K~(){t?r) zc`TZU7Y1PCj@4xNh1`!f8%GsAoNE{l)B&i5DBb?syex z66SMyc9S*q=!lz_xyNev2>cVK=zbl45o&0w1OOK=@6didE89LCd>*$EE~S3E1AX1! zckA>O@IUR%`#@OOJiZ0+o!ZZ3ZOokqohtE>kKt3mtEWlIT(LN-;uIsy&rJAz;rJ)< zeEuoBfTKk0BJQX#Fa>>^;Z0iO?J=x!e5a0U`TqDF`($|A;f3O9o-K&$dLv^o6SjDN z>zjQ)g}C&^e#`tY_^aR_iJClqCGZXU+`tqUjQq1iM`uo;kHmT!_)aCtF?4D<*Nx)# zYj5};)sF0^FrivJl--|W>5XgRBvu;H@a>N4o9(#rp-{)4P&$4!-TXSzn%?l-YLH1a z$toI9H!NsR0D1vjZ-Txg!)17o>VtHBH+xstUK6_Tz1@tOrG?B=iB(6Ghhwft1Y@Va zraw=E#YxhSDuMJcsEc=YI(5~7WVC0tVn88yJ(Ld_{d_d#BjkH`(US)RB;E z@0>6{U(2O^-1>x*X;$#8B)n`SjE~{ZRX<;s)1`A0+DSBp79FJXfI3%`M;9tGyba>- zx$`fFejB%k?L;x#LlkGEYI4BEXpdrD5y@*JZExmrJ#Uy5I_ zcgNp^tuKhQ-8=hpPl=eBH1D28LEYi|z~ipX!nm)7zZrF}30(Y-w6ggwB+dXIs3X(b zzpZ^IUhqb!hT7qp89~82a2avf`*Fp79)H1Ee`d>H1i@|lM*KLlh*`Xg2DuDNg2r$4 zS1XP)pOO7?r`EpPCE}NltsE9U2{~-``8DU$@;q!t8|vxRrLyRLihjs`I=|KYA8&Vc zcjnv!j=Yag%vas_o-)?#P(F`$Z)%b#6C|sg?C4kD>5t04oKN^B&+RRyYZ{w)15_%CmaFz*XXCh zpM;(n_*vnod^O>?0wN5{A?4+B)tL7tzVn22g_%?MH&gw!+d{{ zv*B+B#iLvO^Ko-2-*IsN0Dfo2H#R-E0B{EbitVnSO3^gijayfqJ3A{lntO|h&e-CO z23gqlP%C~5TYESfdzVOKjZ3@8LMkGRmSS)K1CmbwSK!y}tNTs(%l4e{x&4~H0mE}H zgB8--T)nV9ZmV(+$O4_ex&(gd4l|E2@StP&h(<7qj8&4^ADhl@PVZKEm+gW3QFy=h z!SU>00{#s{DtJ!H>6SG^woOKOv-{WQ0d|5>TMHTj?~YA*ufo5D)BIW2Vf~-}2*moG zo!RnrJzmU{9;V|3t?l1n#e{4(Ng>*97;Y!a2#0&Zegf6JN2)LE#rrtKbz6;2L1*Gk za?GrYbQpf}!qQRy04~Z@o12gMcx5UNlD{|g5&Iy1$Q}p%mAnu%-7+MbS-DNO7bE5f zokUUO3?lmA7CfH7X%*DPzJmVA-?B%6zh(ab2rOEBG2Cm{aV&RtV13jah@;OUH7DxF zlV4f6Vap1!D&Ty>0=^9WyFYF}hd;4b!wI}au1c4B^}-d?wD5jsbJ5s#Mg4jTw(Jn5 z=G0<^{kK1D4FmR3__cO{~X#K4y|f4;&o)jH<3O6BxkY55m7iKW5!e#~-wZim!e;{7=_hP2&cJ z=TW$y3?Z_OoJ9;Ma4-~-anO2kU!^}6{vZ4iu+sD^Ydd3ix){m0(^XdGvapyNa6=GR z1@a>@9Jiv3e4alyn{%fKu3f)F_x=*#3?&HRYuB9ptgSzFN0lwocTY=OMQ8XQ7I;N` zC2QbFbS)OvLwJ)t)E5$kUm-T&q9|aT@DB{fjDbykqz5};*($87P&)7xGj58KHay|(sK zl#VqG<*6iLn{Ob4&?)cJwG7&IwwjUaHbUw_ltQcKY>)W1w<8z>J_d1X1n9BP z1;gO9%7s8yAUQ3@KJL+sfCW-l(B`dQX+|e4q~2cJ+&nLBBnB7zG?Cr_cLwNl-=3qM z!~xWG9v;>WhPxfDih2HiVxG{SEeaqfg1BVHa0tg*&9&BajdAZ1+Tz{=U{uT`ia^tC zINjy8Kf(FpJk^-twg5IYmAv7-e!@1**yL2!;pjD?wnVGO+SeBEpAyX(g|(tdPBQpigEyE z%8&w_Z5al&b-TS&QFv|Sn%+sBj8Pav{JTi~=iM3H6ds>+4{=_-qv+aQ&x9e8-7X_& z<74w}CC{%wS95Z6xOM?WZ5VSNWlHM`lwPf#5#rhIbxB)Bu}6d^T1X=~@BQPBpmWC+ z=eoXz*pdg2?9)4L<#Y1!!Rzl{<)+{0_m1<6*rsa(HMpX`#fy}+ujiiZouR3ei;5#`THHw zT~2aMDhrP)WIs5_5^W!N{sa7umDgHX$dcR7JmYjka(+-xL6MAh$I_X1aSfe}Ut^V7 z$yGUE3jH&TdVMN%j$6h507__?Q)?g@+BWy+2D6jU-kQGWLvglyQLvLD0rK?8Khm?V zrMzviqpEFCdw(y}G|ltGfg(^hE)V$o@k7z>|j>2#k7>iXF7CbgGJf<9gZf8H}4qvq?8f!GXIqG$=G zX;7>a3rnIxtbs!?eDL5NGBK0e9jnsLp&Tlt{{W=Spn>IpGD4i~`MBedKMIE~$1Pfo zEe?7=3u!t%lxbsawRm;LNdXAYrr>?>c zBM;mXV~rR1n<|6;`Rm3j+I&T&%dI%jG}XF_@^DNsBN&5|)NUAUy-stt+P=g1SMc9T z_<5teg%Pz2pWan^@;s^HCG94?#rXLjn)rdozu6dXt6^XJM7I9w@BT)& zf;Y+ZH#+2OD+^5v$11HH!e>>jQJq+ zA34F>UU~Zqe#mX{?^Cw9)nSq^hk6A2<#X~KUhz+UaiR<=qKp6sU-du1)>5VI}HaM{8+r%HB+e=9*o(+PDWB zSDtV?cNKfZzCS)Rx=3u$L!{d`?jkGm%;TeAe-EJkwdLMEv()sfJ4<`XH_JRGRDm*D zKQQzo92Vg5+aA^BNy>8Ok=3MCk?UU$ygPB?qb{v@aR!{C2qR&&UTJfTX9EBd+4bkO zbUz97DLyy&%I41Ubd?fg3qx?73bl-{p01#$BZ5I%ruty{8@WMibC=Vxs zTw|?cR-&gC+b*0Oo`>rH0Ej#np!jFu@BAUZ4>b39p+$%?CPIKQ)DE4^Zhpu=2X#M& z7Q!1%K31MfhPk`Ec)X--!-1RvLXH%3Uv+#bx{Fej` zx^9yVj+hI`tG6ew73UWECa0puajNREr=R8>v3>8|jtAq8O<{?1<>f|{(z)%v0M}Dr z(c!s_D?(Etha$Q^?8Lpdmex4c^2A{D#d)5A93Nr(SBB*_aJgl8w(zbvI32&0dk=%| ztfR42o?Xr|31Tx(4KHbEOM4Sla_Wzjr1%r!w$v{T&bXozCSZwRU)`bp65Uw)@mvPC z@W;g76V%sR(zRi6b8fBshl~i>NaTFHgXv#&c!h1Hwz!@tHZWh6hidp+;vd3qiC1&~ z0Bd+dTe^~K#yAq=?6Laf1;`%t=Hsx@q`6@h-;+Bq^&GX6vOQDahLIku@9`(Zae;j> zFx!li<;0&U$NR_IrF+!(5ZcZz&;?HCJXetDAKDhWL>9Nx3z_YrNavM$7YYGCkJ7L; zjdt=03~1jW{vOrcMscG>B#bIS&VEPIlZZ77Gb9hSit*`QV}BIl4<>h9ll-`^G0;9C zUD~@?$k`aeucbq%>-x?7m@$Hl+RO14O=#yA-Dm7Kc#p_#b1Xy zM7oHYTXZUUBntX(Q`Df+tQH+Gf(RJJc>b`r`n-&920e$Rc0&_r@|e13iQo7U?z}c2 zwwHjPA?^p(zFS|}kHGrAt1Zr?yDu*Uk;^~v6@TR2ffe=+scU_pLi5>5Ru(@pu*me| z(z(q;O1YlQUqDnp4e6h1+Lj}soT(%6&*LZT_u$WgS{AcB*HTRLrWB09X&&A4e9m6kMuQ>(Y0EIHhGajCqQVl|{(L4P2Hy9M*h* zA@vo}VOZxpOEfTedr+F>?L@Yfn4W+IHS$ONC7u=k0H3)30Pm{lKWd+e7TPz2+rS!Z zn7`H&Y?wImtBi!0Igq`{{ZlcNB;mnZ~p*)Ybi-zV>je~|I_{f=%SPi@jwPW zD8clpRd&;;0wrE~6)rK-kemtu&q@GVF+~&s6O0}yl=tgQpaP00pa&=3sj3wF)inSW z8-NEDB{BL?1XNK)02uVCm+sROtxG@!7!*-J5isY0OsR_j#Y6^ppai0vS8X+9>l)D@483rV!g9J*)3Og8u*s;M0~X>!jz5 zh4e z5dai$E2w+7z$EeMURiE#1=i!h73dc7nFdOq#l+Q2^h-In$0av|q*E*ua zbKba#^`P?XC#6=oybT_6iq3 z7jdLRA;>?UO8XPQ-wQ78<&W)fFO;NmYv%8Px4LikgB#Za7}^bekK@nU1Hs=1ejQJ% z-m=SoBI3?0kD0o2>?t@HPO;|y0Jd-K0ipaE@EZ6xLAqG4r?Kv@g6Vi!B5XJNzfV!#yrvl2a4d0O zRDR9B68t6b*T&YGhl})hb$uzNR=Y+YzhqE;L~e?#*x!MjnFAHwI7UmBO6cH&X{L|U z4-foH@tuZ|r%B@h8r|JOB_ zy(ZVe+MbQ4X)g_|HyU-0p7FK4s8P5*%SD1@+rVCSlk+z0h4b$Q-}p;anB3iJK31za zb+htQ7R-BP_+ys&cWuYME8(+_4x?T&mK7xb0GFrBxAn2~_zeDVv}GeGJugbkI;tZY!)CHG|ZEh`GfrbcW zjOWplJXfkk@$*4d5eY~`+^Z=4K(EO$oGXl_CY1?&&fnyHZ5-1LCHzJAqx4q8SByjQ zeB}D|tRE8H-Wx-xTZ4uB+;E@|^V+^j@W1T|;K)deP`pipwrjoAKWPsEO0D5rS?(Uw zQnPq2WVDJ$$C#xY34)dVV~d!kG=r>c<|P-;4R)L zLpr=KW{i$k?z%{WJ3dp_C2{#z?2d!*BgB6ZZsOFwAUoJtb^9tG-#GMQI8c8-rF|>l zzlUBM@U&#;R*MTi-+r#&Ry>;aO)pBuIIq^QoGnGVb+ON$q*c2wyZT;7!(y}Qp7lH` zDPZjvckwA)LCJe=fS-6yWjC$=q!_DxMQj7Lm{hwF+PvTyx#>-#UZASk9?5{B~E}vy85Wi^KmAkr5AyQ$F z=G;f`_v~H!9C#!4QScNRUY!h5Yf{Dq%;7-TFfkhApa22K9C}d{-lR%a(HB2s{{Yzc zz~8dBf?}}K;f`HiSi;Y6ql8{RcOL@^Fh02K00ZoW0Gx_hTXz-ZpBX+a_&?xJ!!2Xt zr;aCEeLqOIX`3eoMjv)H^=3E#R+}R@rzVtlJP-EW{j`1?e#T!5W$}-TrAhT$lx5R& z@J9Kh9mwhtd!N8p@wegI--{pcQ{URK>YhH-F=CR7i`RddBZOqEDHtPy7jGYQegMz= zKzwWXq5F7zbieV&xQ#udz9P0p`7P}M;xq*IUrxPi?(f*6;NOA140x|w)M1X|(n)Mm zLO11OFcKJUAmeu6TxX(?1$Y^bX&PzMkHygakAwIMwP?o`Qa`(27oTMvHCMMay$?@K z5&Q+G%cp51M7m}ID`AkiDH6h+6;%p`7ytkUdM#vlm*Rb{wP_H7;_K~C2#QsS%vR{J zfD|ZK%N86t!Vcj{uQc%SS|9Y zJqJ!dO6Bd$S8<1yZL;qlDI8$+_Wo7xJ`B?3ju?{Mv5;)*7&|}*AOnt{&lNT~ zD%HGSDZ6oCY&@%vw5(Cv5w_#Z{s4Yp3ZpRUPSdo0nU};@{ur~Ic#kzxa>&jQfSa!*v)7lu#I7w zVaoh;t=Y{p$!xqo3sw%OS?cnMy7{gpe`ETvg2%R<_nem$3bz8qy+XfhAD5 z&qEnq?Ss1`JurChz*wf0Z>P;>#@|Dg8FSfX#6{& z&*n{N?w(|*Bi#uDhQjlXK^vHia4n$N8(=BY)-YBGD^57R$CkF$ban$=)QO28Uwt)`vK2e?BdUU|69v##? zHRCN2yg{X2dCHDfJ&Cz6F_g$1zuL#AJ-KQ0d2jqk(AwS?5p9sD01n5muWrJr#L7C! z-=MSon&#)ox|TmV6bvZ)yTy5?yWshn-L5UJ7tB`54y1$Ar%$DQ3_K;I>fTK8LaQkG zf!l%yet-djxnxyVJ6IFK`{y~X>s2;Z;Tv1KOKW)p1@l{AIOD%>PSnjBFAiL%*))5J zFD?gBZOmX`9=~^?9CL~emsP0T=dHBZV_|POZ09)~fIgU{mcl!UKEWE9T=tA)`g2+u zj<2d&>Q@?OgQQ2Oyg%yNNit3bPslOQ4xRf~mBKu0;>lXZ>s5~0)xkbptVT9J-TR|) z9jLTXQTB~o?jK6~Jd#++G;lm$dOT&u262E1&3Su#LepT=Z#A1a6}1gD-dDFQ9$66v zNgQon`5$|?73^AWg{1g)*}l^h^UWFBHxffI01P)LIL8LQJNSXB{{UtDNYrhvBel7b zB4hIrJijp;3^Jo>^zDyIaMg*)Ezd^q-mh`-8(BUk5}i8oSwg`SZMHbL83?2tXTCxF ztJ1FiB*UjzwB91Kk{`3m(7@8g^UdV-ZV#vV^yB^#=@y#2HsecXwvJzv&Ty@}8+gY> z&(js?SL>$V+R3A7HpxAujln8z9J2YPaKxDcka+-tDkoQ>p-}THLR$H z>*aytOtUB$!5uQWAe;kL^;cekAlMu(3033eLGh~FxT`r ztLBvHzFu8>ZTBCn);gut{+k}9Ya;o08Am@bET^Y$@+(78)-K?%M3GD;>kABUK6kcu z?c3aE9e<^K+u<)0$)b2{Y1-R)Ha1XXYGZje$PG;pTPbUoVSUukVP?znJvVF<%&Q0Y~!i_0G4UbQU!i1d>;5a@gw5D z#R$9~q7OgD+WINJxdZoFHlN*cbG6F;BR}3XLrAswk?}|3HJ^-c?55RxU*bud`vXjr znO@vVa@Ujp0JO*E&IV7Sj92ur`yBqs+7Il_;1O?ew)e!Eo+s6{0yjLfH#4ZIsR+oxaj z%auGsANN5W7>~$)75UYFJ(jtyH-@8;hMQ=c-6QKOFF~F_EkD!sVpvBI!g*1Hfsl9uryZ(PlGIZ_q|f*!M}@EaOYkF3u(pfrbBnu) zZ=_5Xm7I;fodyR^g1++culy5x;m^dIYmG|U=I=|@ZfBO}Yq;1*Ws*~$om_NY+e+7; zf5A93YaKt}wv(vowo+;EHM%>io#l#+tU>f793C_2U$Q<5*0gzSQS`}#n}^I8bNN?^ zHyVw`XQ2r4rDNrff?fmhU&Gx3Jr~CtVHSCXB^V+_Lxo|2cJJWeoQ&eTTVL7(!ZLhg zw)ih)sYU&-a{-ks@JodQc25`|C?}FQuc9Zv(_ylf>hQ0WgD!E#Dp&9)g*;Drr}&3Z z)1aE~Pu$kG1qYcH@wDu2pb^2xQCzhsxiuPcwZ9{1&f3Wu)^Se;s0`N#>IVuqAd%A* zV*5mgS&+kXZly;DjzH=BwF{q8ug1n{qlyi?&QmdnI|?s4%PP_rD(~E=#wTe zbN=t8aK1h86_1Ye)no`14f)5-$6$Rc!^PlapEW0{+tBW%UN*GWhnje6#)dckn&#f$-I&(RDbWwM9}A<%ZMq9F=B2 zg-PPii{2K8OO^C(LfT8BHo}q0t-a3xVQTPrm{FwXT6&XSlbr1c=w^7KZshS3S$H84 z1iv-Q5uccJ0CdhP=HG}qH^!YiQ~uqs-7ST@g__rqaQxK&0E?p7nwFNvH}-v^ebbUoTRx_)>3`eO>L|V*U!FMu zlW@wE$0I$lQKpw9m*(avTMP%u!CnV%O7%Yt#`<-;Z5Z5h#cde3<>*05=06)psKoM! z{>s1&h&czpK9#EWw%7197a?3fQP#R~c_fR-;0}ValSpfA^TebcO>FL&ndI7+gfz`O zG?uHd2EbA~8pE5zkh^(;FhLdWkEI*P0k|0%2aj6FyVH^xJgtO`aB*5BE1x#&-Ve8( zcDe$+a7A#_cw*itj6rg~ovZ2l>lJmuTHy5!C?gexd94(bSr$HSZ-%-zitbX+P=%$8 z9JW1282VFwDf~M~v~t=+5!<>%VqP(XKA85edeI^N$=1cWTtzS=CyWu#>s^+sr`YQD zvqf;AkO~3EO6sRwnYbU98VAA+C*l5$F1e_mI!K1_=e)0yPCH?66j$b-#m|d+Z-zc7 z{{U{=x0!Pqc@brD6=q;L#ySz7ps(HSYr}d^h%IKeiY5se96ubd2R*C#>iFgG?@5nb z@rR7Ac%IW;hRtoy5g3|0nH7g{e<9nN^zfeZ)a0(3j)xuaxZmp^8MQr2T#!wFZui1T z6nvsM-C>LmO#IdK<;;DE&TGg%3RuT?eXn@$VMWz0+(roJXkx_vWLLUKi6wb=4Slf{viynAyoc_n`q!~)O%1$HEbJpl)Uz?* z1s!YfzxI*%tKu)(d*Z#n!!L(w(`i<-i+BQ@Eb^0r=i3?QvWl>qvNZk}Y5Mo=LGecO zUDsLv0J5TSHTfKZ@gkxrljiYdyh-*iv~a>R^P z>l-<)Zl8NFl z_#04ocNoYxu18YR?ljpvsfG=Hrs+Qnp}Dh*WJ_^rKI|0dZaXml06f=-d{6jVZ7&|q zEgn-Fg)vJKmR9UR;|fQi>)N;|<~Nn3k5dtpS{H*p3fd1^jPP+?_lf))d!XDD+HuqZ zFmd#*8RN%MgI=s;q_jLaLB-jk8YysnD8clqAXNL+HAJB1rJw?s{VDXv=|B-tMJVe) z4oJgwQMwN?Ltu$D0#1I@TbCSyVA1Gyl`td)r>bcqlcYGWRDorbSv#jMMw~w*Sz>c z;Jw}86WuST-WBvGg+3i=dU02k2Ia3+A7{0UBwztvWn6Q5ME5Y+RAa3}&%814n?|sW zT4e;+tLeHbMF z*?Xd95;uCX_p2!KRP?SyUJd&io{}<3IjuPieWPb(Yk9S{XQvgp6rN)2&ji;cX1~1L z=C@#JAN^Mpxk^o=rm-eDb|+ldTW=8afMN1?u4djL8lyKEsdWz!>AE~GJ-g#IY}WbqHg3*FcEO|)cf&j5N<-XigCrQ(f2=bMOn zA`XC82Y=zkyF&KkYx;JtUX}vU=CR>d%sHPb*z=3XzGI#Uti?O4v(;qOwL7PiUuSt$ z5jVeGr~)t4008<|Gka>*n@0stM_;;YL}XJ?RQ0aIMp29rxSV3St99NnitaR*XI*_ z6W4wp>%t9oJjJ^SDCzi*;rwe^CmqbAB`<`r;q48U+BLuc04vh$G!UQ~=(MdGB#eB$ ztGck#hS9WC8gw~*J4WcBV0+hdrf9Cxq*1QsVgL#`C)ifSj+|a(wt_wKF#sy^N%j@_ zrTc1r*3(;E-hR!W0pybJTDl)&)b%R{A(76Nk!ijRy#1Qv{{V)2 z_yA(o{xoaoBiA*XbSM51xaLi*Yi{5X_V-2$B&&gI8NkDyC(|LH_AQ3r;LpMI8o%uY z;+W!!Tbpc5JTYxB8)A(}AZijXARaA{hi{t!zgmA{f7s_r{hNF&ny-pjYCbN!MmJX^ znFJndZfN6X%MwpANKRe3r4`MqCQ?yWF8;)Su|I?V0A)`I!E<*MSAHm--D0@90BwOd z{meXYynM&2o=tu08sKK4j1Rm=N^G+#t~yr60=t~o^6~!w1+4w4BJrQbFAV<3nj^_2 zjlIND>T3fH&n&$c86R;y$5ZKF($9=MRj2qn!n%ixH8u-xr(4N!BOZZbUB~%X@Vy6y zd|CTOe0|kFYQGTO5O1|PXRx>DGAS$o4ngt0xju-iz{fv@dHI$qG-$TnkF?-ilD1`- z(~T%yCa>hLtNAa+$D-)|4ES&G2gO#pD7bhowM8*o!@qfu;GOB*pU`cuzbq`E(|#R8 zrraV)cO-vmYlk0n%2cpnmFfq$9Wh>O;y(yz-Y1IAUl&-U&7{beX&X4-6NSNKBN^j8 zJ+acd?-%O!&1jm9q-ko1iyVqymxy!egN)-Hc)`JVj|)j@ zCcSkP-Kf=v> z8`}%h9E%G94h|*V^6kQ&0UQ8+UVZ8-ZxQN`;N@k3o8-pZ;Jj5gtJfMDcW48xr z8~{7ka%Iq@+%0eG`myR49xHzo_*twcoZ3telZ1IT;vnvkg&!u*n1t-PI2~&jPuH%k zEC!2r1T*P7qaIQWFvjD#gnKf;a9bGZo{Nl~GsIUr2n&uT(M2(5rA(I4!P*oH$&&WC3S#vXAWeRD=)~c7)yX`{LNWZ#|QuAUHk1Z9^A~O~nM4nr#E-}}U$;cz6L8rrQqAi}It+2R)W=)D2 z%v-kY4ZT?KyKp*Z1P+`itxA+qgNk-HZ*&_QyDzXvvMc=Y#~^pwHslS#LW~ULZQ!1L zYsU23Ro6UT-X7FeO-K@ATuL&aWaK^^U>|G|*1gA7)qFXo+I@#YhJDuvo@AxBDUmyX z0`17-obov*HRRJzVWRkg=6f3=?-Y$Gn{J7<1fP&&x;y2VC)=#)j3P@c#hBs3N%2!+^hJQH6Kuw`kmWnUmP;tZ+e*77X$U@8+6x||AeH3v){~w39RA8`DN|=Bai@4U#Suk3 zICp@&LA4e0kwj~>1to#ba62#b!O}h=YrX~3?z}r4q6s#?l_j)_stcjP`9a9!aG`kM z;+IkJH-)U3w5zLcu(j(eD7M;>OzpwjfLA3(0mlcXE6=azORKZyxme&;jRd1~sIiP- zw|oKq$fn)3IEJ)benZ$daEQ5C-2J4KBj(b;8;%^>! zUq{s~bgvS$dQ`F@MPds?Hn+}+@|y#e76OJ7eQ*S#?cv86>!r29Dx$ znVYuV{l}*rv0pIw!^B@4{tan166=;SYL-!{jpIYSDe6>}2dC*?_wZ|7x%jE!+iwvy z*|j$e%{99t7lf;DBRp|}0ppBUU)AMK9P4e59}Sq{@fH55D77@N%GJAkzT@i;4OnaX z29_2JgFo5X1WcoFSmWw8XK5Xe8T|!&kA}4dyN^?oS(YiUUH7Xf!g(Ze909e6QP5*N z5J4EPB8NlPE-bBWH2qxKK)XbeB5w+(?wow^L>eF4s-)XxG8D&_^E>HJAxz%tn{u~?p zEBJTd%`nZRTLvj8cD9w4Sa-M}g;??fDv`nC89ZQDVW{f^!&a(nZqj(}K30J@M*jdQ z{v}rVyKO25(AJm1p9t$e8FhKJsN}qZ!fJ*o{?R|$KGfT?Vn*1cAU}BX$j7aAU@-Nu zH5Dp<@W;*Z7Hfsda8k$OrJ~U;vVD;!!Eb}tUO3bv@pp;g-J{ybC9+9|<20Lg?-|Y` z$6><|4`E+_MRWFeHp;*i{d)mcg`7HNx>{QxL`4dUqW}TG0=fSH6l)$8_<7;G--w<% zoLlMIWP6*QU60+4G2Phk2lN&CRtpInJvB<2KaeJNNXinM>@W z=hD8Xx&FdGvGo4{2DE8DE&Nzjx$)+^8kyT@+h-0wSR?-K2L~TY{T@{q;8lc+e+|%d ztS=ULv%~%X@YUCgJWD8s($#W(UArQVI?hR@>;}C z?(L$_3J3SSee1+;t_{?(T-`C2O}WXhI}ux*@~w`VaI)0LI>qwYO(px8BaDK9n}E;3 zU^|nH4@KgZp%$Pv6c*7@OI#FwQJTtWG~qU_8XP)*TideOUsz^XB5A3D9HQg zjtLz-1%B2182z8Ev@6jZQKiM)ooCyCv7g~w*Pn{7Z6I|~qow3wd?3I0%i@oOOYq~v zC^UV3;QgKNEujdhZokBe1{K(FPBGMhUwV8q_zm&i*GRV0yg}k5NYG)Vjxpy2JwWaH zSFwC6_lL#E}cQ8-}c;Ixc-Ah_c8U~f-RfZCvsNjHj717#$>r(qHtTOziXLygm zdN+yf=7#-BNi<$jkvL#FkI3>XO7!HTCqsYoL{n={eTd`N+GJ9QGI}*bbFN-#a;4q# zZ&x7{^y}KZ^Wm3-b*met(zP96UE5o+DaKv7>A3caK9>jF~s~j}y z;wZU4b(d8}oYE@WUffL(Svd5nmM~vi>b^~~fomu%&Hcvo$G^2%bQYpG*Yl#>%(F9` zWc}YkT~dYAZ@M!~#_`91^v@RPv*}VQgCOAJ{QWED?N{J`!_6M|Pq?(x8rEAU0L>I< zY0B;S$m{j(n)>5Oj`~e%>Fy3-DxP1rt#uPv!?*Win&qjM)5NZBTRpbyTrpCoq4Pid zEe{HKXIDNIwqmz-$}%>UQSFj3P1n98XzirSsNKJp97tV@VM2u&Jf1OKSB$E z*HXAZwOCvukJ_j9bDPE=4am_!X0v}dz+VxR0OzlD9S7lD^%gRUl1VtNy$z%6 zr)RS~e(&~_)O=s!10Ixu($)teIkAV8J}NY5DQUq*O0_Mz~8gK{nJlKm~LAyu%_oo4zv1;mv0ENAcCfl1HmfvkR%8A~`U3Zpq*` z9V^iO(53AZ7vIq5`n36O&(z&NRE}3@uPvGpk`(Y%f!`ybuFYeRLR#bIP(3kTL*ZYD z-VN6-=F>DQ%X`V~LbLg8y8!gT91-;CUG?vXY^KmIt*4nImQ*UMour=F{{ZV(r3puu zn)f-J9*V}ToRK@UNKl~00-s8C+z^kL4%J6Nu!`Q!J*%}!fI&Rwwl1xfIX9prW1RN< zs-rJI4ZdY;m=9seBkulHm7CbJ%swReek;3}Z7pEkkM*2!fr`+y{=?GZk^DnC zr)39#2(GS8COsEQlK$s(_Zt9bnTwnbd+<8;{+06IjV8A7ypN_wC(ii;<_e?ptuM;% zE!B>PT-S6nXL%%h63G?MoA+05u50<2@kfp1_`~D9Wi(mjxzc60k#zW2VO}!aNO9}K zbK1Vy{kJ|f{8{jyfc0O48VYUx(lA;70BEVeSSNf+iUu&oFjOAEmp~saIbdNr)IX;Rj7xq#384trR3J)7=Q0>+=NdExQ)xPvFH&#B4CNu23 zSBw779||u%Civp>;(v|%7;VFI*RPP0pJU5UN-w{n96gwbUxMfABO%PUPTJ|F$m!bc1Zh2 z0CIBJAD7`?ZYvuY`>{QY7F{W>d2D!|o8YTxltU;Dz~x6=is}3}r0MY8#iIC@;q63j zKFxFUE)@R&0ddnEvN2w%Wo+84%Qb*wYi{ogmu=yh02`MbJy*8VUd5#NBTK)s{?oEw zHWfkUC*F0(ark?2UR3c9sqRz4CGeRZ2iCQJ0>UD|k+SbzE547IjkrXJub9-Q6o1%|%#bkIh;pU+v2HQlqjN6&kZ!JJZLU{Q}9B02c z`d4S+p95)D?Wy={#AxX~-|uYYm3+9Fxh%LNhX9zFhH7?EQPENXQ|WWA3)!mHzO%V%flES0{NXQ=El<8(QA+eoos+CpmZht@CBKTXc z=wLO(OreHXJY-j!J*&pWSYs9a417cQPvO0L!q;%vz_GF}QZx6z4m(%jcf|byZw>gK zYbF~;p(>ybEc_15cUZ0|TIWjP3;QAWQRbw5^ zsnDFXNb$Q}Hty}hLbX+P!wcEv;knpis4sZE!riRfQ)ay!p+&&tjE0#aSbwNy>7) z#W+Z1IBu1gB_RfTQlz1dMP$yT&W;ud9`n!&(zTrk-(2Fed3${;PQmu@)trr@ba34H zjf&icU)eaZ<~8w)g_L>up8iO}XZgt{Uc1&2u=L zBU0|^<=}Lx4RBQAp}Cwx7o}oJf0wC9X7sm8qhRYs1`+JHlj{?&N+#KlVkAuNT?xP%!#(K9$U0_y+FXnrnmhn=E|T19I`lAD2q>T{%w`-f0p4014?= zI!Z>6sN?8-%%2Z0;cqOrT)$y~->@C) z>hBHcIxJd;+U2%c1c&#kdJ*mEUgL4_x@}6%*7M={GrZrf+{`eg<2 zf!|J~Bz!HScym|MwE+eFtcfN8V9Iz>a1W+4$LU{t{3g-nifGzM*$jJdp-Q$-BOap_ zjo_^tSk*otTT7|f`5)OX9yT#6OK~dz234{!3jBzER34=B`oZvv{t5g0MSNPbkH-;P z$2@KxXOc%%l4&!OmSgjA(Xq5+XgT7uTAOnvx%w6V00iRj1@60jZF@XKEM&gyU>tvi ze*Ri%rWhB0Bn)R2`S1Gxd^5W6SAg2)?lO}T1=4SkC(g37z2;j0eoNJ)WL z19!c3w4RLPO|6f{-xqu)*S;S+%V!6lY(D5Gf$Tj#hP>Zh@yCLE72#PtZ}CsWnqP%9 zXfmxnqdGyBAHf`a$s>9!YQ**x`zzu<2wGmNTi!+vd-WgTUkdzU{g?bd;tPviT00wS zi;X&8v{|&!$17XTKJ2l1ioqRvsmBKxv;Ncg=dD{uC565g zc)TrVHa6T(zvXNB0VV`|g#Q3o`~kb}4M*Vji2nd(&kpN95WXCEeoqj1veHG5N1p9G z+f6y8&zX5~7DHT0ols@jv3o>+Pk*t zF7Moo1;O6$otsCN01LEn#x1+eTgdEfDB1_xAMj3p*@nl!<52k1;<=Mm@eTG=&eDCL z$2|W4ytpO#-zh9N%aT}=U#yl;;AGc{{?13S=O3sk{rO!c3gX3TL zC^v|1ygU0Cd_ugnl*I3)OAh7g1lU85z}NC&;BOk8?$Cei0Gk<2ZgdPUi*;%@OvJHPIUj_UnrfVJ`y}PuDVcT#U%aba; zTL8qwe(wW}dvjkc#CEiAwU6F>8jQbt?5%esyzOiK$JM?s)U=I7?Mpo3^4cw)VPnqI zz{h?oj`6pMbve8~kWG@xoxb*RYteNtho?}|VQn_)8f!ur%uqAQmixHg00TJA4RbQ- z2`oqL54nfR$}!M%$7=Z0(fugt$*Cl)c-O&swZq~o%~NZotWgN}Fe*;>847!Q`&ZR= zcX#n>me*b$xM(hI=Sk7ouvSDMf(Z8mj@YjcwidcOJ&vI)R3&{W3B6 zR-c3XX6@p;-wfJbMRj$i$M%yB^>#55ppdMaLA}f~+9fUGA(04k&c2;AR;;d@7*70hw#W|LH9Z@Afb_hTl zM%~S~b4aHfokA7r8eAbl&<5O4V{eQzB9Np?xX?t-b)^f#fc^e3$ciAB@E)*T8MtH_> z0dCk8gB+eMnqt?fF}H_q)R;EhBcMdcflV4{{Uy-8Qep7G<5LxsV1tk z`LRnOh`@}@k;4qG%@as={pFApMhd*~*m%SCUc2$Xg|)@-{fbx%w)Q=Ya{3oVL9N3Fxx{qNu zBP$aScQiqdmGh=ol_!svY3qxiX*zx5TH7-ly)|TM7T!63bT?&A+^EAIK3pEW`%tmf zujSU#($vj0suT8Wq=BKC%deK>a^M~bR@~h<>BV*gX}%bmLln&SI8|p>Z!cimv<#L7 zaHBb4fCmDz)VcD<)&BrrBbkA1FK%SKX_*{cNUapbnZU*e1S+=~=%8`XV-=mLX?lF~ z+C?F_WJuWqla={N;E{&NKXQjlWzHGmbOUnxsrCO7}%`uHB`OAre-EoU*ATZpJ{z%t*&WQ^xj6KGbHM$tHe7 zmXC493in?599D0LwePaSZ>Cv*i#hoiAoCY-%JNT63F%o8YH?}NwvqaOGIl<34 z8#37#2c|xh^fZhmYuLNu{Z3&A+H~ttsY-)kR&E$$fI{G}``s(fbPaZY9BBsH^3^Sx z-Ort6J7;(}0B+A2^0Dpw_{SHkHWA4dB;dg$Gv8FyT)ED@P?TFG4TDAAq+ofcq3FT8C6_1)AKIjh7EuT&fYRBD@@V+Kj2Y$ zqRV$J{k6=JOK_6&wo*pWyeY;r)O77!CbMUGC+@Uso2h)?++eUjFF?ExyN(BJaau+i zn96f>wzFr0=pHxm2gD6-;@4W!FJ^g}qky7qK`MSrjEr?3rG01MPY5lI#7nH{%WW;) zu#(<6CJxzA%K)rdN2on3v-nMErRaVfv9h*>q>-mL6EQ-dDxP|-FnVV_v(mf&01P(9arW7xb0Y;Hq_qETexoR;%n%n0SiFGbWDmi zw&Tbc7$bw)yq{OP(fmQE;yqSbo+0wd6ssqg*(5hiejeENt~XEkGx7W4=fqafeja$% z@(6V=CGMv{%p-?CIg`u)mmL5fF7BKP=)&P;UTTy#XPum8)UwPxvDk&~MQdw*w)C;; zp9=gJ;!oN;$MSd!N}T@yWNEAyS=3^0l@xLIcu3=We-RxAJ#k;O`ZtBNZwB~s)59Je zwo7X(IMNtljBQm!7ytoWufb1)o)q{c;7eZw_);uK2rVS9%#%vov$^ifYt+m#NjOf} z{{TOQewCKx(81BL__IUUuH=7el{*iGUpDgkwb0cGqCOmwA@JQd#EqF)lL;E00 z`$X$2V<>pOALz5B`nC5V^=q(m+1f;~!30BU zM;Z(&f#?NYj%7tJGX+E$B=i7Oh0P1iLc>YZWz{t~mfqUg43fy_3W~qyfnSiHv>)vT z{{RuS{{XcHXtZk`!%ww6SoHH)dY;~by#w}`{h=*9;R?c`m7`yQ++~K*mp?>;)t@ z(B97+1^k%U;iNs6=sWfmji_paRJj&bQJ*F;h|qOnJ*$n?Zm#Wa7HIk~9lHwQX2^HN z3%x$~RF+Fs^56g}A4t?-euBHN*=OJ`wc}rjF>Br;)8t(`2$aSpTn6*L=^yYOay@I@ ze`Sx^m%-n(wZ+Z<0E%@{1(usUX(gEAS@)?q&PLV9z`*Ql_jkbF0{DO6HHX8tNpBs^ zovH|x%7*e3Z3mvi7_NLxc*Z=db80l3cRo4NJOkq|68M@uA3%^;SZU@sQbaj?N5Kj` zhC21H*?)x}4lXneDjS_(?v>PKk5OMW_)haiveqKiEv=M9rz|4_ll|lBYwj-&{7>+m z-K2JBt1b?Dit%Aul6uJRaFbdib+x@eNsE3LWL#&~t=jxb(`Jns9Z^ZblfXXJ@@K_= z5^1`Z#a$1=T2$^P)FYMdVT~kgLl)37$@{cs-SUU~z#M(kUr2aD)4(XM9IZVrdH0F5`+pAKuCby8x5G4&fx_UAnEDFW z(e9I6PqeK2lZ=2z71!9qYpvQf%dYP;D}p%1a|z){H2qjyY4C{5pDeiO!_*Ffy!q3d zYI{mw47wYsMaPy~qoTc(JPYOX?@(J@m2WKd6=HjPiKkdqHpYvOYR0=CVU&~5d)K72 zF;=m1@5MGA8xM1TCfMUb=a1oDwT0r3j2;uyY~I4g^&3%=J<-CWE;n>6I}fdS{{V~) z$B7&JBXCPu7w7)-5!>lsC+dF;yieiJi3i7U%CBN$0{qV45F-F10^N83U{{@v$JeF& zDBXTXMGQ12@iXj?1W&4519N=k+4=dou9wA=!>Q>rXr@0fN`*)x<@D=G;P|aBG`Q~U zTrgk`Yfr>6-A!(=X^JeK9%|(toUyH>K|s(+se2YXD+*xj)Zj;c25O)q>Hw}eh6jb@W*QN9ur*=)~i_j z>>mxSr3qm)0kWXyAIMiprT)#(-KW~sN-t{rdJl#chEFiz7473kOM#HQbR1U&t@w`f!55Ek){S)lTwx9fKb9*+g{Ml=lJ`h;Wj!K#bQ&y{ z@f0t+o(?;96%UeRQh?xeuOrmHI2}&Y`#Zw+QzhJ(3i7Dfx%LN~epOn^4~TJT;yEru z8BgCM45Jw199L)7>9=Z1=;!t_`&t_QJMp9*4KZo<_HB7Isz?ez10KGWpuN zRIOw4+r<9>wg!!5CGL;onOecF=KDRQFhL~DV<2UisKD>}eJex48b*mWkshLt+x5Y6v)6B13k0b>0ICK5Aioy_(kCxO#;qX z4zGI|hRstOt>Isk$jS%sj0OJldjncvXw<7wDQ-=9#u2&j_w5;dqxiGOP2sCp)Y|D& zd6KtK_W)oo`~V((_^&DB_{3|}) zQu%*+$bZ#Bew_*JUy@(&wtgdj&|>5M`!$W@j~MG-JMksgiM82GmltvuKEtpcfLA58 zy(*G2yL%u1*8T{f)8)Bv0N~fK_(#Cv<;1R~*S_2mKMzXx-vfLewbmjRSCRQDz-o^nCPNAksS7y6c`;frO_yfGKe3Hh2q zxtsy-+))xOtf{;F2rPJ&7&rz;A^vKb89D?!n%7q zDQ%1O@=0yZE&(%N*fyq0Y}~JIj_qf+S}k^n)K?LVkU50 zx}1vpm*PLg>rDeppH%b8TgM;oBvtL`qrHB8{6YBR;tw3@$)dw;EP~)2r9eaqJ#Y{7 zuG}N33G+bn@$sippCv={R_4-MJBEf|;5ex3*e*O9sOs8`(Z>i59Q14q5A(%ydXIp$ z8+7v`8@`qGP{cv?J{ovRk=bZ)bj^abXVEnaUS?DHR(qo%#yVEXu7@*=TacAe)}#7U z*fBjRHn*iQ4WgZ~{VG`Pt|efRp(OMGPy`fK)ZQP~Er|P7yT_iJhAQQ+hVJJ3%Zg4( z6rkD71t)+jO4~`fx!^Aq=z3R!9q?Mc2T$1=dU!!~(-o8Z};Ze_0X zN$_M5u->3oeQJ=X0547|@`Xc?E0$HH&qAIS5i9B1Ev>5M6lS_eaT!s703E9}CdU-N zXu+%`+1m>wbx}^B6Q0#HQ!BPBmRr1#xUIWs7q6ve&7%4pBn${6(wf%*Fu|;$co&85 zRa)TZ9qT?*WX`%9vA2_37LqFf)yG>*2-?-AmjP2?tlBnxwNg=!|dC~~J570o5! z5^U{DE#GCV>}$+)vEH*|Cbz7v^M&bLC8K$m$;EEPbF&!7rDoA_k=P`sa*|D3`%=c+ zatEby15oH!pK3`jvB~XN$|mud)?7%^Cnm9^e=Ty|X{mSSMg?bGO@_x~QDU5m6W%wO z=qb@(U~+lwRA&i|*u`bE%gD|vOCu*Hw>Ib0$va0B$Ztp$$z0rJ@(n#M3NB4oAs*E0 zq`E*dd(|bAb#q4br8`ttN2f+)QJPx^RVb5yK4 z-DA!cy89@A0Jjy%*$tR)%Dqp*J`vWuVSH?CQ*3CPok9kPplM0M@UJwJj&a9v0Le zOV(|g-yDEP$Z_avyAKFB=y-VA^KX{ciPT3Tvh~g?+zOLOu}I-Eak%s!Uuxv6wI4nD zV0On!-L-Xje7s{kR_-=96B~I9*0URXlqf;u8lY~4t<-ZN;C!SC>~D+ecCo{9-#iFb zC-&^8*mm@;4)(!iiP|squ^q&EuS5BND$ zyqOMsql)&;Ek1BXdMKWUsAw=Ocs1`n3)Hne67nruNtDAhBpiU}B2}+4{NgSHb@P4}WR9Xs$Hqmqqcen48jQR2b$9+{Soeo}(EZjeghu&;J0m z*N40hX{LV7{{S4cDEu+vYmJuHckr-XA5(0FK$12-&QBk?d=AP;;=h`If&MJkejR*3 z(!61#5|Ufaq+@`?pqX%h^%zkHu2^dFgs`MZLEBYV!f8&1w z{9^EBuf#ux7x8MkHS%URkDx2TR#GyIy%|UZS8|8CryY+rrjF7)wnz##!k>**y3&k@ z_!(6=&MVXP4-C4#U>iqJKl=5@i0*cI(W?kw`@C+>_>Ob?vAYw-6-)%3Nsjeb(UGUNEa`qlB5hyESG`#pH$ zRq?-pbV~`YwV2|IQn*`nN#=?Sl^$Rllnnh385|6Lo89T~$c0#9xU26DSo!Nc{2pX0 zoR7o#R&%7JYYMI>K2bdY6c`pA2ek4~pPv zVSkyYjWAUA%8m&4U&^`p98yO^nv=RZgzk0|(w;i$Z4;5}o)S3nrGD@m=G9+;6z*YYRf4P*NY#aBKWz0w7arJ_n~ z?hIwr7K?Ow+IMhv=L4;OW3&96E)Eaj#eXrMw&%c$zkz=ln)kzZ6HjS$2CSZQW?9wQ zgpqC-+)>8i{>jCB_YlzMqNbL|>;4Yr(Zc3)!^!+5BMHTWIt@tNpT^BRWf({kt4Swead~djjq7g!JiMJ+H#= z3j8Y9w5@)4TH0A6ky`FD70;ZRj7T%WpRNce(-p;Qt?{$Nw-2dl*OvO7gj-%~MpY9@ zw}n+~ao4c^RF|6H#IK5WEui?5PhB3@mu8MqND2T|ZNr>kW78e0rX1YA61Vlx@auaY zYwpl~)opn1=+)Bv52vquGk5klFVw|)vm&#bck>KqEIhN0G6Lj*pO*%?Zy4)VJ{-In zEwqbsdJttGkV>0d1RctF$4qtXdSbi^FAMAb8(5<7cByl3DNU>*a=FObdK{dC`5aeM zqIiE*@OwR#{DwP;RV94#{R1dIp54c_FJ~0Fx*eZh#5SooS^ofAA82?w_8Bxw-9@1{ z(B4^>`)ug9fnF9HzjGpwm2eV3zcM2s0~{Y`xz>D3Yo}UxuJcWf)ihgpR%M6%4m*g3 z(-flv5*@}(`&8}(b^F!eKMr+?tn~P_tG!IwS<7zIICqSok=T}HWnyD*x&}bq%AKQ{ z`f}Ft_8k%{v3Cr%Os#JYLe3*&D9e|`na?{{aWR9($N(B9t)b&$lZ^f3zaLM_BX zjHJc$+o@IAx@KgNj9}oH1A=~Bn)t`THogsq#S+b~X={4WLvi+NDO5<4x0L%`xG}0= zm)+299P$NvHi7Xf@FQ8?!zKO6GBVwHa!DC?8(k)33e2j!1a5$w9>-`_vR6KT8<$d- zGNlbHzoqDV_r!fvc;`@q#M<_w63YZ~q8Qx+&E>N^O)3y##H)ZBL!E&D#{#|=@l=s` zA>Mf|TTr@YCQBriCY~snR1C1)y*jc6QH=5O;16q>>gVGPv{3&5W?L?yUK$CT;Gqxs1fcNMOYyE+cSqfu8{X01gIg zCwVr`oChWnDwRk70B);)Rz7Xid~M=;D|l`+jXum;va2vujb>$!A~ieEd5oa#7&-0G zVz^Ch{8z0{ZFi;KTgCQ}9L8ovXoC;9ReohZ-UITlWAV3yw7&#+l_b29+-cVtTXQS8 zDpc^fV6mqdJBYy*<5Oy$5rXOo^z%2KrBx$4Rknl2&cx+$z#C2mIW^5BeRc`POP-W^ zcK)_Hh_uV;zPk^<&N(Za_F4D^pO_ zY#@1|e=0XrFfW){0~~$jY$!gM+xb_^P<&(XAH!c0+GsvFxM?k*4>n`Z$+&&(!>%!a zF`BAzZ%Cs`_=r-6JmjLb*F)+pJ7hXo@V=I?Lu%Whx<(PmT|o>8BMpTkCz18wR~O^U zxa}?0Th@gd0E^~s1yK{AQZdswBLtp3E5!VN`%?TB)qF_WHizOVW`->x#nop=aP*%>e0>-D(hDtbTk-$0kGkAdY|@GH^H*)Zry%C2x_z48nw=D!6&Z zO>|9uKi84Oc-zKWp1b3#8y^n9_cqgsB)Db5fZPQb8$ch8dpCsQ)^(fcbzNQKw1Uh> zZ!kw6F4OY#9Wpt_an#^fmi#O5FO96OEwww;w}#&80L-&U8+>fra6%G8alkk}*sq~A zeI$<%UPXUAA7iv)$RIvkE4*i?8R?EGRb09wEV^>8T?$}Es<_;_Y-2g+ z1NIa4as8hBGy6Yy0uKym%4WNe2fMnJjFL+y`_0bP94P1jtWVjW_Itbdd8D5kc>XwB z#JZa9mN>K9zV|1~9vYuDPmTT_NAkaj9Ag~Y5%qi; zwjMpS=6&CnWM>Fjv4P3p3jB-z0D_eM+z@;y@bv!x1HKnc_Ky(iU+A}O5X%N&KX8O} z$NaTn{^0MAr@wCR+atr@vY*45EbJr_c&}5BWu$SD=0K8U1_KB8jf_Sy!1=oXf1h83 z{{RNQD12e~pKYZ_5x)3^;(wX9mT2@_2N^FRIokVqIpk;i#=V(N)=2#3*EYXopV@QA z-?U%EJD&%^J0O&-{wCGH1VI5K{T?&wdTk$0faCh9@E3r*CGfk!I)B6O0$Qz(m8M1_ zh9&Fn%zLwSCb-{#p8$Ll`#F3zwD@b_l-`iyOS{GxlJek;G>`55(flI4ibEQWqM&cc zgQb3de$@W}v?O+`Kf`|r&$Qzg)2Cto06diX{{VRQ73n{;$H%Dr0i{XdUkyqQt8jA3#@w0bD7$gBNQT|#qnI~|!Ncbu{Blj==#*Q?5`i>C{W@dm9BwDTSOvd^piO4W$NMGS1vX zhT){+C8WRrmL!w)ui4KE{5!FZ6#F|B5w0*e&3W>~sXudLvNUY9KP*%BO7P!;{21OJ z(Ji8kSjFWl?Zkj5TsN=ttZ&)l_D1lAgL!i&iM&H|s$E{)OK@S1aM4?=jk{ncFMxQ) z;0Iw}uzn)&C6)e+`hK3O>PLS5weW_k@KVc7@!I&0PiD39n5?^aE!$|v81K`yc=*^$ znLAw_*m)~kW3G?kSBmr)?ldn2-P(m$j!0#^4JK1?-5FKsmBewldE& zj(0ONHcup;J9Mt|UefL^^(C=|wyQRA&~P}fXqM4!V&r$td9R9IXtfA0t;(-IFm66y zYV5?(geY<)W#n_ysVPqHaJ11)g{yt71IM)P+t(GnXWQPDwEi7ZNF-qe4-9u-Cj@9PAD7{-~ z8Dtw3hB@N7-CxFV+uu#2OpG2g_qqZ-N3|i;^xbOQY5KfkSXT^y`j4e^tVKGBs69=2 zB^x%6H`BaCr(2xM8#E{e*XW=QzmECD6BaY??)1d*BsjrcIKjF<*!^9Vw zKDyG&VT_dCTX%|u4$pfgu$T`m#{424OPl84d zA8O&WJvIa7q%$|kjC3Ov=qNm&lf5cUwW=y1nPQGcDZ2xr@_!2WTjCbK z@oGOECa zwu0GW+rOEgCuC@Z<8db(571ZO?~MFwulz{yb?1$IPE_6AHweFS7}tb}KiNU~oSyaT zpR_l|o1cw7FwUJb{{X^2r?U$bJb8>p8d5#_FQWZx!Thbfiu&BM1?-{ECGS7L@v(LJ z-zwP^B>|ScEB&fIG~D=l-amxi2e?@E%~B%w6A$@wv;H{`p(BL%uXXt0@pnY{G2ly& z6l$NmxNqHA+mF38?ZT>$uU~5XqtQGs;~$6~DSJ&so39XS(+RaXv&2e1Wq=;Z8uH_& z20PcKDOlu6=V|am;f|f+O;#@wYA`bB8ZOZUPyYZd6HN=-Pz0BWz zzbdU|rfE73mv7;{2O`eRc7{D4S$JY>U+N3{ z_Cw5Z9^88Lug0&m2qHsn*q4&VtcG2@{>QYt^7Qp-<4kiP#(xgqoj3bN)Bb%^@;~EP z-`Xwag@olv;W>_Iul1497L*@}x%mc2kUdSBzO%cw< zQ8;4>IrXsb^|w;mJ#C^z{ekuucUkt z@#9@z58G&3t4mQ>`Tnb``Fa&hc^*{<3x>=9}bOwg*K-wbLF z{n$U4;~tgr_l|xc_-DfxU)egX%#S>ZNQXH%_dm#WtPO9+o;TF&BZonr7}8Kovl$6E z{{Uwn&+xATpGniL<+{~wV2<}tY?PMbOzuBGI3C!g2NXJ-`y+|?-K^bs!@>`%>9dgq zins2N>{cZDbu6Rs#e93GUuvzOOKEp{_Hi%(sww&X0mdus4QE5v?sa6=#+wL$f^MM< zpclq|O~--jfuBm}Pk`4`)&Bsr)nhO@GCXmp+@$r!Jpkgp3K*EvhbugsP7ZZv%)Jkf zG&_A^CKuLH?J$rOw|w*%uS?Qw^(%=}OZz};=zc&{Wfn7@4`6a@)4W09Z5mxPSZU~1 z`J^lr)9$ZKj1Ks&J~=g=CfS=nv`Fu5n>&-|4hK?m>0J*I?H9UoLkQ_ROJlXsyc)W& zj?Y?&eAgHbtA#%2(zG@2*}6-sQ>kh2jnINhjA6%Y9DqL_)tTVWiMNgcf2jjM!k<50mtcF*Nc7@#qjp?`x94aXSRZ()vm zS6&|xUZ(HlaAL5Scxa>Z=y*57uYx`#_<`b!e+>9?$?tFOSymf(*XD13WgX8!U%USR z0>9vx{vYsOl@_(}1jBi*%DH7hm@WSR*-umI-HFXvxZ={mL3{`TdzM5B^Tdk}<@ypJMIJ4I-GDdV5m2f{ulg{CtS$7b-mC58ih-9S6tpcH5nelTO=!V%nH9}O3$7zM*5^%qgl9`DV^1&+LqYWZQFOsdsiE8 zIRp}OTH@?*IIP@FV=~RlZI3Nnh3v*Kd9H%;P*-QQad*tl_~QeL(h)MLA~w4v1whlx z_)uz-sp(o)k%hvV(7`6zv0#`?n(VaaDn4G-&Rau;=C|!|vH&XyYIZ|IrHHc(ny|5% zoGfE3NB~x^hP*qj_^VQnNbvQm-CRgO2!lOq^kd;K?7gb^9F14U8ey8{!j)DWDE6*u zm19X=nm)++!{F!a{jGS%QCrPw+DPqGXVSk={0;E4z}_CP{>$>@irv#KBxo7)wEqAD z6$BQ?@eD6dO6fc|;rr%}K?Sqfq-A`)ryIsHay>g&Z*OTGrIvLQ6n=iqk~1*cqpFag zj^Uf-Cjes^=Ymz;PeUggA3EK5OH=S1sjv8YShy31^T#>*SZ-dB^dQOQO5 zn6j?}cgwrb3c$V9W0o%_(jbGsVtaP0h9XdU7|xVsq4-zv6ZUJ?{4%o7;W-)CqQxM? zM&DmS^sfrj#-XO_i>K;{Qe`;lfZKn&U(m0K{2`#k{{uXN$4y0 z1I7OU5PVdh2rNHhkJ&p`mVGl#gY2Fl@q2uaJhQPf`6|Ej(b()o7hjTA8gx^aWixbzkWhbmkN9J`Y+%w{1RWmI&qugpY3m{TzLD(n&jqG`5!O!8)a; zun=JVj^QQ?o|)KAJ*&@e?ez;wlA5^0YynZ!=lm=3qs3qFPag<)Z&ioKAG3z5sQf(f z9jDE0d!)?E8xhfDi#r()bs&N`Ad2jMAAa4x5dIZw5qv-Wzw~(YNehT{{XrLQj{m~@7Tsvt+DzEJ%pQyBjx@QKfW z_f|R1!Si+%%wK7dOolm^gHGsPhvn~zJ`?z(!58nW_&!F6o?1L+IX6xQdlfB#e$je@gkU;vc~+55#dwe6Se$b*v{x zPju;pRrZeoy|tbZ%KW~grF@0^Rs06C_~r2GTRQ}fJ$FwAT|VX4lyTKF*K3XqeI=?{ z`19eri+N!>mYSyoIsM`%`zOD!Y<+7+%I`$+O{l!kCCi=~7U7io5!4#ysRbywNoaZ) zJXKsZSYl{O(u=yck^CrXo;vZz!oS)ZT=;dZT3l$eMSLQb$&(3z^A}P89G+|R3&EPb zlPhYPTyoEEb^?-vb138}BOGzmf&LZxm;V3-7yW`{`1hgz0K#?f?lrLSWrjS$N6B>v z>aGXz0ryYgUyQy56ZjtU{t%YDk+o}37JGY@*kWPF%f@k@IpkNu=Xeif)S(>@>Yoev zulq)nsnwn5y|h-pM6b7^N56l<8R2>Ff8ip6(UqczyGW9Nfq?4K6&1~RBf>r^_?xcH zqj*ZnOX(vGa_baj+hTP%1BD$=L63Y2^uGt{*7sUn#*^YGR^?@ndDyEXu;@S>aysPl zD@))Hi5uZZkFIPrsi%g*{4_E(yM^7sF^4@i=NpDN?OqitHm-iBjGb6Qe!@QT)fQ~L z1*OIDZo<+VV?C={BRTSdk28#_g)9jGk6)*1!T8PazgyBZ%fAv^vL(H$#;o87Se7TC z9Z4ss0G_z(UrqS?!?#l0F00{ti#TsB)>5|<%5#hiM1=BOZNM9UE^D3AJb$Nni&I@Q zTeO-vqbVHFT%DoWn+1?$jlDP+#w(^)wT}XwdacI1b+WTZjrarMUB0=c+UlB{$t|v+ z&1RBi)dbi81Z*i{k`*j~6S29*Ls!y&4g4{vcxy}5d^4_H6`t*4@*@o?NT-K-jGgIJc--NO1#97dD;@yRO^oP;GoPnEI;NKw>xaw%?+E(mab;(|l0p?N)Z* z+www!?g*`32xjw)0ag6MlM^4zcf@hLbs@kD$$0+&<36{g*;{Mg7Vx#@o~dPHBsZZL z#cIuPHpe6cDj1t#WjhEPxd(3*=oVVWm#<#xR+d)Q)*6f|meP4(@7Ri8_i8>;`5>3t z4tPB|^PdxZL(}|q@l5I=ZVsJq0*I|;ar?R1gj*a0$IFbUE7XOj_Dpcl=9-M@-9>(z zTK4qYZ$shlfY-mY!f9H*g{es{t79yWHsyccMjdwfep3;WK|ek|m>I9q%`e69_{dv$ zcG~J)BIxak_!0JLlHtB&IRTU)7~PY>IPF}(Z89$uY0G16JG5S8`jnOt$K=Njc1FdU zA0}Jp0CBYO&MV{Vom*AYHQh${M)K^P8c14j+mcxU<=h?PsRVT3ag0#rcT>Tt>y|oO zRek$w`gvIUhs7QYy3wSX!r|uCtl|$5XeJWoLb%75g$xMFlag_Y_|Er0)2#emHKm=0 z`naKF+^0AoXXTNYsmDwcp1)6_{2lQk!fiIv>sY#Li~FeB8NSUU86;pn@~-SKC9%mE z$3u$vAL1XyTdR$JJs07`1KN4)`a;z7d|@-^4J?v&5ks zoR>QftekXp*{vgmD zd`%2;O`%@9x?H^R6A;>5B&?)?C`@icjDIeltk06qW9Hsbfu)MPUo87{f7Zvy)}AG~ zw}#oFfg(skjqr~wupi&sfX9xc5q{1fmVy}!@7xQ=Ump=2j`!+#II)Yi?V zuCJqbbgq$TbqJDACH#dXFH)p4k58p^r^CJ}ZwbqLa!fX+Nc`z1T*#*%DuI#4JC4;+ zl#};ND?)`xa@K|XzpEdaA00GVz6*RuwueP{uB~rZYDJRsB&&c&u1V-IU(pZz6JPcp zpW#lEAIIN}a4eoN)uPR%yn{d4_Rr7{Nf>?O*l}N<-UIM&hVqzlM zr`d5NQc4K*3RSwW&H{Szn*NkLJ#pe4Tf>%Dn&rA_x6;C=BMIs#jh6r)m}MokKY$zY5YGN!r#Ms9AN(d#M0(GDk{g?1S6A! zpQ$zO%62~?bnKH&9Nbr56n;86tK7}mY-Y=xVS9@C z+vCT^O)KFag(Txvd`Qr~7x-R(5o>M+!HA>&Cw|HWejR*R z@$S3v3&aoNZxE7A!*LQGBa6{~pRH$Bs)F+!I9%quQ{rpy6Wz;keLGy--nW@?DeCTf z4@Dh^O7TAv___@tWP&n0$nrDjfBMz2tZScOvzqGE1u~B=HO_W{)Yrow7yMpq5W{UI zWWnF?4Ao2IX z?Kj0*My04)_|i#XyNgng7PVK6w|Pd~6<;m!w{~}q2>$8#5B>=GrZ1s{e{UZT{4e3w@b8w(jmyS3$8HB|=EPQ#ZZXovlqsf^ne%V#o$xpSI{0Fz15RYdv_l%Kz&X>O7XDHKN9pkNz-iY+S1z8`HzMDE6O}!X9TGw;Z1;MDE1u7q~ogI`AYFI|>RDi^s;vNGj@_BHcPh2zaD#7nvzrc^tMoN%@DM};in z)UDRy$aw($?W4Xr*MWeodX2%Jjv}3+lSinHNiQKG9Al>^7_U3H(0o6sYqDKx_p+#k zLI{Il6}daTjt^gI?V5j# zbr|oov1Fes7Qy-o^6vxqyTdxW%^siPD_ugyJ<%h>C}HNVGBeu)A5cYljkcQuqg>uH zvW^$u6<7(=L^#fN(L*cg78?CUa?XiSaGIgR3bMZ4$;hfHFO+m`i9O(k^bI`C2q> zfQ%kGAEjBa@UESHtEiNc*d8{V;1TEzTJd&>CDMd?Y>x897jNRnYU1{CP0y7Z-CEZ% zd?RZfqN3vH?ti<@b$Y*oAk>gw+n*}t1M)Xq73sm^zJK^huj|uGt9x^k$r^B!JB8yG2hy`DDqdQc8#ZcZ6nMy{U=_JRSmCBj=M0#E)MT*)!4{ZJJg5n zhQJ=2*KL2KSn9S59HWe9jtH+b)a>s;aINzv3V+?{TdGc!ER65W&Anevo(5UL1aP`Mv-|rbsTzBzh~KOlQ|eQ43fJpAznvJ@GE5t64*SzjN_$H zlHi8ey)j1V8;J-XPL-QGApTT2b`^%m%6WSHoc+DNJ=)K|;BSNFR+EO+rcgh2ycK~@ zW^N1PqTqKm^{?$!`#@U!4DdNRVH0cGplGZh{^=ijRv+C#`J4*;6`D)y+qrceQdgeh z<=RJrZv$L>J@BZ#)zyZ#s;=pG z9JDaMJAdAOt@sYr<^KQ!ya{XKT~+=dYqq{#k))N1&foYRRQr*E(1iQX{?-R-`g&<6 zyq&HtIc7Pirki~79Q0hMs9^}83h`f!-xc&vggytm@iwZ%axU0YJhPv=1$!2ye{ZJR zz17Qvjxb9%Kmf1j_x8E?^KtQe$5LsUT!;G~O=BD?Iy(-n{uQ~4c0OzIpNw^H9{9HJ z#k%~4N$#X9eY4OHU^uQuNE!DG{VEA%*y;!6R^Yh|L~`Iar8^RuGA`jNM!}2)wkU9_$TE6cmHH+de-R=ZnpVGZ9{{X@f zYhD{oL*=@ZZIt}k;B_PEUo%arI#fxYVNx#=&u0*;?84#Q&D(- z!kX5Hr$c|e?WR`CfZaj%`c?Z1ZW=KxO-RnoX=CY+8|&8^=uLLg&0@;zluB_PMtR^I zSBj+GD4S81PYzzgBuEvS-3es`XF1)|=qs+#UG?oIeG+SfZy8^d4te#geMiGy9PqR# z*p@YvkVwfnTw#ajQ3RvEs`^<;{DL30V48IRGqh#7Up9)>Rk!=;d#Ls@rM5?kMm2v2Uwt7~lihd*9MHQ{>tf-SO6C~zCywW(V&bo= zI%6lvXuBR;ckyMlJ6ncP<&%(+N5BuzmgA*yns1IgO`>eI(n})BIh5pX9^Qhve;9am zJU{;c6Dv5KH@gB`l*v2++ofb59$rd|4aK}{sez7u=;H&^itmOV4nE{=k0z!zYF4Q$ zKO@$)kBM6K{iR3Pk?Ni?@q8LxtD_)_ z^|uCC$6WUwfcC|B$??yL76*!1518X=k-ydj^v_&Ymx%7X1#kqqNVxOgmoLJv&}9C# z_b=?p`w-3Jsja+y@lZ=?p|DwPY@6kt=ye7E0J$0b$mxO2Z-&E1md-zc&5OiM&)t%@ zk;#6?e*pdoIP$ll>hWLNT1MAb?x`9s=4LomQP)0zSI6HK zzC7#xITHA1;r-Fmv>3)OH8uGi!Q*UM`=IpQ*ZZ~PKM+1EOXF)XsB4cVmEmB2aSuz2 z(mG{-yXd_Kdi0$G!Cn^dw}q{KGJex+bbky)>mHHe_=A0~P?+Ol`D^ntHyA<5&pSt6 zYPGGTO)1m9`x?Io{u06BPY&I9vs}4f3H&+Hp=ob*IayIj#d}rH ziZ##LQ^U6Y2Jn5P-Uaw0ZlPkgfSY|+#A>)1b{{-8l65RknJ*o=81U%cG5F7UW8zI z{{SJ*t^L?+bb2hBW`U*J=sq2^hfdS1kVS86Y=SsqLC`9X@@t^`Rju&}bb+(R2;#V9 zV%+DpPhbUn75hZ~*f!cCPvC!qvaHvP40jrWoQS((KYxDRt4XxCGFlx6?VIs8;b(+x zpTb%MF<$HPrWsQVmf!DV_`R#~qg1zfW^0)SS&1gQd%L@vo2R_Lk*1nf-0tTjlT_{? zS(Aa*y!p!SL!vTh^I>fqr)d~94E7}l%Ux9ZX-P7JrDw>m2g@M9skyvla^?~RS22-Y zL#zfqI@MpYhF&T28XW%smsVA_j2gJs+YWP8BtlN?Q@AEIq}nnnt6j>rkwvl^(+K#Nc-g{{kb$8rGaUTk=sQgfYo|C1yT>CXT>Q` zm~)EKj^(9bxa&~O)r}apDB4YZ9q`!N|J>{=92C8d(aV`hk8?jz+5`1kd%GZT)4 zA1r_2jiVQ1-+yLr*wf*U!0jI1-IhD8U9j@V_%EO7?O#uLyTUpqqh}?whB)La^7IFf zL0)6qJN`CATtIu+Ak4pFLS_w6a0d8ZA1ztJ%*E|q7;<%IM zT+%XkO!;fX9v;*59Vp&O8hnYCQIAUaW5)g{)wS(LTkG8X{{SzqL4lvZSK6N&X4CX5 zB+=~d3&`BEH_TLhe~_<-bui8hV-=p4E1}K7E>vzEKMJ~v+w&OBBvEZUPAc`d#JK~f zrB^VCiBya;sq2c-T_VzG)>&g#L)Rj-p|*lZ$_{_Rw!AlOE9vIi>30~(89m2p6z#B@ zKQq5-Z`kJZ#a<<~_>b`3L2WPx?U2X`R!}mfeJkM46WeO`H`f|=s)8$s*er?LAax&F z{*JC}RqV_xH(+h&(!UHs@_zLy?Cf{x`T!xuusj))zuTj!1MptmJsI`wM?=%~2+X&+wtL)!^p9eJ` ziTAVJU0kM%;k_U%y3VgZ5n99OKfNRLL>!atUQ6&-;T_M9P1lGud!*6)EusM~ud7YO zR?&6B{{VVM#t{c3`&a5e!7qY(pY21XXt&yhtIJ;k!;;KUNKbF!5Cf) z@Mpz8*^wpj-|Sndj}-Vx+_m?DH7STl`OAHdM$22j`vh!0oh$T9_I3TN{ty1skX&ef z6Pj%!#QJatQSknzqTf=qJq}_g=RkN8CRh1a!a8ZVvD5~orcBlsY5xE%V&`|WFGe25 zus$JtIrweyy6)dn@ky7$el@WF0FZBqG^aMxqkBp*k92&g=~o(^CPbi~$L$>2TXVnV z#}&-pY3C%yxZ|gK{NM0b?V<6H_HMk3!GE^pkM^I35f(d79cgeA5gix%LftZAY;~3} zQ`mK{)4c~*@YjtzJ!#_a2H!!c>6h^dt*)({NffcSLae+IT3OobW1Xh7X9VzVJ7gKf zW?a~$symNgO7x4(IUkZau72-L@(xBnr6eQG?sVY4aAa32s_4MPb|w~ff$ zHj~G-U|d;b6yP`c=~S3l`NzZ_4z$%_dt1mu0x`Q4@&}H70nKF1taxm-wbHiWV>rn4 z^ds2U=|$F@gzS)D*CVQEV;PKOP~e&>GjZ7bvpy%6O_Tc%$M*hTAr^2k&u^#D@n02w z)qk*-f$@$O@%O{Eg5OT($Ok*5m z<2y!2Y}fR2;~$5=1Al34E&Mg{6HbcTQBUHLa80!uTBG#rTxrffS;I@+dg4=beGE`?iPIRW#?|s;UH(A+S9IeTWg@Y<<*_cw`=E0A(X=I69ak@ zg#qdqf^d6cv#0R=&CEOR{J6f&wIW#VWOtQqz{3#5hU51_ji7oAR%|*Bo8rs+nRSgh zq`cH^0z_Dt&|nrt+B3U&!45i+fyt^oc}_FGH{g-cYWBzMce?h2a|2B-*_LhC+T-R` zcG}C!3}a~L7{z{M{8ae8;#~&r+ryev4`Hh5cQQ1$5W33J7m&Fd)bbU&V6d;!JsZQZ zc-qQKD509-&&pE-miF^VVlXCSMvvt!yY6lwc7u>O!THbjjQDY^L1Pz)(p^7QPMCKj9s|k5`LO`8UGA%hwyzZ*npl+MTX^HqdUgf3&+b z&Afx=g)$UKbN7^Rc&tAPd@hGW!r1BFTLmtwz}%qr_WDv>yPmw^iA_?BQjNF&03~Z2 zcZ6&;%}VtyukUVSg%~hbB}_uS@5DN{hI}h!r&!s`G|VNb`f0UIuW!)$BdK6V7++8dRDKCX83-WmcQ9|t$XCLjX{rM@r*DTBoogcN>bJA zYlo-oA6yT2QI92hYjLY>tAw-h0NizKZtl zLwNfJT1Ls~$3DWjEqlaz^`pfsak|Fm%YFQA3)AwhwoNkQ`#RfHDnObo_+UQoOpo*0 zpDve2Lae!*ThyN3c`q&g!E1xQP2d#)vIqCE(E4=Zy*ohBY;R<`lf-FpX};D;R#gh} z8RACgF3W{x`{SGrgE+4?((TsQQ4rnqI2?11=cl&>aBJK2dtDn_5@``xy4)8-8eB^x zay(<5iH;8B;F6qheXA#-yq%+SE5lcoej1fl(&`^FHxNr4cPhy)@8e98tT;Tc$~t!y z^roNUp9%P)!YxBlxtB)NB)demh&rX5kVxlge=u{g!*9%AcWEp#0QrLd0Kk4K(QU1D zXKRR|hj<_c1}BM`nMMZ~7+^;nlg15q8s3?6u6!(?RgN30*t9VMTubI|M2_H*5i7Ul zUHM(P0Jbo5nrU`8v2^1!n|9XM{4M$yymj!FZx#55Q1Ok{qiyEI;nHb0MH0&6J7mHH z91W}6q0TaE%1wr$acF!)Hl~dhnX*E#fXcBL*rPrD{VOHR`^Xa z-rh)Kj^Y8eum<#wJmVibGiPzzoDU+@u42&qFLQ5j*3%oEpl)RnAzw@qeMe!88ZFSQ zG{1Fr{#LfyeukEj;{Ec7WV!NSW;^_`hGjf*2lTCP27DIygYhE5`fVMjxv;aC+u7^~ zb=;W^v=BZ}7wL?Q;8kzg+xC9(H^%<}82zOdLH5Oc4pWlKr|+h;=zdGb{v%Mp$>sP) zy``TwWV>nA{s{SF;GgVQ;V*(;6pxR5H56t$>o9IS^(J_(4mWvcl>qnYUvyvJ=^74? zb+7n>>N_1bPK3{Mb1B-9M1zt~@*V42L-4(xn`0h{qajFR2WWX2@H&!N$vFC&{0IHG z{{U)BzuK?Hx4*Mr!Oc~)&xc+aCSMin>O!QZJ_;%B0pSOJzLoZnjA=&a^By-DQx#D; zYW7};`N#II{iLoxZNG?jx}Dt8>7NID8zG1Y{#Vqd9U@QfNg*FMI8n!H{Vx52e`IU< zuPpxnY5xF?x^#BBgpjKEV?_h^T4k5~vy*N(az5^V-t@vL8oRZyZ zmhm>dqQE~$c!#|5A6x@zg75^;)yjW^Bs&CTPtJ67qb!l zt5U3YCd-NhiHs< zC5|}yX1v+mzGihs$ym_w71o_Cu!3n?5`~rU2LKG@1KgfImHG9rS`Qa^v3xIQD312} zZz|nir$Tohpc%z`-;M9@Z6kOKva}nFjDNg4eih@t4!#fgyTv{=(Y_(h3R?|E2Sp9G zSByvl72x1@>5589D(r_Fw2}Qgf59#E&jt7^;EZ+_*3sN}fi6GN?tt?cy5c>`pk^i7 zqvaf9sWtnHqjsQ0S0ZpOl&jMXKURz;s!+@vKzH+^3PW-zby*g?s5&RhN7Pa7po?ST> z*G+5r?j2RnGEVH$5*wWEKQ?=wE9}iXT(GpaYpLc)xck}Tiutcy@b`|juMNxLtqS1B z=I-3+#y0te56mz}u@&Q%z6AJ};5bdbzGk`7*@1TrxI(_;xE1I9YFKo+VA}84>h?;e z@<{t9#GWyaSus7EmjmSovCsbis;}7T_IK#Enxve}r9^nnM|$$V2R?y&YJ%G8H;N?; z8*uxhztX!e2x*ocAl2?XZ)+{Z!dZb75vuKsjB>-*C-CC7mL9i`0w0BC5`0^4zu z)1azaLAi>}bq^8Bu%4xU?gwo1U2dAj=0fA=Uf)jD)rG*}=%rGeZ2thRhBdJ@DrqAQ zT|G71wY8ZD9GrCGqt$*IX!2S48%T|a+?_@}05#IyUqhuxlDthC@VmaW%UyDLZW0>2FlOQcQm$aa`A4h{}-GuUFi&tB8?Yq`Ap`&j0bjf2bp;d7jQD;|Fh zXqH-pdZoN#8G_*XSWv+8gMrO?AKBF@;ytA)B_7u3cYR_qr^_{{t#H;CLR8}%bl_J% zs%!Hxc^TD#Q7EVv(&Ui~SMtlg3BAdMa(~a`L2Q=-&~7$hs3C zAxJAHQS`50wU17f7U2iXK+5*~E698|;afiq+0U$MEU)tu@{H|KU2W%xbjY;zxzpc! zkP7z4;ZBs=U%UxLeUn-kG|N3s=g1;G?Uoz>TO8L+lUvOe<;Ti6#e7%e-}qSGBJtdo zUK6 zInn%Kpsm%b?4C2_%M)i0=%}N+ zF~9=7CgN2J(lOYoGEYH)Uxc5v_s915(c@3}M?5zs?+$3P5eWz7xwsrOzMEH-PsH~X zwi65OY9-UD6>6wP=gz+qekJ&m;{O1RuRM9;a#DMfwkUIzw}F1-YwSCo=bGT8WdoYH zRP`(RSCM>U_^IK~gMI?P@#l%$EH^&b;>k5;|--Q%;n#2zE!)-I;f)SLi=v2VJ#uf6^ocny9Vctpc{2#Z_Q z6<+Tj^2RaBLOz{a-m`xExq1uXS9_gFvC=f1HrK;G9#1Pz)8i3B4_5VN_hZSXbRSBY zR|An=8}a`D;^vR=Z@^c6CBI?j1S<-6c@GK&bfs+(f$^8^;rm1X0KzY%ABH{_xZ8Vr zzv(Q!V~F*kYrEN>4+um-vgyKeAH~@U!_49Ix_6mKBDRiN%+kLZ7pUSz~ zqj2PL>-DZ;M${s>7J8bMagH(A^sb8T4MRb{l3ROl@j8#XaCr8u8^dj_!4#3Ki|1|m zq9bs}eWsqmYUVwPM`v;+x5yae*G1w7lU?xQi#HNY0U=PSEZ>)ID+cFI)U<2zE~iN% zWZb-Azbep%;z&1J+{NXVpaooT0Q42X74f_t zAePqFKlB)#wh}&Woj9)fZy^(^#n3N*TGRVv7dMOLxXIvlIi`UrM@bUSzZ~uO)6*xytMJ8i>)bC0P`O-5*vRvO8A_)|WbE z+gfS}Vs+}QFhzQWp}dH+a6+vk6N8ME9^RF5{{B5i!tY4bt!0|w6AID=9!@ZN<2-$9 zx)mIBwuhHek2k#8<*Q*W$RaW}brfm(Hm|AbFzEVZ4{>umixhE0yP4Uz00CZ)@XNq{ zCGpRR6T$u%uzShwI_(@Hj-`)9QR`o|pR?EO(cv$Gw!>SRG`!d5Kqb3j{-|S>i$8RF zo~P->b;Dgc9BwMfo;~|8e#5$7g52V#@>wKYcr|7*Xk-{VU{87;1XIh&9U%XX0J4zSQmt{{Un7V%+2* zJsMy+E%p5=CA-(B@p@WlO1>lU)Z?6E?3!X8a({$v{{Z!KU6+Lt`@#Mzy6`W-(I1LF zB3ytqeR}8Hpo_lxO8vMs!;{GOF9&=%@df9Kf8lR@W4m2<#qPUc+PJdPVn6c6 z&OU=5#VH%x+P1zEd==5Y8AWxfNp&W#;vG}w)U}JO8=Hpy=*QfNdMs-luQ59xAh`rX1>+dwx-B!>#1R#%QZf8lS2z8#aq zo+`Opcp@JtKY7n(QR`ohA09tw&lP-U)J>|EvC(5cy0ugH@%&HfeYy%$s-i+YNA`;S ztS@|O(|9-Gt395O2}y1`OL33nwed_a-P}yJ_Q~c* z&A5&$Hva%mSr;Hyym$+^bf(KBWN|}7i#TbthJ5g9t65y6K~Z{pS95@>O>_yT+Y4)3 zs762-eE$G@{{XINN!(=k13P;wMz_)ao)A2iKA@e zrDo3GDa~&kgtRl{Z=ZLiMDsHYQ(d-_J5&MG@l=r)Bqu#-Q}0!AGIT#ml^k^x>t5CHSN3A~yYa^5Kj9S9oozBV zT}Y*41KZZWb^icoU-%`D!+(d*X|McREv(&TWC+9eUroNBTJ!NZ+B9^N*Rk75l-|oj z@kikA?9=g^<8{)fg0y*_)NI_2bU z$cQ>-zj1se;a`U~wpQL1@Ku7_MFN14K)5^oG3+a!*KO{89^J`yw(;n#C_81D*!@Rv zJ9n=G7mb}&Xz3HSlvH=z@ts>i&^`}bX#N}1pbewjVXkf1aSLrZ9SI;dGxe`1@qPaQ zg?wA1FN>flV3IHkZnzARkKyOI$EF2+L*p-on#YTDZ|%#Og~ZcD5tTZXI0FYB!^wfAIA;K}pQ|re=TprE*jTDzt&3pl;>-KuYRyw7_$!&3Rl12-1qxiF5 zPWaEq`fh^{_I{W?(>Blbo4!&10N1VqL(^u5R+n0e0UKGPC>hG`VU55aL7pql{82G^ znb`BsN>Obq14+A{b>fTrol5EMt{Ag1$tJw6=FOgNE5FoE;&}G1eB1ACl+=XRLz_)J zWDq`YO0v3U)4nh(qgxy{Gs)?ik~>EdeBXD{v{EOKr=nUpyD2F7NUr+I&0_xmSHU8& z?Jd84n>Yvjis&zIt#peibsa`UiU?D4vETqJNZw5(g(RZ2Ie#4Zt3&Vyh3`C5;urGm z?Uc8>HaJo4Yw*LxUMJK%cjFuH7V5l7cOhWCkq6yT^cC(uw7joDrj26Q zi3j|&Z||S!Upwir*WBfKch#ZIQ{sPzk@Gm*QmZ~#d?@WSq|P(YSJfW}z7>DMarU1Q z_>$>#ZwhFkWYu*^KIAFc6F+`e#t{eCpszppNASYa#MYNyGx5FX(L5)l^M9zeI^!ZJ zm-yvwz;x<(uifw2H}*-_{yE-wU&J0Q@VlpnV}G=GGr(t$w1I*BtE{jG3CL_dI}BrL zH*Hz+m_as0`!{~er}l}5!@eo_g`(YP{{RbZBoE;42By`%@hs{$3%kykhXZ+Lz}hZadT!dXKNCUY^-=(K)A$`;54VvKm)V8)ASz$nKVBS>PxNa z_ki1K+R_55>7V%Xv0jl{MnDH|E=MHx>NIZ%S~MZzO~F1?UdjijrE0Y^_>5UL*2*DWWF|JcrQtFZFM{d$-&EXr_N={c(wkY z9lp^b?ccyP1bR$zE^B+nUlBeN#H#~q7Lk?tuw)%;gS7b7@Pa7}GwZfr_xU*eX{N)J z(74ctnU$>h^rSJEu8>I^a6uebS!v_nhdR1#(=}^XVm?x?2&>5YW$K91SlN&FPg-hY zp>R@Sr)G4!jL2D_rT!&y(uM!)P6)PJ&guP5;D!_OG}CDv#7o%;{#V!;{Ax}Ssf z9RC1i+>_S{_pPKoo<0w7MST+vl<}}u7#QNJ>Ruj!=NFc$Y_CH8?(QSCll|fMSbGtQ zsMO`wB5R{PuizK$pYY$~ZmK>Uc%t5426*}?^FN2RLK0`&)$L^aml1k2e1Z3sdRNvL zmW}~d;=e2WYoK@s;$lJJ?~a;mG2KE(*mZagz>+nfvj65?PpGa^z)-n`1_52zJN=&sB_J9Mp2fVKHubX&W*M|LT_zBJYy%N zV##dD_xT=%m%zR}_%GJWsyixyF78$ur4f>6p1=lcP*pDGQ9mODWeQYVTvv*9&>S}LnM1+ z`AM!LSN+BMGv^~` z%2((|O7Lr675J~={_hX`K!f{3OXy@HBzEO{f4sH&t9_)Q1gBc^FBg0|(|kb#M?4Db zcErgKti;;K>@?t7OKUqaStt zqP$mkbO3RH3F5xN@f&F#3Gh7L zD7>5-8778lB8~8ca@dV=jDg5cZk6W08GqoN+K+*}UcMBabyU;eAI1>_Bt;s8p_FuY z3{+Z~pma8-sED(QLq1v)iqIh^6C6#>CnF{-JYXRbg{= zg^WO*Iq#ZLU43c;yu4-<7(2<@py{!4^c`fFfi^=cTL3C1P^%mu4a~mVg^odON>uxG z;jh;y61O2ylGrn)vOQ+0+&hppwfG^+W-~U1lZ9u|Izp$UYy9zZ9O2H4yr0LoUBxnR z;+dO$ZB}wsH(W`e;FEZ2wvoUDe9s!QBAesSRaMMOJr+j`i>2Y&6BSFDc)?A_`(H%I zpcF>@h0k$j0M(dYD**u{?GB;HV%<3gFOSu5M321=IW?rPyddRqxFHSUKQ_oO{ztUs>POv2P90-7_t+jYKs~=rBp7tJt|rzR zPTcV#kaQ8^V~mc~`c1KFrfDeqO-v;AiL3IZBDE@*kOn5h)2Jbtzz4|HlH z+MKqYZ0|tNR`}$Nd})5ovnKQ}nMNr1(kvDoK#eqg?>SuovZfFcv^D34q}{BEbfcBZ z5Wsr`s>D$~Z$y!=&{KovkukKIX#bbKg6sk(t6FOg%!o(DI+zLKyfchph0nX>nMJy+ ziZj^6+P?OUZ5Vbh?fGf_*J|mY1M5gzvRActJ&QG692=667cZxxca;H=`+go=8hq3^ zmREkvaKKJAE=3NCVG+*fsYBL0HXU)HvAXbHw@qw#s*1R~Efk+V!Z%fPo%I7cBwCar zKYc5|Hh2qg3=c*+IYy0x7QoHb7*sF3voA zOQEzLa<*R7*1u6z*1ULDr%fK?27$0dKf12OZ}Zj(X4tkH=iOCe44FBUpTTuk#x=LL z%q3oC^GoEzw}Y%M;DtHKdbbk=F{Y1h_%$@C`7n{!G^LM#x zYAJTwUKFKvcNz8Y@fn$y95{&-p4BWBuUMl10GxSfFCI`|m)TV>50C!&Rp;LlL)=he zgRkeQq1$$;%t-$L8xj0#AUh6#{f`K9+}GW0IBsSPa&r|&f1Bub_G|I%Bl+UBQ58su zjp)P6Va@^{Vf{-oE6ldz7p&rb^cO!(uj+axq95RAbe3=<6gbijz>=?f)-=R%&m2n+ z1bBp|b0Ji7Uc?6(HGSXKEpIB-Skc(QdGvi3FBhrm9Jw+1LcpG^6~BsaS$Fvhq=1rO z18mn}RD3YLmmtNg0mTC7C}Q~nHQ83gYH#Ni&pqJ$fPcO=S@8@$wzhO|(LD{bbD>^> z0 zW8TgHE;w&9g!Mu64pq5WO)t&Nt;xfVI2o!~)vFHvtC|bR+XP8iywODt-FT9wG9$#& z={gK~Wixwz|7pDD7Q9%Nd`!P_hd|?mFT_}46E|2Bf>)_plXg(5|yexPKs_L|=U zU|&1zVy6wJVM$6T1yE*&$Rj$HAJphj3cQKcpwJXm8aU@V0eP4oT*m1XgH8!P!45Yw z_<3HTeX;$bdhUbdf)}Z?sa1k+)tD9|BSq<`I>|;FEp7$e2uW+l)6I+C+d?FoA)B}R z`2JYS`zDLz0p_bbLXpgEHSjpmv_7?f{pro8z$u#LhlIBv4mB(RmX!O2caW6H)0>rP+=_8@0CU6jB9>x}guR9+*tRc7}y3;o3 z`%*CZo4P-Rt-c|AAH)$K=@;LJ?(AxSQq+|WLMmRhU9qZDBcm{vhoLugP@~&Fidt-k zNje3eX;MVoIeD}gIlqq5*)^QN_qzYCp>;(TYW`Yl1!#0hd%$d;{mF1^SGU;_85`C* zhZo`5An&?ib>=U zCIKI}^TPg0v~u2PKrJyYu{#+`3@hjN85q1Dl}rDUh1k00Cvy;0v3l1NxJfB^xxk_3 z+-@4>WVfPn?^!ODnim}{FCM5E5>Z$)(cix6MXmrYuSti zt|+!w78Lr{&Kthnl#NUq?ble)dIBw04+44D}RdJ#X?BXj?Y(1+6s zR7hmE_Lehu&^=V=rqaQ-tHD;)WkGq)P%-ZA0$eM=XmtfpsmwFwB(fC6TSANb# zo^3BggtzNc2(1CBjuI;iVDcNgVXe5e2)gF4HRWH`mjfWbtr{{mw?47@V@wRSI(*d4 zB?SjIM)baS|C$&E~GZGDz3x zKDJjXHAX1EX%Nrr=Q!W>?Um_m5FK$A{zz8ox4GC>dE9l)edg$^$Un)CT2_8{wn@Lq zE{NVYUieW#CkS-?Z~~NfFMi%N{9`~SZ)kUlOM;=@*4Y@Jk z={&`7k_E+NSBjm%ztm5c|EmLYYZHq<5bO0|83aSqq}O`(S#K9tJxg?-jJ=c9)$^9d zDHIXfkVLWtbh`l(*1$qbK1j9X6ksJjq*iiR)uD8G^61+@l)RteL_w0_)0XP&R)_JV z^_|)BJk}TYuL0UtpIlPZN!;|SNPkv*uH{aYT{=k|G4NJ;*eiIrbu=nKP_q^_HtVy| zBAwz6zxACgFGcq*h68awZhByDB)lhOvE~`EYQK#90V09RxwY?hrS+k7Xj{a!*C7le zks%Qt@?ejNGrwQ{ns?f0Ygv>u{~^mdRgh;z%+1P6(`jd^3g=HvZLJ=Hr@&QT))E57 zU$yrg{{wo*eb>g%6A}E1gGoC+pBTz~9E%rpep?Pc4PVjeM|B#ofhGwuwa^&hNdx=7U)$vQYPrN{Mj&JcJx#a(N{-Gi+%6B30z^k2{)$?`y zJ*)*|v{VcrOquDKlKcR4X(((rQH1)ba>eC)!xZid!+-KF6YMQ$;9m{PX)bKt24to} zs9>qMH(_PSHJ>%&hQ<@@cg^E#+myKTV|#fkqekY!tD==fIfAs|PgU$o7t-C|Z9wI7 z&`$!FSzTJ}d*yJPnO^OL=arYh6@zS^{xC@8K1{Kr4vrWai_MWsQc`{)naRUP?EO)8 z1y6r*iNf5dQ}T4;_|Uc3rgYd8Gxc&T;q%!9$5jt%4jdTK*{r>YTl$Y^NV23tXgKC{ z%33X@8lq<=JrNmoKXTJQzg@l?QXIs7ao8Kbd+KM{ z@SSflp!^q0+l5KMK}wr$3ZoPCf0K9_q<55z^exCjjY^e1Eh$uV7@SKl0-jQbNfN zoZ#L)WIp9KEi7FcYL%wOtHwI_ToV+m$PX+Hs3XobKSZ-;QC9u_l7#S?n__~$Va9Uj{LqoFu<^ziD)ZCf zW7kM}QF7|rg|UW0f8MXkG}zpYqoUMR=6CiFnHpR0OseO23_`7vah>FFWfKs+IMWa% zwI%_Y5X!1Cr->5Q4LtNk?XTPq9K*V(Or8nJE787e&~f~X7>ov{9|astgp;i5Co4An zW8eI0ZJmS#q073~NY5n4TcwM%e%w(S`j4n0J;Ix_;aB{9|Lj`gCki2x4X@Zbx_+Bh zRA%2@mV3)-F~*t2Pp$8`of`>e7RG^g_LLts-cUuc8{QZ>%fPA_+^z#t*7$zsk4Aqv zONazBx;*V5Y?(mhUi^Wfbxce>=i!B>U)C9yQ6koT%?M-OkU`S5hk2ZapT`*VckZ2P zFmAb$Y`9IOPDl#z5GL`c4KSVdDl3|cNGxSk`20mHw5`jpis`TprA&_X-vGw@d6hHH zf{?%n!w$?rVQ({Q%sKundy?lkH~luuO-{R@f-=6BY0Bu6O!o-Ge?)zT$vD~DT%SEl z_80FAF=oGgj|%Wy4c3>rw;heyP4Jd>HmCwRq3f0yNCDv=H#IF;MJVK$P_HJbE7ml) zI~u*5ILoEpGrZ=BaAM2O{zZxmp(l+jl_etE{pe*;Fk2NJvK__fV#4Cur!@e7hV8|y zbpalndH9qjy7?X4#JmS8jy$l`swk89|6fq?EgP_25eA&uT&5-Fbx$z9XazRY(;j4$i4M*9 z;qGeB^Wq@5PatZb`bxcT)R_LJZvS9|{8pw@L!~--DE`8HD84+uVY0R8{$ioy?|7NU zNz3PVglJBgtg(K2P9%&U!y@AdCW#~cPMRss$>tT)JujWI?8J&cz|IaI<&-WSY@rE= zRa5E`aX5x{(ZSpw!nX8@3Z@RgmU1Zl2sG=o|8n;#_ly5%pfJPldZkKw&zViW|~yfPvw#THFY zcCm`T%H$dha&=z`^EGhfZBb&T5X!df_Yy!huNmkMCal&u#nk!U%=D z{}FYYtPNK=_G`Rkx~d|O#mVsTttQ15pJ}ciWNy_|b6>c+_d(cEe5eJ??^ua@O zsqs$g2p2lK+hHo(#O=+)@0HykTLy(*T#6n`ctKDKS4C|$L7RP`)A1X%RoCoq0fIrj!0DAz&U?5Xe)tWE$K>#T%$c_I)mTcW0e0|E&A5_ zWv$&zGZ{XZAl?G8&-w~W2|^XNdo!-bYTMKE4>jj-z$dRdyp3 zAkt<96}ypvyhNg6mFdxK-|g**!&Hd$f=%1yCZ0~#0uRFy9=}R|+lm3JjFc`YHG}Tu z<%=a%!>IfX%`Z&>aZb<{xKLpOk;2ohC;r}?zv29zO)056=KLlHRp{KH!r9s}fQeS# zLt&e5&Aw`Z*1<#Uq4Di0M}d4(6*x0Hk99DOx`-2e>y$G+($L&v?Dd^knyEtbMoeGa z`ilOJ&UkHqs_My*vZ$sKsd87V$Oa&$)`fg`|AIF64A*z^Vf%qc(ERb(wmCYfn33HP zuReKMkYT8tg?`NrNO+a2Rnx2!Iq{l*soe%)J%9Zk&)z!mZLIjfiD4W3P4wD?p%{v5IKKrc)}B#VaN&L6lM#+;>mp#gmMJEdRT zxl7S5Ws|MhIzDAzE~;mbed5Td)5N@!lhw)J$oAA!hnZZ2M5rkTDXf`U|V0 zon}7wPsiWbND$R0pzZp-ApgG4c$BtS+-iTo_HBG{(LkBwW1-eZe4o5Ii0%iz$4`K3 zi;6M)*B}ar-r|}@Eb61I1kKtW%De$Fwg@oC)|3tu?md>OkE3Qrh;#l zqGWrd4-(SH*oUsjKw*Rxet1q!jRm!HT%}&$qhfgei+~CKy3iSNe_x0FVIRqChc{t% z30NRO;AJsKF{KXIZ)&imvrm;1erbypLe$52WqgW{GQ;k2*sNz!3}olT`3=eojcQ9~ z&Yw~*yuA2?0U33E$J7K|7)geY)Zu{#ouifH9Zo4%XI(>zR}~>!8E! z0HLEye^D4Mwr+_r@M+WCf5!8GqzSD~y}2iDNo#NM{mSgH$Fl7aWroGwh7$e{lOJ5<0{MVieDQcJWhXd0!hfea`4Hycl_s zh(xR))GHB&EbPI^tAg1DA=J;<7D7}SCJ$>s;25ltc88Z21Xr@GRf!|N_Oq@|lrsLi zd$nA`O2=cU5t^iXE7K4Yc4qkKRm+yW$@}Msl4l(N!VyyhD*`_pKwJ8rxkQ|FM2SrC zKc0S|qEVIuW0!VIaO{CQV+G4)VHlUcYzhqJfUU7Mf>*YP<}B0 zu;hBu{*~5R|LrYy?>{0En!p)$`5YYpk(n8R*ZsOYz~&qE_O;PJyZDavU%B-pJ!WaJ zN7!RMa$aL(@*sFX1)lcBlTyOyrH7+G%h4Mo=_pANtgB)X3+$eweJCvDXr}T*@klY@ z13$gvZj9$)9HrwXsMo@9b9e@-1{1T&%^~cheN8~DF<$(XT&--mp(A4+@3o{#_;ZAe zN59vt70>g=$bv6FCmjMAv08S;59-^po&=u+Im&&Mv#8_KYi_Q3(Y*=lU85EUa7Si>LUxkK&dmA?hY(p0;&u>xNX**BR|a@F9;Auwr+Fyi%Ojuqc6K1`;%mkSC?Y# z?(JNYWq>1A+2ZF=c(z%6tz@S7*A$y4#peZSxI6wNlZfMlP5@T+bTc>fl_7wsRmrJh zsgnL*oizo)n(ny`R0Vgys^@SFp%>D(#hfu}E{{}^ZC0JV@qfA|e&izKb*+_`qX`ej znt7LgI!bhY^Ebc;T|RzdI@F{qHXbi#L8g^YWtedOyS8<_E?cg4VN&4-?^Vc?N2+HC zl6`?^bQo|q1b0mBJS(X#>!dXA`zPVr280iRz)$0Dlm8=HFU_rsIL!BOEdDHixHuQ_ zlLKZ8Zc5?^t)?CHw~oel13w&N1p%GF#hW(`=+%60Ns-4+JsD0*_q~B<16V4csW*(H z!}@F%fvis~fy9Fe4_m!Tm?n}IN6o5o-@?=Iaei)tmW!^eoC#vm{j%NG(bxIN2$WhL zB-9`RIo6j+jeXf_I_WE_A>llAyi{Yd{N%wH?j#^77Vte*>?+iJyxh6YSWyiblUmg# zaMSV|w5#z#Db8UT(%sE2Jqbp_Q4fn$xn#&5S<*dN)t7$^434lqFzYI-D=ftsoiCk8 zWYGfOEOj+h-#sPDR^5jY0xm|XO?At*>;7ce?(Q4`Eb2zAi+g-4cS9z4bXJGY=3U8x z+J8j4r_%R!NnU+FI)l1_t*{!KBHbOac=j|?tWRJ@_#|fr1=CJF>_~^!YSU_0rJ~jW z;|Bx#oLf_Wn*B%eZBW@|u&k|l*qbJ|yU=OK&coJQXd4K%@uPUxKK55R=X;cF%!=GN zTDIKYvr$+T(r$PLe+3y&PMod&R1;fo%~bk2lU=6t^FuK{HzqlFRv-tIc|uXumKM<3pfHzrPA8de-m^Cy=$#4{f(?E-uG3ccLcNw*L>a3>k%MSW`vCo_s<3d2I z-5trd)Voxa7=KDF|0k=t0(eM;vZL2Y^NGg4&_(jM1~AXfSm%T2tvWX48Q`RFGQTXz z6GK>UCUEiY`t~Ma*LtjY$BRO*1NYUbMydw>`!iM#e^LVH0Weu788tQhdFq57?H?Y@ ziBJlIqF&Sc=}v_uEQvx*zHi!}BFp$hM6lKiLdr9CUu^%}PJ7t85^-$KfQl=;Eg8|E zIH*^}lD+AdcVkMc{i!}{RrGlCAF!IH(T7*Mbe=Ni{69vB;Z;D2LOv$xg& zkEiXLm8XpT(^LOUts!~P=<}=r|6tQ#@Brs_KQG7lZ|$L~F|blTv%Fh%nhwhOI#6Tx zefu4>U2x#sI*euidGl=MW0RL+RI{aP63y!L8Un#du*iQ+C|V>>#fB!~5N^MHexGx= zHp0#RDS3-uEEPES`#EZ{LkR`@LLDh>#K>rxS}1kMn5nib-Y6hmDJwThJ@^1QVu=mA zftX_1g(P|aWcOUYI2p>7i!fJGh|qfTI*+R`y>*M9X?2}U))rHItq%F!^)AI=4bSX- z*3o}`YB5zQ8Nj~))hKXyWM<)PW@6Kj1YGIzAujCH5AcrHD73{U_hkqN7lS;09mq&a zXrcso7pxy`QxAn8QWUqujm9PjBwi;x=V!n}r&b!IT}u_@$XS@HF2n#o+EYr*-`Q`X zoqfe)$izV;2EN<930U;FsKS)53Pkr2m<>)Zw(l;}FOep_tW*7l0;$e6Ba;pM3VdlX zBqVR#-`51lS)84=dR|G)DGS?Q03KUHT+Ve$gn2&pUfx%2+}TzmBN_i-#EfpZV8tF| z5{yIt!YD*6FPQBX+}u@l{68a%KHUpFBr&~ByUp%cPhKRTH|#3$6EX*n%Y2JLRa^?& z>W{Tx%}R84kE3rJvHs|m&NcE@Ig)TFN%6~~1(lzIuD|sV9{T|YYeVjXwUZYKy0iH5 zKv3TkkD&6cKc2@1WgZp1MNc@1ZXjRPhZ&&MUcY8~)!UDlwmF7Y7@53ae& zYk_QKhF;twGTy#xDBBx-h~33o?B{lYme3oso{aq4WR(f)e~t(BNY^%KovdJh0EQ9S z&@~o~2VyhboZNhGYY4<3UKC4TdEjSHN8Ua?YIM0{*JjCHpFnYIGZv0X-Z%s^CX#k-2D?tTQ_%wJ1t6it>=QVMUEual;B(UvAD5bLmRB1(;T*y4gYKM{Y%<;`@Vrb{N0_{B z1k%imbn~PzoI94ZlZi}}L^q^Ev12_Ym*QMWVx1u*N4V}rxbO+PvHBZ0Q(QFagNoce zc53b0hH=Rj$i!13Xx;Wq>Au350^ihD)RsIr7!DOGQSNwZKWWetsjD5jY~aE(c-k`& zN%E-p$HJ|^?yoZEW6D12JcI)#TSK>|ZrY{J>=PCD2jwgxHMd3+R$dR{!x684=3!S^ zNTX%+eSdhkr>vqvo|RY=^j^}k-<>n;Nxpo@a74FV5JgnwkZ0p3Q%v$b!T-~!%dH8Z~ zu`)`??!`87;?iWZQkw8T=uW)Zo`l1t2&Cfcs(0)M6iKjoS`R6T_Ysy{k8GrbvWo}5 z^@R$8%5rtwCFh+S`fi-YEQ??h*oMV+JZ^otzD8qwAhs8>`Pz4KS-Yeu@oK4M+DDT-Pa0m;_5sTQ637zVtBw6?|#HHX)eVyL`1Zgitbj_MN zw+UB$4z@gS$xhfg* z6>pQ!ANjSk-#qR2hs*p$t_ZWa5eX&}8K1b(Lf*1MSx}cw6P(0wGA}KbV%5+2n{%@D z<|2R~lX?j$eh%bmy0I83S2j=1ss66ulSL%%Hu^x~JtoOu^J}lh1pHY*i06u_#^e<6 z!lpW)$iQnUdwvhAzgH;yY?#-bU^VCM(Nj!RV;?vEy*WoeZqEX{aLQCV@jVd@k1!J*3S1Ngo(S6Dr84Pi2tMt1*Fl<`se9#K+ z!9(W+Cz9v}nZllIdEx{Ls_lXRVv3z4D|^OiMp6Er?8|I_gA)C$*;C3YhRa_R51z3x zWGfYppNijM{tMTd#E;Idf()!07HcYxD{SkQT*CP!;)|mrbV6o7&$s*R0-QV>CLZQPT5+R;$ug@8T86 z5a^49QVj@ZUpw=UG>U26r=0dX%rkB5On7L+&c{|j(V+kh>2-D*)0V4wL? z-5Jbz(;nXPEOz0K*@ymrB?_s#4h0@|+rLFzo>j*TdnCL*E7MGkbzy?Q)zC5#!Z?!+ zv#|QAs@MU+h^#xi3OSu*cvzqa4uqlZ>tHvBteLOS@m?AlA_^EbfBi?)8hGk|d*A;U zWrw*w-pGv{3u={f*9|Thx|s#Fc)1c}k+HkW{$y8-4j6*t1K08J@b=2tcRqUVlCvtC zOM!74o&<9QFOqey%XIf8^|p@qzh_s^CB8+C^d0|%7vMP{Ybsb5q{1;XB*!*nv`>Qb zv0Ij>-jN>}j(P@aR)K?JgZ(izeI4Aff&&Fhz?JuG)ZSPBbD2s>hNN?PA;FcJ_|FjhvuwNj?ci$Nkgn(8r&em|p14jTX*s zk{}ci@0h0tHz&UK#`xGNQV+JRS4GVDZ;(1?N#qg)5X7>}ZbEWpsI`!KRK!`;YBYRWvv#v)l+HArOw65pX&S z8m-o<@sUNm{BH^`>*So&CE>$&<#sjuevWo|!&W^;iCP;~ivqr?OaIitv+07X+2vR0 z=S;cmD=o{(cC#e5o0haOwIHb#;Uu0ouS|a39y2(gNZ~H?TBxz^*Iz0R@>shQ@U zyrqvjl!ttEKlvP7_OFwN@bN#s+`3cR5}Y_OOx$(TKBI>nE#1UrxuyX(_P-4LwHj_V zsw!ow(OVpy#MSfTf2G4GGDua!%#!OGE8|yfavlqP>qtWYr1XqGG&j>|zQ=?p@$rP}(?su!&vnhL9Sa?$BpZX0%0q5S!LHAR|d@VX8=LUE4i zanIsc@S}FF{O(UwH~tThh3wUSA`N~=fp-E2(Yp;3xR=Phg=iB37#5WpB2x&P(0N$<ILIN{GtmDPAMP4bz?cmVTgu>_7Wf21$H&Fl(=Jb_6iVZX0eO_{v z0&C0Ri>Y>;7%VO=?r8JXK)@Gs4KaUV_yseXZriG6abMotqEZRyTjSiav{SC~c4>!uKqr>QudyVI(qZ{W9 z+%k3#h2MOIIjDDg693WDJv4GbvdQb9^aTX`8K*0Ym_lZib!1c`kiHbIO` z1e;d$9KfmpnKZa|ox3Y=0YFj{_9`fFyyA?S(*D<|>{|}=z*>hNb28W>HIDW!J<*NEszyC_fvUh)3E%~pFKt1@s)>7x03)r!G4$QA=jK68h-IF z8I0jlCZ6)NojP2!eWWm!vit)mssK@d^!aYoU1x**SLP??3XjT;U>b&j_x_+U>>qtg zyD!x=pF|(5>wG?wOl4YRO;Tn3bN9Kd+!^AFS0(Q{BJv1lg&Yca*&|qVB)RY>U?R(1 z>@SrIx3=ll}g`9CzFh+?;pp8)*j&`rUk8OUo2SWw9X}(V5@hK^|hx1iuKi5 z&mm3ge*fo|^(I8%)e@)A>IM@-LYLa0zzvs>MZ4&_=Mj-SNee5kuOO@{2Q8sf<3<$u z{fAyfg;nRgU0_CFZ~sP&4(;tPy0}e2Wvyj9#myjwHiUUIHsRMx`ae^=lzhgW3;&4y z?DlT?a;v@5kZp`QP3X&TW5Lvkt2Z7`sg58!pNXCybP^dY=TKKUY6>4pnKka1xw42z zutr+PQprajbx&S=GxX0gupPXZ$0lrYnlEAbNd<@X{`&AH{}w5Crb7Kz7hUr_U2UWe z#Z%3mAJd!u-I1A>__1ky@=d8tRA4W6k+B#5c=Ajko7h4QdecPA@qyIpcAI@AAkOzP zy!>40*$U4%nweyP-_uecfnHUIF2hfNjn2V+yhfuHvhr`@W6Yt*wkpk{LHw5ax$MQ= z-~)3&2=wjjt)uS(?&dPL_jFN9{g3=%E%LBe&e<0gETgqHxuUl!IvuB$Y4^52~KBSU} z1!&jV5`bC(Ru$)1i(g?c^Iqi1_$9B!=W6T4=2Wpw8YXUmnLML}>X^?e9)1j;R((=B z6uT8dt)e750_jV(z72lD5_a&AJ0&U*K@^NoPB{a`9}1PPYE?QNaT(l)d8;#tuwi?B z)guae%fUk)O2Il1ME~R?4NfMe&G_17sd=r~=~h*j(&hT}4{(=;?FugkTays?u%a?|pmvyb(zX zQR}B9Zqa2@RZ)F@n?Lu=j8cz0YEeq!0{w(?=r`j8Tdm(4kDjDJjQX<#S-Z~7=-Cv$ z>7+0zZ+dQ0wLw2F2kcs326guuC1?XgGt3z+)eqjVo+rb3V7CcA`(7Xue56dQe$pD+ z>)b{q&rlAF&1J%AVLZ%aGLnCw-rc^w8}zHyhJ*f^{oVKOhwS!_z@Y&ETf#7RB(!Rh zEk=58J2(;ihM;bl*epUV6|li{6~)Z-f``xaouwX{9U)laT{RI;a#s2$3F)qP0d);= zijMYz+$^P+lxDC+L=^k4#x_>8lB(OGcq@#K#Ch+OLPoONuVx>OvTB$W@&0#Kx zCmV@U6qfNK8y)HjKGmA=?t7^8BTGAEJN~w#GMottRKDf{-zEWVK}^sW*kqSYYKFSX z+!8{2`0$-I=X7M*^FHWa)}y@6C*RybLjZ5E4;Fdz76N{c9+~R|EV%uvMXMh=W}M}6 zA4gXPE!tSm2$8&s{XmaufDgl1fS*ScHhdxyI@f{{-~58|)#eU2r-D-nJp-3oC%2`j zaMFl`m6x65@kE_)@kzx2>A#N;)jG{Rd+oGiBHG@W1cK+zw5i?C^i zDv_ZSmHeOdHzR23cawn(Q2B=D#yA@3w3LOA$O#Ru44fC14H+fHrY9V^;PgdijBi8I zVBh$H2|j#`H3WhyU_uR34(!~RSIwygd!)~kAV5ww36Y;sy8NZK_n8tI#9xN)Oj#4d6XA;#sBdVD zdJ)k1bbwT;jLdOKM>DK1?P+3k=xO|&GgZp@19yB^^J)cykXR}UTQfO8-$ugI>v7ww3GHZGI zx=Ysa1{FS(b*r~YhpvrJ+%o2SejTR8+XZ_&;n}~CGR60C0!NZVG-kBv|yqZwaN+m~SM*@`h>Q6xC(} zE@FM%mjTbc1Z(eu%p|TUZ-WU^RSRAQYj6Py@2JJ5nNGehw=aD@GR!Y~yN(xC@|!y! zLW4J5{(=`lU5_9Q$fGj*@_9>*OTBJl(!IlgcmZ6K*N>{e8!8Xn!S2|C|3w#e=Y|K{ z2c*M2pV_We=^m@~XkN8tOP4o{Iu-aflbvvenmAIlK==Eu(tWqs1D{O2Y2N${JRQEk z8_*+yS{X6CJuq|?{xKmhuF+RN;Tvefq}ArF9kHDuNI3QoD~W-lJJ6s6tWd(=u|-o> z@IBmHNG*VtsN}*wl~^tTv?8~OCw9Cf`;SNqjffRm$MD37cgx52D)N4bxiV1K6~28n zS=EHhT}TG~;_enP`ljYaj&EKW0Kc??t`4j?Vvy<=tTHe8(6K`!v)uhvPE3 zY-estyVL==R1JD;eUn#*`SgjlIsTHOoiqh@<4Lx6t>yhJ0Dj6@X?hPUAc2_c0Hrc! zl&Lan^fsQFaQU{zR8N|L$%ilGSH1yUCPj?;+v}|R*d=gsPvo~*tDd!ea#9duU-zzH zaa6am9?PK|Nzg9i{-$F4^+WthSyj?F#kv8wTEdHo{d>sHed%oaA?QKXn099y`t$=r zJRD2_ygNSnk4S)uUn)dXYVSp!z21_{B-kTWLXP1JaF-}|ZgReGT&Y?xQ^UPiOl zM~tsW@Xe5Y1m_Y-!yZ@)GIa3{%e;~2mw^bcuu6NIFxZ%M@5EB=?BkD-zy%!WHcK*r z23BmwGpDPUyzQloxHjqz6h;H1pp0%8u*Y4_LgC)azfSV&F}G?SgAey4y{!(W3ud%w z-2!#j60uO&zz%_q$7v`QYi!!1ZtjhTdEaY@9de*kJWS zUgz1BvC*0GweuP6qG&2-ztxCMj9V7V*nTZ6g*{_Ka6roM#QV4echO6*BFXt@Az&DG z*N(H%VNaZ83yg}eQ$EDs{rimY>ts;jP%2;WNx=)4M0-7Vbh)6PP{XljL`Jt+YqV?| zIVLufQfdGTz8XV!;~(1J(Bj0zK*d<;HND2KA>7j@mGK%-Ie{y(o0`{X(AU~HA;~u) zuj@iam0CD7Q>-nrD(NWPf};;!Dc9L^L>VbLz8g-lZ!;5k^`q;e!njYJ@n_L5!I(OS zE9nbwxQEZPUOIkDE!MDCmRh`jh(j4m;x>d}-K*qi)MX6L!j~bt(EeLp@62$qg9Zyl z^0lkN9FnmQ6qCbq`%^K%7Gn*rZq(5+I%%av*57qcMY);OTbKnEL!-Kt%0XFdmnG8!iu7ab z29T{_J9?6w8tTo%@Vw50?f*Kmj|sQgTEn`{|A<&eHnQE-HeJIhaNiI(0ld^vM*{%G z_8-whM9imgN&B%2!*3&W*0M(5Tqglm69HwIVpxo_EYx7CoeC%R&aK3uhSVqL1>qv5 zL4X`*?34-IJ(1E<|h&WSmcG+2+^GTv!UDnwTpz~%63#`Fxtke~rYJWj5DdXeK95`jd1OvtqcrvU}O7(>Xtt zz!HvfW8S3_H7Pgw?m5?Yxh-$0qZ&2m7e9$`ZPeOVHL6&DCl`!_emiNfuHq&|&o1l$>Ae%#9hqHFH zcY6dlHe&9_O;4K5kCOBKa6a3puLD0=-hbRxfAdj|gzEywU$neb{nYfiqR^AsW{H8v z$y}jZo58`WEz}hI>MJ%nv#*>b62R!vYsIyNS!=D26w%TphzG4FHyRee_csPNS{G5< zWD(3)fjKF?hWdHWM>rvU5^vZ?JVcf_7!41Z1Jpk50|beYvYmf>zw8baBW#=|5_FYH zvzICU$cyl+!qsKm^zEdQyv1sV-jyxsIOp5rFXWAaadz?G`w)izi13eJuP&-Gr@6yN zn%v*t*{3FM|Av-46z2Mq9P|JtERcik&9iTAk5j5IxD$l9>GY$A3y9|V37sp5)(PqH zRv|m85@I~5>7Ytky;5Tr-zAPa>YDn5w!T4siMD>RHILUi$hcw044Fz$u>EQ$!2&)) zSvRt+b*5f@nuQ6!{fNxYR%-SrOuA3{_r0S%Fh91T&vZ4n+#%y%iL2A$cxp5^ z2W2DvLR=PC){6E)P%`$l(^WMmE{-02V1FN~aQpYkhv=PnE8Pmif;uzCb6<6qF9CN- zbqXCGMZQI4Q+;%K0W5adw8L;sQFH2r{rWuiI@ZPy7UsdQnBLK@O1|TUV7vc$ZE0~$ zmnI&XM>sgPS)7#bu-kknDL7O`#~>R*+u7_*!y3Y#h4V}wb5Dy{it4;^6EX6bt|!tv zB-*|2CC#o-6IYP5GaAAa$~N_CP*e$mEBk?V7(gD=m=_fx4jb;g_B|xHXwTqQ7|Onc z2NC?>R~785?gH}foYN!$Ld}el-sG!MmgtiDiXZ9kw=CjCpKf)+6pNCdg)B6Yuuj z?GYbK)*oD>{1y*Pj__-r@UQ2)osN_a8YekYN0pLdDej<*ggpE6G9M`;gM;^?SHh=l z9ssm|ztj2fC5dQ@>PIiQd0Oa8%Dq9ieLl?6x+@9&u;b&vV^37Ai1lp6fQGx1$Ir&k zE6{#|<8RoTB)puQ&FBMZaHR5luajn%YSZU1%^|DBp2+tAZ7e6@+P@FwE-dakerQfp z=8$;Q1R2Bmo3TqEGy-E#V-D9cn}nVw!1`Fwk)qyjkxX}(m%TG7rR1Tq0sCruNC$oLLsip&l|pZ*;GfN@FECT%2sqrGDb6$aPmOeE zME_b{MDG;=j0cDUqRU%lT!)ZKcZMwcBghq_rUOV+Ut^und&>NfUm{e`Zgab@^#z-Q z^@9E+v=I8soL|n38)aa@UDbQ8#yBN(0qo0+)I)FDg!7sPLzfHM68OY1HRPo>)*fjL zrTmqsSi0(TKIb13I(2__&c}pYs5K0>8Fa8-7B_{Jz*E zyee((<=CO9$Xn_|ss-T8>M9)=yR+)$=%+t}p673*JB$~T)Hb~G3pB&wY9iJPgo zItfzpF_y(hEfp0kOWaKQjepi4OO49S?AwP~ad|$(OWi?<#1p(hO^HQ7c;|NoFxDY; zt9EKD`}@uln)x|Sq2akDeead{lrhI_ne7*r;4vSLq(GIfAb!g3@!9VTH?EWxCnR|l za!KPuN!PjmP2$=~`Oy?2fORb*1kA2;F0GEhAuSaC$zmUuF7H$=8F z>GXZmTTXF$u(LoH)>}DaPEEZoXGO*PO5@4Qt0W>5PQ~lDcHZiz?H^l%9DgLdjkEEc zWpoeCO#`HmI`{8Co36zoj!AUo-&LOLXnpA6Y}}+r2-fVA{g0xv@N4qz!Z?ZolA?}= zDIqaJB$ZN;76FOTNC?8DyHN=xrKP2t(cPm#I%M=ljogSa^7lUPKd{f|*`D3^IoG+) z_hRByj(hiu<%kRo!T`HVvMf9864RDbd&wJ%cVey{`%LWhj@oRS{kxbh&7QBmu139z zsCBdK6EVw`BMrDkuw0Ae)^VqQnYWmdZZk|u&>q+AxaCKWl|v9v_I#t_%`B}4w_N&* z$LvnGTL0aS$@kIsdtO+)IoI%&cm?E+!QrE*8P*J1=`yya#rw{v&!N z3CdS3hH&t_*&JxWasQOoxBjL(bM*##mP{IKGU4W6$!mahVS0`fz4Akr zoe)tjUJh4P{eIc_^_+t(2uizu4T;Nl_>oV0i{tAn>!pU-d}W08xBY#62bs{4k26KN z!xka$*b<#N$QO0*v3sEx{yKbaX3K?}^%r6$l5Hf>f56vv(FhHCSN;#5ZteI`l3xU_ z+I0U%1oC?`4SWsXiC;x!iEj%s&b5!6?1_xKpX4TV$PfL&(iPj6!@o;Qf$^aYF0Uwa zg=<&LiDdFJwyRzS!uSXAQ^FtqZO)ujD&f;Fw&LEE{OTX7?^Q6g5KYkW z*L^sj38vaHjHqZ4JV!zOQS6BI80Q<>bLhYSh-!cxgFZ2=DzGMUFl}FP>dJ9{7E=e3 z?7qwjfZXou*Mm(@juPoCO1L2*`5JV#Ow=h)vt6>98$P_BigVtU`vuw@^7OPZhBoK; zyXMOz#ND0IR42}+1b~8)U92zq>)b9F(%wfK{gu_IIWPHWUr0HTwI(xklZXr3!{A(1 zaTSHdh7@O)p2*D4{0EN?!pX2f!A`Qq$I(h+r_y+)sq-Eab2HKrNZtL+PHLiJ7d@8Z z#-Wk(Ec`Fr2-1C~BFf%rg1__Rgkh8jo=_|IlkTCNyh8k68dvsyih&MK=PpKv^ozM| z&!_+f0AyV6x)j`)@J#C&D+AUetFIjhn%kTAH-EX4DvN258*1F*xPtT*zjt_d5Q)l( zliA&?@_L|UyV?}|Q0+d{UA<6}k$dqJt~_-k4hvwK*l5J@b@_5ae!Q^~!OBt24NnrZ z`MWL(@7$%p6n5EgfC2p+PZg;87>jQDy?qe=QofwuGmRbOUS ztYA7*Ym^2-UHUbSrsvr(>RAm#O6%`VqQ{Qf1n6>-Kz^X5ktj~-(L{wY^>1Q$BRrgI z0zLjDOy5X@H()2-uP_>d(ZyvHWUrpoL_PCs<>p{kKJbZjD}Ux0kyRUjV+c(R4E>Y%CQ2Re5R-m6+iB=SXurV%h&vkV?E(}hy7j| zO-B8~~r|H!>mV;z;XKbI$ z`#3=T=4@+f>ZsgC7go^oh7Qnm!n=HhUCF;Pfvk-)WrPW6Y}srm&onQ2g^oDUX+`yc zfk)j&!60qUQq}0Ko6MPGApqHg1OrDbX)P7*S>tqJggCaCh5o>6Te8=6tw;y1rCm@l zKcQF4gUIyZ)FjN|Jb@Xis9yVwzRZ-Mc`%#GewN*p5+NU_)D!qf>}fgB-%S&iD4MHGq&eVbv5o$iDf}=OqKf3 zRmVNUwLa{0=V?UHe;s$)Z+FbE;>KdWtS8-Lw$J#byCibDEG5{e zlcEea&#%nc6w{ckN2e1X7!S0rz22se8daRe3_Ho0%6%FOTESKch|$EzG1fr>$*yFv zJ-h;&}Kx6!j%xg#1dejP!K#EMcz%^ z>|G@6Xk2{4tqyWz{U*80TfbV8^_uK~ImGgie-9$fSLc_j@jIF|i!E0jRg06mw_@w{ zIo5KGRDqQUbRvO}ySu|C++iZ6ev6jmJpap`DAEraA0dMQECdOCTzdmuRIXCS;vo0i zC{jJkaU#h2%EA~Ph%;s**rD-G*XfS3dO(1NUgWV8qC`>YEx7z)92W=dhMe?2qPs*B zts(_um(n+XJwMrZGy5Cq{P^Xd;iiGV8z2rmg|kpZ-A}P4-S7L1imrfz*aug1VAbCC z!ED5vIo|AD{dZ5>wQYORHy>1=d;1;d>J|(9JmkITTSG*07VOh z(*32WD?FTk@-*%5ojjynQQUn-HR@YtcH8itiA9R?*JSoXL2P-OO;v?IPYnbtBtIzM zxzN)OFg22i=rP^tUpAxe%SW$}80@wSeon=859|be00@_`|EhUNtPe0j=VvvFF@#- zdlVaiOsQ6LjFD0ucM?aCwVo5PbH>as-8_9!Y^)pD;juA zy_`}pg_7%~$Y6dQkEnO4J1Gwu>2Y+P+Vt~rhi9-8ve1?^|MuQFF^s>})f^zl6U8{x zVJ4!yyWEtIVef;vZxjs5?Q3^`1wH!}1TN(C>?{w?98$R6z3)#1S0LzO)qv{OHz7=Q zJTKFhp*H^M$1t`H0!e#)<8_cMHxgNqOX;BnISDmrXhu(mEErAqP8J@F>RF3cCfY%| zUxF#l>q8~~Haw@MQ9u44iu3@`U`p-XUgPJf#2?>rv+5$nnQ$DnkrQ4SC zJE?ssJ`Lg(ntk#dya-;_XG*HqN7r9FRvXM1^)U`RmMyO_kB<{a$%rB4xi_@s*e_dPDw{P5gm{+qdM`B${0;qA=3ns)Ur zMS@?$QW)jBF*C|%pcCs5{|c|CXO|jX^lx~6A0$mQ0W}nLKgMk197_SnuY;EognAmL zE6!Q#N%;M5Lt$qwU#|o+7<{3p9hYQ*UG~zXINV+U^VR3iej+FX@%}C0-h2eGN2IaD zJw9VpSrMq}a~5_(@8{Q6H(kNigw2W)T@C0hHc-L)HmaT8dyG;ZCD~r!S(>dA6+(KV zH1@86Qx1&WTym&QVIHc-MQ7ZjE3BsY2X$_+Tz>uAb%_*&@|b&R_Jcpp&6)2R74s9! zwoc4=_(?e_%DSe&DA1fxCs*&*z|2rNGEg`tEm>}El*rR&xp37IVER_|URI9J`*NTQ zd9;m%1+X@U<3~5-d!;E^SvB5}t%k>`y@wq2^{*Aiw9_mCbzJv#q`SUc^=87D|C-cf zU`+wF%Z=iTx`flAZ^5UAmmI4?`%vsLTDK?jQL~EQ3)F(3kSMDft;tilaJjW44y5vt zY*;u(Ma5JO+b>#N>pM}q1$tn3*N^6=&W-+=K;tpTE=IHa+gf#i_DL}p!(Yp)v)(Rz zL0yU3SC@_`clLka1bnHAEu+S7>#zS-Hz%d*291li6`a{P3C`Js@R_(xkW{Eh_}LT7 zjd}~r_3{kF)@jJIN4&vDojtz3e_akpWpmZVo*cn4N!avU{?h2DUjx}3M>0!A(dQ>^ zrX*8`3FzT~s2dEhM8woKVV*%L@&~f5;5UQQ#@l1VV)m>I%BR()+0rlBW=2u7{criW z^J4naqF(N-q|nDWBx(=&oSFSFBWY*%<86#n&hb~%Fb#@q=J}6kJj8p^)V+s78Yz0I zOzOfxdqJ+MAgPa@m5z)Lue+O2T~D9yasTy}L?SH5M0qLb=h#UF1SJsz^wkmafwX>R zbD|xj`fpkycA&zDC;guyr2btH)gIJ8o;M17(wfiw8(<&EAzRRhD@Pm%$N@47n~QtB zy2P#l>A7et-M{>@JBD|onO8DOAzgY;$_Zt@e08(joZN4nwz{&HC4J$LCLM^AzzOsy zzL@;m$i75PY0qta(Fk_1KYd97~IEk5#?joI%A-dS}MYU|9GK`Cr@zm~}PQJ+Ia9_2vL+S=MS! zdPd@anl{cCeb&2XX)!w}QL6lU>mT&6Q*@Mrt{CN&_yM zCJHxOB-wCYbSn}AgO_%ue%r5nv5A(1XQ5IKY>vWYc6||Ghh+VPel`Nf6aB0;_QnkF zHmVuXn2G>4FNaO!h^d{>=1{+r!04@g0#pIz4@#MX&bRFcd}72`(5vEVpP9(#FS17Q zZO}MKLI3PbiCusF$~oWU#GtoBn;`+3I(}~72i^8^-}>qJWCoNoB;v+eI^ovlvZeD) zGLIP2ryx7rjrVAFwtxF1PB`j(CpTLz#LJ-^_hEEcQ_bY|GfnZyJ>jWwJ)KW{yuM0> z78@WL_L!XVZcIbTLp(kh?M6(&7zG`#)YJL_Dy41 zGZMP;61H5fml5G!KY0ro_IMOW_s+9c@$urhC3c3zCbQj|rAfqhsMD0>jOT#VwgYC^ zo$8>wHan!BDk7A9!Q`JitfOU_-7@o%Uu?F1@0pug890;c>8IGogsy zjb4m`eoDDY#nq2URPKKh9ud^Meqo3?wI^Ea3x|@GN%;FhC1yCd_8*A3+aItAt?hg2 zVy~y2c3+5(HYyQwd+nb?$B|TRtphcS;MnPBV8b;*Y2HsLlNYrZrNbwVOOjHmlG{pC z0HwwR8oVzgI&IIlyCuGJ&v`0yrktYdsI8e+JhfSR3S|?WJub_M37}}z3-8wqRz3~| zLmVCp(!9@rtq9fn20qvx%U2M-J__*|(>b|d!ga)espQtAu&IOyWw#j3N!|6xmf$zv zgH!xo(4n!*d?UG={upTwRBL-(8_bVP!V+(Ey^tX>m99pQ^y~Y&E*a z=L(_JfB)Gxu}W?>WYX!B^p#5-xGOrB6_Va_9LIaP@jUs_wwSgNPzD}`JIW;4eg~Z?bbg|o=A!k{Lm>8{0;tV46Fc0;WF_r`&T_S1S&tdK9<>x`WOS(72fkWJ>P!DZP4( zt^K}7_l~-losBEOrHazS41LylS`p4*pc5CV7wTG)Q{jihrJ>r3aSi<%DBqF(LPLj$ z_d$Sq@4pyxC@Zq}3^GDSCb?PWZB8=7Lf$(k7A1u6QRr(c8>z=9Ri$voM&B-(o7Vmo ztiw?)U2t$P_UJlCR{XOW(G9Z+VMXubaY;g5!woMN2XMk+$5h&84-Viryo3wwrw!%S zAB{G}v6+Y!WQ${j8J!?7+0gh2HKX-=!|2EHrzzrHJq91jO2$efhLHo-x)FoGY$HyC z%`4w|?V3)(CWe6?N^!=neLa06-=R7RBhNMBg=Q z2xBd-;-NCEucVg~khiY-Dy>D;O5SEqdiX|y=mt~@vuf159))Di2t~x(fCoU8i0K%N zeuF>l#G|B~9s149tvwt|PwOr9hI-<3%kWULXCiv*8TRP0fP`{;Qr`m=ew-0^@xBgV zQ#ZP-d+Qx1X)Uue7O(RsWaI+;-^>mFL2Z%VzIqvF2=N_{Qp}^tlqXqiMuEQL7f|kk z9b`UIi7IYrGGraSLAqR*IyK|pD+RirYB}Cr2NH3 zabYvASGzsEXZfLA%5*F0N{XO{0<*3J{y}VXb_94b0O3@Vj$t<>69M4uLCbvjAsLC1 zjQ(tWhD{-cZrG;qZI$9nzDRO06YGAZn)nYLiRRygldR#2k96E^?y9yfd~thjN|q}! z(50)W!c9be9*B|v%qsPU>V;ko=)GC3xSADDN4y1iwo&M(9G%r{@?M20>x+&VWYP8M z;NJU}z_tbYOReqxrwCn~8yMD+P&9S&Lvh3SWI6XgqSyP&1^MrL6lHp*_Rt-Zz~}mD zb-cskU49WqL`+?E%626B3$KDhE>@y@>vPczfi-?I&`xd*Jw3`jr8E$#Tuy~ggY zUd0qCp*4OGny2YE8|rpJ4TY=}Gy@*+d^ErKbRXSJs$2EHxyo>4_pFqf@+?Cxv(|lc z9qJ(RIhCaj@{MJT`T6E!p7r=n&@p!}IJ!li;M!Foa%R;C9<&_-;_7mzZg{R-rmjH5 zwuTP{M)MRUH!Lpz;l>YAzH12oQO7H0Z|y46Qg{6^?P7ad11bZ-)CBiP{EhT z5|pZUu1l)=^*u9jIW*1-C1qB)q5cg5udXknw{I1s&6nn}-sXFxxZ)AJni$a0?W0aHnJV#2^%d!h~P6tkHL02LAZamK1NX@91z(Eh?{%JTePDu8B6! zT@RUZG}pcSAZq8u0?#Qs;*x!R3#(*0}eG ziwMz20ye0+PO{ePRG}o{{hh$X0E8bs;Wc({?erVc_L)NclKjW}G{c>%XwS(w#QRXm z)_81LN%aUhfYu|aRecf1#tvOFae3f;dn&>`>sSz4lLE+eOG{EAKAdsTm z3ww0TKwd6Z&Mn}U7f2b%j*QRYrQDlK_gPcr=|tjUX=h{1fy3;qG_;>S>_daSIB3W! zOooO9`n@4!U$#d;P`}k~i@)ikjvGn)w@k_Py2hj>r6)v?g3-BjQH z^jC5b#lN!Nxs`bv=V!NCWj{>BW`&h6sMB`~3?PJyOhc4Fiwg+8h2kj1<3YR_hIehy zjjVkjUv-~ZnYA>_=u@prkj-SvBX8Yk6^-BatKmm}RLeoAZ)Sr9aK>!q~aL`Ww~Kv=$CF;=_#r9;E2zl6z*MEf6Y*Ee2w zXy@4GK_`M~YaJ|PNu%BC4A9`M!r7LhuEu5M&9s^`4ZbF=;OEyk(Me+3L>NZ`f40gzQ}#K&LnVy+!t_ z)AVIPf}~RFEVYg0RtyW~?1lsP1yBEw|JlEs5O8-BF%Z<-AWHGsZHbqo^;Zj#2Ril$ ze^r_{{q|)>?ZLvkua$)it9Y@nFI73cUwU#Rkqne;ZN1)^e|eh(3pK=0$>yo}s*(jF zBwy$5Cf`eCSuJ`OnQI7C3wxRs>83n?xL)ODV?eVQb>w@dN7noOly;}^t(Lph9og64Fc?G z7TKj{hM?(N+C$=CM&CjLcK60!1!O>m07McnO+c#<%-2$`q}EhE*uOszf3qY1Wl=at z33qFyPSNa5fnaXoUHG~lUmnkEo%r#ij;O5KfzjQh`2gZwXX?|tST*H10IOPU>STMQwW zO1 z+Y1B(QPwCS$Ye<5yo9e9mQLuqvcAu))J7U+T|ENdkkgi^t*wMGiP%r=Yqb{F(Z^hy z>P%VEA*ib!a8wYovnkFsHG&vE4kh zXKn7Kt>PYLvFEs>GoHUUbCEh)k`!SHuY5~OrYrsjO`PuwMPw20w2RD|Pt^V%wn?(= z&WT<>A%bRlaU(o$xToXaS1~I=V;gm}7Bf4c$pN!jb)$*cVT22Su%~h{JX-UxL+1FE z-Bk7Y(iplw&G`}OSOo_#?8pm=ZVgJ4fHAY0nN{DEYUfR9fwg4)#u_8ha zUQIu5{%d0mO)G4&!tHz`Xvx%X^r{A|t>5sxD{tW&(en;+qKbREO%r<;EcoR7!Mq~d z0q?J9&|#A4lbL+%__P4$J?BU7($aDwPl++6!ujP39LCEP=$4pw&}`cc>%PqYh&VlY zEX|F*ejMA4me3@>{VB!f@ug{AY^jX2Lh!>g{tSj7f6XUX6`F!<$D> z%+6pG2)Zdd`eYA$MB~t#TamauK3mcAh>~<~^e&2aF3h<^FFB<)=WZnobitx8-)>UakV942|Atp&=UMBtc#mHV~!9;ZFmDmgd`$6tDAzZtt zT3Pqia%<7RMv=(*~rsE^K zg8ZzT0xpS8HUxX~u7HdRv#t)jfdC~5^T@9qcc3?6ivI5J`bkjM=@||^PJ+>?TM|KW zMgMUZqTiuwj!p{v(Zj)0NF=y;ypp=VcQg3|&ZaN)Q6vjnKuBFs!O14Lsq2o#^WOdC zSAKLvBs){afV2m{gazI^No!OGx3$Mq4}^Pbix>pF)ezo(QOm=C#Q;Lu7u@ju|Eq%23HfZ zP7H2*{%w#GwO%k;Elv3OGlM~V9+>m**Lso8RsTIh@Qu-(*vw8mTt(tR zUS4(e$O{hEK=O^#d#SMG%Xi^soR(iFn{}dGlU7u)Ni~xM7VX6~Ids(%_bBmaKw{DeJTh4y?Cv{(G4zqG7Z*)hA1LFNF zrhg3{Q?-J>K)<|m!u)vT$_)b5RS(kL=ds+?DYOq>a|-~R7hPh3tJSiygj+=hetd*8 zQDj311zAp`YqPKLI508D4+Od=YT2h=N~n%tduf~1Qi9`N1=@f;iSDjy?7rCs#?@j_ zmegV9k_Kqp0Tjl!XhI=gA9o5zj4>{&NCb<$(0d;3B9IX+A?QP+W%M+&j~ToynCxA& zEs=$kWov{PeV@B5-$OK(0!Yalx+_ODBe@?Qg{pI9`ZCd_pEnt^<7?!foBz7BA)Nwk zEYsmUGSF1FS9+R7Bb;qLnOYr!pYb806SJjF?W!vWfCTu)C8+`?wn1QlL<23fH~)A% z(3Xc2rx7e4T#r)-Hmb1Cm62VzW4TH3ACXKhtL#y9-G$Z~%|imT-{)>axHNN-(T<_} zBxxvWepY#2Bij^mdW!b(2&jYl&4n&JnWZN;~?3bGWNAbc&zN* zo4I8wLT*U|F!y2ZTz-HHX=FYb-n<2T=?=4|4}gNtbJGl^3$Nv9Sj=vjX{>9CD$jnT z>0mOf7gZA9*{hwj36Zb2AUYMRx>G+D#NjxUXKghO@vDiJqBW36;$^u0w^-0ZOvaI! z80JkMf1G{AQ_1!TWkzNDa=&o?Fb4`c7r3+p-U8XO8{yB;hlGbcO#Q)Pzsm9cO7bmn zv-%4x4mwXv{C`9D6#vMlQ&Txz{3H=!29`1ZBI-k9OD1j{SL!78+<^h}8o?2FPnL;b z)Sn5#ZKf0}TU0epIc_Tc3iNopES~O*Y6R92?{pGcE=7j(2?+ecbVoxw1h&#T=?4*5 zS&*wqaBrRFq~3WGNNM@a_aZQdKGN@Q0c_iPVM84}5~p-j9g7LSawg1QlG`S&(ACjI ztFw+axqH{a*&BZk9C^#HzqKBn8N;5G?ar2D4@DJg7xJ29MP}95>|yX`wA-2F9Z=sk z9~Hs8m7eG#5rL6EZVcxfdy-nq9f7s>57n@JVfNYERb&F?zZ8L8A+2zN2Hto|TEiEI z?t6*vD3EFS_pg`2SUHVYIy%suJX4QH_std|THXKnF?IIOUh}Y!T+gODTQ{Re4lg1% zD>vTA@hxOQ=*2SlzW#M%;7Cb&x@0zVcm?Y30f5|>W>3P{Oq!#wf+VkD7Y!-e=3}6% z$iOaxla|s$WH7-+Bk4tFlh%$R(KUo`bb`W7ykj*~j`$Y0Ak=5MVG9H?$jFAkMYXFZH;;wf1B4E3%(Ec6{el4ZulavQ$r0KEihO z?Kf{f8hISv%MaEyju1Ucy!t)nDqDyY$pfkVKQlRh*9=98BRPkfFVdaQEm}VN6}!xilHCDLFJw+f z<)x}plcMfds%=<~jzv0Vn&z! zhBZ3GC@w9`wX+V+=BJHyD3-%V$FSvFLjEJlxsH)_;l37Sl#vOlIPMIPi0IcGa@ zX{_h?8?FEtAWvjodcZ#EPhWyR_n zGHldiSOP5hU%{o}0sW-e#GsRccxM;sg%|cqbfo|CgKo_c-}P?^05wRYbVU|*Ue?ug zX_boUIk%X;dRR5G;AMnB>XlslEw8n8e_7`EOl$Lb;nIgCWkyNX(l>`+)VIin zEx&&?9G%6`r-?t4@zoPJDE~w+^tivyqeKP5X+Gi_z+O2eC=grldCmM7@@$>y*hKtb zXOKxir)QL5wKt=2e-4>~j-i!jZ8kExz1|Sec>#^|I!4vOyT_>I!rdo+Put&{24r+H ztzBeACLPNnju65ZlQd>YfzP@AV~GWSha#5g6BXM^LZj~nNPDp zd!$&_W?g$L8@Mxkt{ZN|m;FBAf&*)yh?b0cVcMq+ZebgME}iKOhs%9AM7~^Dg(8|n zmq0UD3Z=ZEqmR8!79INQ0w5|d&((dRZN5oU*z#j`7)IwHp*G{bWqQLah_{r0#Pg$< zD-;AzO+qeOKx)l1DEG*XAm_=9UJre=wi$SxyxgHxChGsNyPd5X3k~5os$CM6ZtVAF zCSIv%zAtCG4nw* z37>^RtL(@13(Z1U4O_ z$L^}mZ<2~cnDEWACor!WQX=xLD=oB<9JSDVcF)|G=T%#`{Qu!WU!UjnzASE(BI z@2}Pw@EvR{oPTSaoN(E{a$hX)!H%=R-Q$Db{}JWRE1hwi1?pYjbqV0W6~IyUdPeqq z8nRoYqs|Oh;LGm)+eQ*A5mFoV#>B~B z+c^r_BS?*@xlGfQ&8U_M^RUf6>BA2+>Y51${S; z>XdGc8g4q}zJM0I(y<6G`x0AD+wEWnFW&VDh_E_qnB;fvlX&1`0Srw+UQ^20f^^5j zbz-o)Zz>_TFH>gj%D0G2f?@ybIr}InT8sCejV}&@md!@h$16|V`!?XDyaq%kmYecs z-eR>Y!%ylr=l%PDaAKRu@-9Ze*f3&iY&F6~o0r>l8yOKHY$0E{$8i_h8=BBx@tb~?(h`Up@NBDhX~EYu2Gb)< z=v$uWf1{UIqWm|o8aMYAaAqdcWmodJ2cJ)NB`dO1O@zCg`ITMd?|W?h=@+|j#uDSb z3FcTSz1`dtcx5nZlEnk~r_2jn*;J?(iDWe*kUg=tdRf}@>xvVBF?X-BeaBLryzgRZLW0|2YMq_;MaYF;qUGne} zB0Na#=0R_G@2MQENo%$x;L~@21QFqjTDj1Nly*r+@-i(0RowGFTjRpZ!!&X&x(gI>Yw*H;FOE2i1zLh0SCIwRQZLAEY0KN-~06-&Lh)>NeU?=Vm1r z=6i8pacP<&)h#_C5%~kgf~$euRkc~HUJ)vKQ~Qy>rO}SJlj{{+vUE*z3`-R5-CEgT zj{3CA#)b_`FfwXm_h6J_aPEI?Dw1zV2s1PHE!NoBKmYY`>I(ALPXh&O*|$??>q=eQ zAEX{?X)^5y0(euR|CTdkG+-^bPw-h;L4CXVL^j=r=_Dp>jxK_D_htN{^1~XE^)wHD8*5 zm$S~nc@-IDFn9VYtj*r;rMQ>tnoyX7TN*5@PR;rFr>Bh|$#bPOGeu)tXx7K0x)3fs znLm-VV}V6%Gs9g4J3bmw0uG|Zl!odHm3*51=<$!&E8T7E%R4j=Le*ZV@e9Nx=L)Iz z@s?ZJf5r?~t)mgHCi-ISZw^oP3fe>vNxXk-u?B`k6!(eHa8ThRaZLqI)w~9*^M$Su zJ8GFb?mfxM!0gGS;qMvyZM|6ss>Q#xA1w>>orUh}Enh3GnHWUBmH-mHh`#UYBij;U zWB4h8? zMM$cbL+3T zv5o5toyd|P*Orw6AYHNOa|cZ;7u+ySUG*X7m(ErZO1e*dvVWYBZ4_^@0kH*!{AbEs z+z)tn8ypMx7+PPig(WO&NF+Z0^op3_(V~vM%L{nb@^%1#Bl1p#`Q1f35&;j%TB@ys zQ^*RT6jscAkbYmvvK`Xbs#b1yc2*@G#XXOXAQ~v6)`_N;yI6$&y*P_%Rd`x+ohIxV zD4A2mNLhO0`yo?YbSCaW_TOT>0>HK?ADN?fm_3(_ddZ*h%oSty!Y*}$)84x`=gC=>N#s8}tyYmnhw zV_Y}tsngu@r#|uu=}sR?&!S+DSM!sv^Ndx0;EIvxO?S3*H+;pyJM0M3Ny4=;K)7?? zWcDYEb1&0VqJ=&ghzCkdCrbC1rPO<+Kx_q}OTTX`-jLvdRd+2zin>s=l0-S>iTd$M z%BBaHl!j^0j=@m>TBVA61?u+yR*$hrA?|1@zxe2ZGr`wnd&PIdo>pbs!d-1V_Qls@ z?{inYenBq0&|r79a9!H*<&YGk7w2WB{MP@&i~(ny_~@pzy>|cb%7yNl6Uh0YW?uQZ zKE#4|B1Uwu1D1shdxI@VlM55=qT0Wx7^}_t9T`X-yCCx)5#QBg++4(I4CAf{EOoKn zs+#5EE?baCDVrcJsT39UUR?tk@2adqT*;Q$D{ol@gI3x47^_R4QG#emRJaR>;2pz9 zJNn8lQ--IqYV;8q-X?qXif_dmQuC1p7+gwz;Eyla2(h|A?+09UZ9kb_B{HOhM`3T02*v zzjWKt;zq;`6p_z%kFXx4Mtl7iCm<$>H(M8WZgpDj(MeOsYCg~l>a?jE$&n5$TZjFz z>US@#lxq$p^Wt^zvHI9}K+kz4b!iYV3z%dB?K{{ID4P4-q8en|mV3>lVS6`fQWQFC zue3s*gNR)%qrZnA#~|tP89*MQt=~IY8A2LQIFjqWZLGv#Y(NtbdUK94HxhH zoRdLE;@N5owK&OXtW+-5Z>E{J5kl0)e)kDne8axa=1c5g)0I1bmgx)KII@iqzL}qy zPxWs9QZ@LZn7_UI>mj^~Kkr0F{WC@r79cv4J0-g9H)mewe5UD>^$IdlYq?^`CQ!9?|72v<3KOrkyhYi3Zj)oR7E#Es>{TPljzRZ8+uwS!1|X*hcs?CysH z9(J=|-!F6(G)t`ZAO*}UlZ%Hp$S<{$r$4D@<0Om&y)s`y^R8}Tt&81D`iqScG|S-Z z1BG|VwhIejgMJV5mb?yoXPZCGZiP5elMaCtiB4D{C&yjwjFs5ypir_-u2-0U9&jOg zU9E_3OPNX3@o+AQYGJtOu)xbB^;d5;4>q~NHAD}Io}fQ(cZ4__CbrfL+{ybT2{&k~ zrV{RyKIyFR*nQ70UP%r>9B|3n^Ul@P(wnnAT#LR+M04|jR44MD))m?=v15{A$5GJ< zeK`7I=9+KfTR675voHkJU2XN4VNI=0=H>z=ARU#(B9~Tx$_YE2@A{|ff!~gS3@=`s zCr^knLwGzAT4l$^)c6(}{;$tB$dd-q=2~qF)O`ajUl~^|8qaH`nrJ@^-KjMfeL<0_ zSJnCn zs$N3f^HQs5&mOt^t-|GYA0W2LNwbYeOKWvRwDYxy8U58mtm9XJ!zTmN!BL%zYsd^5 z>92>ZY>-MPJ4T`&Wr4CdwME*}LK{SkSk$EW+c^jI+8qB*pb{C#QQCyj2WLUGW{hV4|;V znIh}%*xVJC4KH*M6}o#f(GHY5!LEIpIXg~Iyak1it8Wbx??_!j)%lI`+gD--jqWlb z%ee^Pu}`zfT#+@uf4Zmqw8OYDr1T0r zDQN9$tZd>@UzRVYws+7MZlG06id-|i*SiE3Hca2vNU|F_sSyrU)|K1HlAEmE-=o{v z&{{HI8jl5Z6T?=dfdA~(kT>z|7M@ey1FoKku~$mwC&lSZpA>D&$?ABzjTucy#xOez3v}GJpg)$Wa4I@69=>gk9yBF7DS!C<^U;#^R`;^ZI6cCQ zobvUpdSget2&`AF1spy1 zE%(Vt-Fv=I9QOy=i6=9;rTubzJ)^t>fhpi9yqw?X=5feG1oqG5y7r$XFZ$?J`85l) zkPjXN)cc7Ol2^e|ne-989n^Jd{x)ijVE$hXe1Xl|P6LSM$~!=)@zdlY+XsTVu%}ut z`}}1}aq)Us#lqDL3!%usn*FmYj*Fhsb`1ef15@~!pg9B84HgmI*WOFuxO5^wQR4Z4 zmlaK^wg#pjX;jYln_sSuRxj(n)#g`^T=n2cos_Pg1v@7c*HvBIGcPUQ8j_S-DliQ~ zcw&rCUc!3%^j8X(Ap01oA6oR41ZA67FFU2b%5xG@Q15cCBqC}U*^^MinC6)Ee z!jEt*oU$DdC|U4~@wuCynJzy*vb=ztbe4VQ(_z@{%V_RubMD=c24f07Nd z6;{Gm?x83z!$8L!ePiTZo`Knfn!va(g0qZfEMzv_@h7C9=qIKLaeF$lqCTFTS;4^6 zTPQ>i5Ab`DlOQ5CkS_4@NP}@|ALhm}NCUPH&Ml1~^a|ZcAT#1pW@kGoZdst3zbt9% zER<(d#n^on@Ux>j=8y5Q<6qnfera(N4o4NBG$*a8tbD)sB8+=>Z=~Ly@oH^Fz+6wC zd!?-NFTsSZLjMO>L8!h-UN~GkL~!f<-YZhmP}B8DW4yAFC5~Vj84f`og*9~OEu@xs z01Y_>(g^sU>zsGD>$gi(-We1YD$NPIZnJJ#*>3oT$==*5F1Bq%Y+nCpR(Ju}k1 zhRO(wNfVE_fwAmsr0{2ebRP|Pa$Pd|6+?7c+n#=Jaf;`uP0rEiYZ)k-cTrpEdQGLA zjE?ySXEy|I|#bw+0f9)~E(vrdvu`|!h)34IEbiE_{DhTxJ zA&8<}D*6i5U$%x!>2&K>jx!X&Ss3=}1#SySVok7cr_kdSj}Vn(U4Vk4n%7Y?#tLvi z>02Ws+RHh4**$H1fIPPMRNMG&etiV zNfBWq=Okz39=}6VQCcH`;un4w@fV6L*y^soA^sl6J4Pwt$vx@e~ z!^bLu2(GNV5ZpbDUyeVKjwYHKsz{Y*zY|Zf2I=@vW9dc=#AS zkF{;uG*(4sa(t%#wTmRS=4UbvhastM@{7Jg_y{y}qw+$Cb8!c78<*cKFA@=OQ{;KsK+JoXn_P63$J`-pR zQYgZ{qj0=@hmjJI-2VUr}0B$i^vs7J<6hz)mDNO9WdJ4?nEc`Fr^rxkSj` z(q#zU*i_p6%0dzdkMBGED~B=MSvt=KRgmNw(h|*bMALG{p~tDDXQwTTcEdcL)U79( zCp`yBbNq@U?rK<}1kx#XCxH%qt7Zsy8(l3A|JVLeJ{f!@()<}}nvOzK?4qww)sH zL5U@Axfl4KL0=?ly7ljiq>?)~Xf*KIn&tlVxc>k=4uY_6Z?Ak%g6~+(y@r2tZl8Kn z@Aa>td;{>o?>wzj#i%2N05Dl7`El5{_*PF+k~y!3pABxjScs(`PtfD!_LAWL0D<+d zv~(RmP0@7p({!lT7$X~1RCX1eY$Jv-5{xSF02lxc)u$A}K4!pLWgToYBw!Q3t{29i z6LilB+O%;nmg65P4%O!VJNUJ0qe9wlj6{?4E!k`4*86_4?R$A9S(s#>Ls-ss(7X0D zwcioz9xA?T{YpZ_{nh>z9+j0eiEc9i`Fa!YRPS#RH}c@8U);|mCnpLy#bLN! z!L9h1TgYP#k~qy}cyGf#JNT)rE`{LxIWKNHgp70@IrOgI;a|erPl{UT({Aq;0oP*x z0IVQ-3jN*vn*IZ6KL|WNZ)|Ln?RNkTzyj!l-_+K1u@I!K)T&fvQQx8SKfsUpC+>&g z-w(s7{8F+v`ljTU_ilE@#&+%Rk9z%2@O6!jkEF$`Yti{v_D>X@I&eMf(>ybx_}{_W z=(d*6d=~?2zA_ZoESb=eli{sPPVok*HifSihs%kM9Bm3fuBO{q@Xf}P_D>GlpSKg` zlE-KudkcVwQ`-&*)TUA&jbYiS~2d2`VHE8YAprP=7lN$ph_<0SRu*58SIO$FVe zX_gXsS13e`+j0V9rYboSMwdKn{3^S+`F_{)?aBFYMmu(_-50{v`Yx&_o=1{N4%Xek z`qy=T;%lq@L+nw_<(ET%JC20?MPc~iX3=!Jd+8@jl1kvH<%dC8$)sFjc{jyxh_iSX zMVjW(w@Dzn-&l5>^ zt3*7OIEZ6~Pp7Y`uV(O9!0mq0-|g4Yl!V8?R@os_{oW2Mg>ys!^1EPK_A2% zU=TepE7&|Kt6bY@kcUq!M!0eAE25mG7s}BUBNr6W7sXq9`z<2o_6ens+d)0SJum_i z0RDo!%l38nxuy8;PKR0W+^%i61)Q>Ap&(>>b>hCBNvt(nCl_#e%F0x)7$efZH-F%s z%X#4+9{f-EdkXK+^zBKevUmO!Vt!x1Vu`^@E%q)FwukB7m#H)o8E<0pt=*57?a`C9 zbXIV|uUknTmHv~%4WM<$Zq(}wjWb>Fv{FXlqHL)OKI;O1r%IRMRVL4t9&u{f$4%sPZ7I6Rh9FF!EH^RKIAiZn?t zSS#%$f$dzrp?9O};o*%$%M##@+xv>vtQ1wQWlp1g4=M29hAdviWYa%)c7F49&#Kk3 z`lgr~ds4D2ImBoIV0-d?#cjuBHM_@krVx?C^AtZT9lgGo=~YgtZ*g}m#8CNa`6He( zeMi>0tP`S8REFP<*X&p>9VqD^zZFUtL+}= zK(%$Z)D;xM=jF&G5`>O+pH7*ss>8!K^2FC#%xEqrMJnMlwEBU-uMCG&hsC;Nx09%` zYj;sJo63P3agadu>0e2Bb&}o)v};1sBB(IP=YBy0CqG^*dKR14V=9^>!oDY6>#%s6 zPw>Zw(NYN{P4Y8fHsOLWI}H76(7Xww3rk7ixw-R}9np|mmd67N+akBTOQ2}iEY_NJ zyh(a)SIn1?$WP6Y>D#4S)MIZ9L5sUZc?Auk;0FD?JEc|eKU}N;J$B)}T_Vv4&rtlxb`-b^LL|t2e8>u)rKd2qK zuS)@do+4ba*yY6HV@fw?U;AHv+g4r(wV%PB0K5B5!{u-eIQ4&TMgIWx55a#C>i62- zp+2*!Kb<|p7iK)sT5tk+1m2#5Ud&@U^?%-(Jd)!3;~i z+DAB5RUJv}EADW(NMYg01bn_D5jwKFGBn*6Rq-~hZ=h%v?{#@}>l{%;rP-N|0044H zJOBlLwcBaG3I71#mL3DN_=oX27(7MsvsR3qA4B(o<|mWyN7ejfNyme}3}3e3MKBd4W%{{X}P00DTX#=7$8)^WzDgLKn$ zNYHg->5A3(dEi|`#5b|&8eCyV-jUA>8jhs;SA>4oU$wr6;4KP&fnNh|2Iu*8Ek-^b zN9a*V?c0iql`Lrp3Hw<7*jgvUuMXUN2mPO>#zxIgQ$zF0{RmUpYw_9TdzqT*>5Ds_ zOCI%dSSFfNEy*n_^X_v`)HJL60H17##MsCLR}80fsY1h`xAUY?bgvvEsmDBeSHAcs z#MU|vjW(NL;xuLD@rORyubXbK^}DNqEY5bI`A2RmVf;y`TtZU{xLu?H)K?vP5~*d- z>Y-jV^|9O9c)wD<)!GYCJe257avmkq47bNrw)^V4Tiklq_LX&~PoxV&^2o=8LCGVf zYX^$T?%(@X5p{0z11ckVWtY$%*sPRPUWM~HM^<@uq+Z`ld59892g#btwzj|3<%3GI zk96~3ZC-I*UcdHel!jf+j413s&uaS1_Hg(=dWJoFRE#K;2FvybpcS=B>Ww~TXHJ8y zQ<5j6{0aCoZ{ZCcZ!f_6M9LY@^cD08hhup$4hI6dv7%p>7WS_cR`Fq3nMoiHxC4(& zRo2lVkN42Xt~wk8UoBRws!hsF`f4;|P6{zN*)-?CS(DcXt{8U5(y^~};3M6PsOJOF zb;r}VuKs%&)-?#5h(99?;Fax+9AthKgLV?;+suRkA#AIU;`I!@0s2#kZ5+mhW|M01 zC~U6axIcH>^RLm5hkhNF<4;(uV_!21@^1=c0CA1j8;3*BBk5lc_$NyjTJ$j>0i1=y za=Fh#=m_9?SLu&{d_U&sdoIue7s%-+1zX?JvBb+md@M>4TF3#4P3q{|WR5|GL+E~# z+s_U~EWxBuo3Vk|=cRVKwvRe5_MGBXXjCJdZ^&MUaOyF~BZ}v|Nq?(onors$v6|}A zXt$-rNx32ejJ7!h9C44*xYM!Ne8c-gUuoV6@VjVGJ6*1M`|Q))gTo^P~D`#>OW+xhSZLB}=1Xu4WXpp)ijj(UDo-A_oI zHFc4xrddOA5b{oCKXeZD)@ki6jKWw{ti)|-_GFRhMk-qyAD1~)*n2O`G4EZCq!XxP z4aV6N1{oRPbU5^+!6O7|K7Ejm<|U9X-6@bTdB)z~g>#yZh0W~VWFj!k8QffW9Va z#{0pE4AMG$vBW?seZBpw&-`Pqc!S}#v#;HF2HwgP)}jXLIa6#(Q~13JKA!dDJ{$O@ zulOFy<67|K%-W2OlgVWqUR**sKe}_!SEUYUzIr+?$DcJBFK1t0k@Gjj4}!ig_)Dz@ z)@>ygZ00qXugkf5u6kE1r%3R!%NHbmSg+Ub59;0l`15UVt$51z;k?*_ww`h1q=Ss( z11Gj?#Xcy0%sM}Wck8|jFw7lT8&_a{R{rm}?Ol}fRWEX?vp8#H&MTgl$LEdQK6mcd z7?APkD;*$`Ib?(Twb<%j2q44yw zUNVA>8zU+A75x$a0KvO{J-LhG{{V=5H#sXE+-WFL(>*Hi{{R%D&~;b1nfpHJl)yHVKe{-Ea+K{{X>D zzBJf;7yX^QL*ZWtm2VN=*hn|1%G;xHxxM=+?rJ2hE25&_$LELq5y$=tgYjbb$2R`} zvS-K5Q0w-v&Mb7jPJf!=2{?@%z|4T3lorblImb`xlTnIgmmq8&{SAKwKk!Ts3~9fz zw}v&p+UNF$f={&A>K0RI+E`7uH;n|F&cN8?I2q}XeJk{{;$Q9e{{RI#{i(cso+|hm zr9WQ~AiVg1$s`&J2QbW@8*q1|0aP8mq|ueYVLRr}-VT+RNPl7cV@G)f7e zjGEZ|t<~(c&k4+7*Obiw;buX|{{VOjz}GCa&l2dfY4VXQ7g3Zr=Yl__eu?oM@DIjU+TVwt!n)i40K~^dBHnO+ z<)mPIjz8XC*1Ut_zw9mW*IM|Y{{Rc$M7cUpn`icxofK-e_e{MX<-u1!#gDyQv_2-h z@OO(K@pKTpSJvy~-P-gMz!ZK=N#q^zID_mQ*ETeKKZP@jMQ=KzfXSpVGPoDddZYKhL#n2*;R;-MJX-Kf9_(@^K;K2c=sVF!_oaL=xkmILIE=li~?4CAPTJJUt>Vs1-tQ83%hO0RVrOj;>p9aJg8uiyuzVOpf)r8r#D5RUS8J*qGG zNHr!f+Cr$EQbs?_S4|$DcNOCOv<#@gBfWOkF+6{|nlhktsV+R3Uung3sy*40pJOH% zRG`KI6!DsO7tMfXKmj#0p2Nz+|WU$N(Uzr%epXKAgTMBFe}aVmcU zUM;2gb6)swsmS^Yvs%c-o)*bcPjTsAT<=#HT7qjBW?hZ}6~kLCgruhrc1y9eaV;)c zYL6q*d`WS8BFNV6ip&8#e6`fv_>)XV7V$_7g>$#sufCOKWbY#Us(**Ico)SRJ!ed^ zlf+sxMiv#_lfw^TS=X;u4^Nrv{zjDP!lbRS=~r6jk$af#rP_aowkfug#)zqM?%-)_+PQQoCmkd8D^DF-V1>0F% ziS*|3lrSV8m=W9x-L%$VdrjcyBZ7FYOIz_(pg&^V8;_d@kyTcm8j-z4u`qLsv`)xa z>9)FO+Mq=hd>J{sXYBA1s!|Tk?wZ7_S2Tq&_+5AG7zvJKq}WkE_hwtU&q>uUyuv2D$M&#uuJ3@e7xl<;hXmSFjcJ2koQ#NZovE z@%)-blOY;5n&l!LorizIz8bl>3c(kew6r;=E3!yWe@c{%+?s4elo_ciOG`@$BkkHv zN{QwkX`v&KoyX}{5kjC};We9OBuf+<2py`@-57xj{{ZT${%4lBW4qFzh|4+^+tQ%b zX`yj19gjTK*cWl#j0(%OQ_~f{ZiNt%Dp=#GSzJv!5IS%xPB^mMdsQfXaf;GaVw;eF zMP|*Su3U-<5c_tk9%aqDsq*rtjMUcQF}pm9eeK#>TRel32LRT7M@6iR4--q~#IYj~ zM__u_5wc(JAI_~@PWQ8kZWt0j3SF>zS3*`+IHxPK|Iq$OekFWYRzqP9;2Z8lr{;gJ zx3zfe8eXTXU&VQ>A-TG9`|(CeQW9;33xY5@We%z z_t$U!vQ`}Z2cWKKtDS~_f<7I^q{y1ayS8iWe5?EseFv|-dquisdFxpa05MO#)vWYu zM!mj$h?MPPSgxl<_Lyyu%RB8=89Cz>;XXR}nt2BqpAAVqIhc4!+LPkwd1;J6c}P1z3U0owuly_+INTaJvT-=&xu1%b1rtk4mSIm z?#-k>qHt#2P(*lvZ74#p6d=G7>vp?B{VX|}UpIY&V zH2(k&cr#aRXTm~8H!b_X%t#+W+PwsLyH?aO41lAidsn<`8bj(Db-=SgrcgQC^LzBJ7sH+u zy&8d@bOtCCk`E!X$4_d)q#r|~5>_AKe+lR;miIcGv6pYSnd^%7G=|wz%8%w4!1Ut2 zZSf|Wkm}LeczQ3NZui@s3F+)OuIIs8zM-ewHLkE7N^rn-2j8VkD_Qc`rwgqQd(=Eb zH;WNp#1Tgzf-fn4KrzWYfHBEA&3P`N;r%{K?<7PJdJa2RYpdAWU9i+_t{3;il>5Cq zW4B839bd$rFVNTQ8br}V@<@f}b=+1p&jZ_>@H+}C;uICwQKfwm=y7Sf4ZVizNZa>n zOAO=_&rib@@UOye3;3VIzZWj<<7AUh)zetJ+s2x{qoqoc!99YU6pBUpU2*G=^NRNW z0EGT5+VI;=gn>4OJt)Fdj>cH1v(mNi59%eYue3QCBQcb@EzUY^Yw(g}{{ZdCiY=l+B1y(qrArNj_U6k6y&FKmKC{U z%6ME+Udw20?xXUe=!|~xQ^(<-O5j&cT|V+n9i46&5nRfnZgY?U9=vw1eerIKFWJ7? zG}|%QbUuQ-f=jqn*n#sE!8rMdU+}Jau;sEegSm_$g8Ir^8}t&}5%Z&T^!z`ia6T}& zf-AM~-Hc||+6cjgc_`u6u1~&2cDIZi1ysksucsZcSl%G;rmNxzirLE)%NmA?CjvLl zKJNbj?&q9itzlX>kGi!rrqX(h)2`8cH4TmYNB7aX0=VTf$3DDzSFm^s#kYTENWRRe z8TmsihFz)091I@aJ6F%%6w^2z9rNdEwYD$WlLiw2daF&2}fX=0331sf5N{gH61~vcA8f&Bc53GHEQB>hB@oWCa8F8!u~4wmEx;k19%S3O+!+= zeUZlq$>3!B00$(V0I$$!Vd>C^v!<+%lB-sAWqA;Ib3ySpkGxG~;NJ&ZJ?^7_BEtkR z46{3SABR4G4Dc)TJH`Gr_<8>T1kLcIeiQIOhsD1g{6iYS`+?`jqxs_jA^>wC&I@&J zgA2j({{R*ISI|FV3kd!Vd;*O;S@D;|YPQ!GHp}LV11BC#0Chg6_`Bd>S4H7J2>3Vt z3C-cjK0N$?k{=iRd+`bxY;@U}PP$7=(nsr=f0sd7POmnfa_|2D0M-0L2f8_m8Xit5{;wTuOHcQ1nFf z1KPQ}RB2@7U56A=9Oh8r+Ai-edhfu?GJ>X4K)p7<}EhK065FB;Za9c zKEk}?;Ag^JOI_AHs6KuF05dTd5rNc`*b2q{s=sQ@8{yW6H^EL?)j-$t z@6<1W!+hC3^$zuwcM*Y}D~59BI^!E2jiPvd`%qNZvHOdB5>FZQ#dMz%yhox>rib1P zkus)w1>8Dk`c`*@d{b{Pg6{k|G;*uj$gk%Phj;M)d{-0Vy$8e>+D6SlA&Gf0?d0R3 z`~`70X=={6$KBBMe-7AK#o@ywpi1tUJAgRjj8~0GAcF2lm~9!Wx*or(Sm~*zGVhBd zH?Os3c!A6}Uug-k2+z&MYf3FQb1ITnRyxlOFdCcNDZzM-)*VGy*YB(?G_8}#yyxWu zu2!-1eNxLxu#;K6Qj@%Rfe6My?_Jl!FN9j}#JzITd+69(Tt7t}zr%`7*K(G~adNVZ zR;=qkXWxaIr^NpN5T}SWsETROxGSCmH(~Uz)Z0G?_cwfv%f;>Qpbx5BG6jGmXYN)6-+=FjzH4 z8t!1&gDgVpYc5wDuG91&`c-R`0UL1i>FO(ga@)w!PI)IGyYuv_62l$ABX>@CJXbVM z$FYim3T$9Mya?ddE$yguW_zC0Z>8t&-Z;n?fGKAQhY0=Ncvs9uBRWE+c6Z2#DkBx-8ymdwR}(TH^X0P zzmD?)MQn|PWE>yU+P_LP%_=J^aXU8r$KQ{TugtCPxXPbuFHk{d{^G!tLuw1YM{#t+as#wxGU$aExEg%R-@bw4WcB$V_iZl-!Gx5~_0QIT1 zdYnawf#Z{Q2{_~d+O-V!;d0I8eSYufSvGAIp~zg=#V_vNmO=i|cUe+ax^rg+og?a26@XC0jPZmT7|-WoZ6L5;cU zE0Vk=d%c#2bsCZ7_g6#l$Ku0iz971i-+z^J8!|;G7!IS19Cfb>^6o4d(*1DpFE#eJ z#-D}$1#cQYfuyv0V6(U_8IR^hS3D>kk3n8h@z23_ng@nq)D)jR2axPnZt>gluU`#b zG3BS+@o~6!E^3RR@~FN`tFJA%Q`_3Vp?~-{<4=#P0^!uUgTtb9S&z9$HXwX(I2A2S~=_UCWqUc_T*T^=;2cr<@v9vQG}D`zjr z-SYh_V(vnVOnaDZP$G~PvM;p|!)UEebCA`*k;xrBtI4&`D}L5JE7CO!t-G_(SB7}g z#CleNXxi?s+-GDEuilzYzz;#*tu&D~*2vEAcgCAf0(@Qm%hGK=*{rUgCzGG`P%Dv_ zbK8J1>}$ZhW8l~L)qivGFX98qH-)t8k2#Bcr&PN`fIsV!9suY~YkVs3p!kccYF3Fc zOG%{jb^T3F8Na)03G4vigwbUPpQHu0tqBK_3^`s1a3fOxM`(kIbi@Xvzu_qDQsZ5w_} zuhSJgcDC^-R8T#Dt4gsbz!lSb#Ydtr*7rPDT=2%5bqCr)+*-=NDZt1h@UN3@`~l;? z4S3*q{{Z1`y#5;T6@z6jZ3ZH|jC79b54$4v=R1FdgI}X%V8$Dd!nvJe!+I{G9%YCU zIuXGXm%9DY-$SYWo`2w@(BIzMd^7#5wD|lb;whX%ZK)y7lYGa8^5t*!Zr}6G@#v+i z^&nf@-^8*)!Q)|&MmbWhu?W=7yUZS-+4gk8=Q6obgko3GT%q|7|NGp_bT&6ySDkzZjP|r zAS8a3kNZ3R(w_$Z0BTPOpA!5l)mGwoPRpwqKX-9#{{Y>VBl98pkEMEktDxG+z2G|~ zQ}+*lrB=?y4kw&V5G%BC>}xJd7k`;BzTK;6vmI-}F4mVL<|sbvdlAyS_Qzeh z(r$)};}{K<%A9T^5x3Cx`d7|ka5X8m_eZBjyl1i7S=+}NmcR_y$@Rxd=xw0|0BWmk zYa5_GwY_Tuc?;=W(MIi@&x+xPP0`M~D|tr18AFdiIS1JH71MZn?#oq$r?+)~Ew&w} zI3u-eyf*T~&djQ zh!#n0I<8c}tRv%&q>=j9ynqUG{8g?RtZKuaD&DHbb)ew+Sq0Vfdz*O!(z6z5$Ih#o zyDh9TH_o8qqiG?wP*egdXL1#b5K84TMP}Y=5koHChO{KPhsxZQmpm{`Jk6k!*V?7| z0wsp&nS9Q<^r;2hv5mE#6n|%oM-yd7YJ8KbLS}4yzO@b67i3qyxQP_K*WK@5H+)j@ zG#605i>5Fr>w-P;Ufro{T6N}_(>57c9C~qIEr?$B`fJJXK__lKE5gmHK1B%iJq#3e zyXbmX!tV=sZ^4UetJy#e?d)JnGabhVuEY^1H;OO*)BgYx{4?_iVqHgml8x_gs{)vBkxz#Ke9K#dHx&eYi;6D=W25|nH?O$bv!&9LJPLj~^DphqKG>k24SJN~tF5_9%q!8O$ zBS|B-0aZse{M&xfe;Z}}r+zA4_`3d9w(wtu30TH>kgrS+SI6aFWPiazKWJ@B#hSwY z&EE`VC8QwgbH;d8{^uQl>tCLJ1^8@!DENy{_($NCCrYxDb7TOi!Tm?$507W~$?*amnmv(KwzkK~wLjIyqxEY2IlY=m=Q2rw*eK|#H~@VGNuy{w zCxkRRUkP|-Aq|y`VHa;3*bZ3xnu=5aoL13~LJY~y#oV~B$6xp;pY2B`iKn;1F9^?z ztChvmq5FyH_}APYwXf|5qWlQ>d2^`AAZ05wDV+WdJ%K^rEAEjkQ63-(~FIK6!RxLt-{$a;T)RlyA;B`F(Lv0!Z$2C3L zfE0>`YZOF-kTF_I8>--Z+E0W#8h(=)jYr-;#K+#cNC_AMqg*-pIH)i0?ex26 zyt-|v7|mgGyOft%*KBkv6M1;sanO32`JcpIDAl}UcrBsJc1YKyY}`k#d_dKKhbwNa zjP_Ap>)`(Y4ETFWj5ePTXNcssIV4vtTGrUoF}{bN{1NaimYd@XJI@)9m7fMS=%Dm9 z?ccP=!+XmOLJe<5XyntULYCz^W>tmbJI~&r%Mk`b2y?0x; z(et6{k@=GS!HxhJ{*~xjKZslGsS5t$PB=f(y1jqE(CY9R;Y3DLy9zlZSDoKzccSho z-w__mUUaH9^*W;t#@B}RnJoNmB1m=_5bij@CcX0N{^MDAr?qIxj9~kESIhn)c_O|j z%sl1_peMf-tDyKp;=@?(@cc5eLcnb-7p-y1wVkYWMwW-w5NWp3+L`u+_4E~5-di0y zXl^H!W{y%-yuaWdjdRyCU-(l*bn#L&U`Du+58i(D=bjwaE%hs>i5oFDTzVRsn2TtT zMw#nZ_WF*gt;jUDS!^H(%F+J-5hIX(l|xYQ7QJ(MV|k!VArO`_t}s{}b^2C|0bG5J;!msVVK*lx2vt)psu+y#>WhF;?w zRUKF37PX`_dX@a&bo`?k2LsZ$w#BB1lH6xxM+J}dtP9)dwX1d06Xn?7FU#v$%J$H; zW9mH{;(z=sy)FxxEn$Y@*sBfPD=T&Uc&%G+6%7MZ6X=qKG5-LpT;wVA{HxFOEfiVk z7Iya*j4_ZJE;f!j*HLBRtu|YkV}Lxee;GZ${X(C;VXGdg;Qs&}e{Fb(T^p$5K-|XX z1JkvA8>(s+mZ=OnU`$Vi+?<1-di>kJu)ff>z0oQ1*|{E-^yh^x?x)a%@s?$YhH?Bg zRO&dZF*v65JIzz=-WBmtlTOoI*~)V7jH@>TjDIT5)_g~7+P%8pMp71SuBpZpk4`#Q zws>+oO-oBi?q&JiFmZ#|xUUfSrQvIhGUnc0HO%Z>1jkZ+`&SMnN-^e((CwttYAp0? zJ0;d5f(uN^42q>?+lSnE=DS$_6X{|s>WpNS7jacMA4QWN4hB$=W{dN7A{XgflM zbw3OUb8i~N=_n&0vhu&3dR_jzu3PHzY4?dUv&L{Z zDh>u1b;V(cpEHU^lw)M>ORBt3+{J3l1I`$r_1HQ5syMD4Z=;e=lyCyCpsc-XUbLT6 zlTR^kHZ0F59J&PH5TmF#?OV1NI%cnD9rd)U^Qd8hD>%EmBAOv<-w;0dowf+uc$okm zVCV)9Lyo-C>E1ljwVP+s<|*cqV`$ibcMb@_{Ht1RQq}E#&2h@F{Ip)))yDYWPS9_3 zs3yKtY0<}#BK)p%&H*1!!j$g_-7%{@8SI`G)nv6(9i)3qSWu^?J%w~WJ-+yL;av_- z8hDpv8H;l=ou#qrJ-(IlFO9rl-Wl-Y>sp-g*(7Ke%P04YG3q@(TKvBF*ZV+ttKwbP z*e1DSWf0mII0qH&WjSQAkKa~;{)os^z+4uE|AMUE-ui|~VZx#GfZ-*AY@XCL(4ZrsHjkUq$YWmaz zEWW6RqDG&99Zxiu-YWQQ{{RI1@IA+Z{v1mmiT*M8iaolPt^TlC7aufn9>jM)pQUot ztIpI`-M_Ae^y#RsW_7Vi@az5wr{URk3yI+I&&U4&5e&_6*q$vU$~g#6@$<`nAzuLM z(`)`F*Jtt1#4E`)J%3SeHs)y?e$sm`eJYK$z0%#?c%Q{H8|`lR@3@`L@}%_upVqA- zEXl#$?_0v3F0XPWJLt9@1m0oXqP~FmJMiNF0LGfkX#(Lovp3yU9nE>~!>AK@>;X@lIdY`(rv;`LQMg%h2SMjD=h`%V7Tx*x-T z4Zp#!g7N_&QksUMpLx%9Qae}Wt(;S8$v&#*EfC&!btKjaO-EOMwcVUX$CowRX}WKj z16x@g!){JJ1$otKXm-h{)GZr3FFQV*Fs5;}fWqv9VFPk*eX zsbH{$!l3>jeQVCN>C)n35#x^D)z(?*7Sr0#eRCl5p#a9c_^7E(b8|B3)oF5~ZC6iS zA~(1^M8G-6W9?qqH^F}q>RuK8*zuw+pQh=y#_ieh=SswUpkuQFFe||PKjVue@g23- zh;3Hx7&C~Q?w0IZ*jLgPdf$n>e-sxQ!^?2e$g!%o%845`5Mz(S+PQ1Gu(wY?@;dP9 zl<(Kb;d~|V^HTV;SocfGk7pTP zKIuGVSIE*d?Mbv!aU+8WU_#(D zK%bX_Mt@56KaZN^T0g>@zX#ehQMI0*BcW{VnnuETE6E|G-M}5aD~c4X(r09;@2*$8d9WBeO^2Nn8_Xo+{G#eWV8%%g78kC%R{g6pnk4~OPp557xECyd+8)RxP=Oi#$h51}28^sf5s{k}0Y z=tQAFVcU*4rfC{1`i10h6wY|YJ01>xn6D=GJ!>CX{?EEuf+R_kY>In~XWUoO-x54) z;lB?gS}v`Awr_Rv=WqoF_m7|kzFYqQg|*13SR5vjGCMCkv1`0Fj*a!LF9x#f`zA`1u}-_M3YmHq;!IQHynD}jb&!C!BX z09QYkwP&ns(i?jgG82q~pkxl%Jx9`|nj@lu^6EuKWsGh6yHo-(>5AvI+rxBow`qU9 zj)eOE0Q#z30d#7L%+J$^&c0$nIGCT`(SOQ zNL=~8Vy+7nKi%i2YWWkxUK8-$-QsH2f#X|PK2s~d_OI{#-`g^eX=WIk&b$d*Vi^$KBuVobuG2MOU#uSwgn(pIWh=_ovNNdN7!>-!>=3>*;{oQI~A>AX^z&ksHPlSYN$uzN}s zoJ2RNMt~YgYsckOyoSdv(N>I}#c)&)9mkc(e$smWBUAMt!GfX80G zr~9_A1k*|8z(PuQT6mo-UPhD4Dd9J3m&vGdpSl9-$M)T1ZSz_qTn=0Qwb}&z{#IyC z2&aNFTFV_g1?Hkn=uS7G38AKz*wx!yUiy2p@o%e0 zKEzvRCUEobk4;}wwpZ$8H(I8nc)8W1dju6%TP96R-}wF)@<)>IZx+Aqs;y@Ygxr^- zvF43$-g}Sw>*CP^U_N#)saA3(o$1hRd~B@CQ@Yvot(K84%_rzfxAlV407KtWX<7zg zvA6Sngn{GN)t_EwZIf&bWX|RcuYFO~M1%GCsKn^rZ{&8Di5djqf3nLsrZ~%GW@(al zZcO1Mn3j^=3W=v~3bGK2hTZqkfKx(}N4_xl-UIyQXy>>76m>grS3c>Cw}TqT*FE$L zp@7*s|62~Gb~2qJ4{~}6`p{?{B1zBN?+N>iZ;7n-FH0*~-0_MjQJw)I`~vc$FpX>M z;k;j(+^k6bQyc|Da_slI^_|3XWgDjZ!f-lOjevf>}42={Dwq(CxOq~)DM8SrgDD|neFx%! zJ|Bbk?(`ymKMQA5QLkq!O!-^vuI2f9ZU&FUf;y<#p)bBIIFeqKB{pjl`q`C3L>=bQD4Fk75I zB)uFMz|0=O>Zg|V%0zzuLD`?B!3;l8k`t(f6BkqxM~639P$*Fny2W0&V? zDvX|7t|7--!i+2A0Q?6z8Q5pNbe&P1*Xk~BKL|C5=jkO|95=|Sw1YXsvglgReI`Uq zmO4Az`40#f2BZbz`LTt_pp!7oUgqn(l#XDQhR$JU_Mo>bCtBlpnVcOFhIln|NQZEL ztfBV{^xEO1lhR}o0|vmr83P6LEl`SQ8-e=3NkR8r5}p+Na*Pe20#M!hM6@{_ZH#Qy z^=MTpHDyk21wWnEz0gcuVwihd3w9dE(l@w_bTs#ntXV+ktalkbRoRm|ql(TydXGgK z`m=4;mztR><(Y0uJqbU`tS+!7?j~I$B0FP}6(y%AUY`=o$wxGi)ZB-NcJ?{*1PB*4 zu)E3qTXv3Tu_HAu#|6`kfFb=oovM#W9$UC<@(n8I37?D&)h6^@^k11t2k6>aTEtVztFOqQDw76#K@qO7~f36%XCeqC3l)osBxUUvEmF((G# zq~n^I=HcS;yg8E3pD6E_KYD>>Xop8Mz~O=dQ2W47BXg{beQN>i3U^x-mOE2jl0Lgp zKp^F@KHVN!R87XK^BmOq$um*=sFaO}Hti;gz>kG^MO@w=NMPr&vw%?2J$2)#gPP{| zjY0+LsY!>k15sV-y2mQ%cstywBm3l;P#2KRA&V0KJaCPJpprD1 z?J?Vm{U$}c2mUwR{U1^ERc-eX`x4Xx`%PpHkQT%|*x-a_E68 z#~LtpWI%4o!&gC^n=|4CQn-P*m)CILzq> zo6%%?OqT3}si80RqfBP4iH;TMeqr1?nSHa}p3mjm4Ba`WJDT!RDAY5 zw$Kb{D#%e!mA!(6OwDyAU-0F9*gw)~qdRl!7$!A!^mtL7#J)7Y!B1%AK;L#aXcy)d zx2G9TSlr*{ggD1yP*Jife|F!y#L#5nDE-BSNsr=WIQ}q54~DBzreuB{OF(_N@7}_R zk*%;Gl#zsuCbJs(9iA4*B>vr8Y4R~Ri{FOzU@d-fKbRC3HZ|6rqbBr=+}gvnbsYq( ziLDaXss6QA&=as3QLMpsyz$$PBB$B6<P)(J&9J;Eb_!$g`AMRK@v`62lXa_>}-7&_oL7IV63BRFoSnGnXP}2tG zCHKduy?Y+c!)eYXOh z!semckXQ42OJhXmBSB$$4~=lVX!u;~K<4u9^dP$5FnL4eof>bN|LU1YAV06W)lSl$c1FgCR`hguycLUiohFg~@}zd*_ZKBU$IB z^Ye6((w-Ya-}|@1YJR3Jf1#HVyBWW0Om^%s_8X%TZjEKzHCu$gEV0!poF{lhjIbsT9d_^tp@Bn?So0 zx=xhPV3=@~bqJuWxAzzU>v-f;ff!Y&V@)s)$OX;0eMas#N&k!7IFZ2khDpNKs!!j~ zy-!Yd3t}}*md}yeB)mJ`EX#B_^s?e#zrRslFLyr>^9D-|IS-+RXU8i{x4l;#PqaqP zkG_C>Z@DoGlJtHS8vI}TleS4%1Ev+w0ut!yH~s}M;U`D0SPS*w8y2dS<~V9Y67{6_ zHr7t@A>K~|K{eaAFUQZWl`1Fw7qVXUj1zENQbQ?R3rlJ}1!BIhT}X`~7Q7BUm-$@Q z@gcs$XXu7`IE`^x?Zbq%lT;$fcmw996jUa2<+LkzBW7cI*2V%_zD7Iaai(cUfVUcz$(5eU^0tBqQpNJ=Ggb}k^1Kz zHKyVB2|Fgij98_m(wsNV+M{n9ZLQ|nXs4~7FU5H{Tv_VJbcc=XsvBJGx8J|aA>h4tAaNY%Vlw0Px7#~-6C4aJ zr`5S6JZ8eT{;f8v@#2%-hIxThCq=ru^QmvAw98I?U4kx4;>y+?(`$l`kF&v9CgxfS zv0lM>Ajm--8@vxU$y0xlA_|!67v7K^?+CERbIw&=u?im;fd!b^=GC^2k(SLZRSy58 zKNY!neCx9Q5j`*^hG5W|2G-037ly(RJJ zD?9f{Fq>m#ADZEW=zUx412tkMl&b?4-aW{$>Y60FSSO-NP_6e6-uFvWf+Y&}Ri4x7 zcsI4{fZHacSc?cWYHJLf5_pX#`4x)_6!Mix&*~A1mOg@uOI*MIS*!TjVulT{b^Xg- z(`|PgwVAox3%X-(g&@e7%m{M@uTB|jg?|@9x5bPoL+foBsWiApl@XXE8B`$t+Ip1-4xa(jlMcp(|U!F6yuf&*lJR%jl0`fkP3O}Ud{$t z&X-HnL+0t%K4^q}Ps;GMXz&0T^jEykpwICg;xG1A7uw#0UArCO5zwe`nBCC>j@i9) zd$$_rx$hBEK6(EUnTlnQmH6shl5M6hEPlUj8Y~*unan7NZQ+9upTEN7W)Tytm_Sgz3P;WQ|$n`GM+#+x~04v5B^+Uv zOTZQ7Np=cFoNNZ=2_f}Lp5OuTlNj+oS|!V3wf7o@2c8%wf3w(Qc&RmHv|jB?&+vGA zA7pu)NRuA4W=6;DGY<9`5m<~s{BZpDMNI-nF5|ER;%VLn9hr3W@zbMH1E5 zC!c6%?poI`eAf&jUb!JLmVZ7WUNDF0b^`T7p&Own_C@C|aW)+LXIKVA?IAHe_{9piqcFc>P zlMl_4>O18~^3m5(KYm?w;_`U86Tx%R!!D>nDa)UDSxfVc=o~qA9e6|pWg4}dkabuY zkeFm-Goq?HM-Iy@l62g%V`P?1hQ0E3ozQRlFNHWPTRz_=tuF~-|09aI1u+VUwvqlFcxi-` z4^6RzT&ndav?I-MZmTSxaTMMRL!W_gkeGW=P5}xi z9X)(?`B{Ypxxh#A^nmRliq-JuC`SC?qlUToQ_X4DKWm;OaT1;t)aQxN-!S#17EE*J zvf4fAKkcQjAJ0{{>79Sel2`~j%*?p2?jr6&S#ND06nivFzWSwhrKn8)*)FJ{uiiYt zq<2F?nys0dH5BZ>>$QE=6z7rKcakqVsgsd=dSC%s6|in+=fdo5Gd#LU(~DmMG{zWx z7-#7lKe|i8)#|j;sZP`db8<5z{K zlhYokr2qDjBJ#d9@V!vLN`%9J@TNC}vIL@JtccdB%>2m>UA&{SfQ?@9l39DQPPm^! zxKPYIbx?!CzVj02keVaAttZ{A#CP?H!|K*oV(%>1hC@*2-*GD0PQr?hwfr7YAO%|g zb+UcgCm@UF&j~+MO6+gK%IL?gUq9u%>Xzfe;u9;znP%bm1rUn;31Olcu(u?E3ophM zm^DOr^C4{$QFku$9zR~yx85Ln1Wc0ly;l70TlK!~ck{GQ;Eme;J?Vg^w^K^)gZvG| z$vwX5*0CPuJPE5o>c=Zext%W&68fCsTou3jSl;vzQC!~vD>@Pkl>{PFFvpJQP9}3~ z$%3k#h|FV@+*dh?c#>+%&fl83bBMu&Ou)7czk%`{Wi~y_USba6kJ9nfV&hs?QZvWD z4vwA<<|Y)On4Y4>XQ||q3^fG-;jE8@)fU|0u>tOF376<*kak7Ym!o~*uEM{CZ$Emr z&%hpWd&~bUeOPK7tSz;oEyM0CpF*K~N!6GeQz9gF6%bvzch5sOQ+N{q;3`wRk9toS znV*3Q#6Z{xbLB-a?vKAr(b-cQI4Zhp>XTA;DafO({81yg-=&}GgPbb57+t<4+0~yG z?Lyc!qXQOO3JRfWqhcB2-A#U+KFNmnx*7Uk{ktD%Sja3!EK9(Y{gYKgS79V0tAX_0 zwl~@SE;Jerq^rMrsMi4|CaV3pg0XHK5Xsfy4urfL70hwo`Tk!zE6uFdMCd;wPk&^j zk0#X^O0$04XU`dep0I;)zk8)NpD9O0@*%06V^Roxk{x$UBAAW{piLKqp%_-K{~k)+Bvd5H)x;-LdC0RfUik zm$z8;k^I%42tXXzE0CaEzG|P;{w0Uy1z5m-36ji`)T=si-Sp-Sk*<^p@1&l=KQlCF zE=R47oV5}ELz05899qILWy>h$;3SeL9?Vkue?&V}u`S>nijxCyK`Tas8pk78z6PUJ z#KO@BNW4ZVI4DtVfRrqSJm^0~Vh6qrQG?71ahV0N%Bdus{d|y9B_G9z5 z@~B4nNww`TFIWE8;UcYbQ>|}>FaHJb$LHPvPDsu>Zt}(1oOQxiRNCFAYg2}vD#aux z>-DU`SJ_H(3ID6m+et~_J$dHZPY8IiA%(LDH_byjHNVx_=G!NZu zOqXe#EY%9Z3CVr!#}nsDa>5g_r{rL;GW183{E^e87h0t!!>xmz58fGIL(Z!DK;u6m zj4MYwmbJ_YMv6}7cb9^ zV_4nzv77m7Vx=<9UuK5*((sGDpO=pZ@L#Y&v4y){GO}+tCJSQ-4r`%G*oQvl03|`# z!Hf94lM_@H2GkleH&6kQ(fraOA3c1+s^72Xsv85$f9Uq-9<W%_klE7bbcunK{6qiR)acAXpTaV z+JAp3lK!zZgHV$=FvbEO9&**sm9LajSTWql;rUvI9nQn1hSJY_3)z>}Kp(_UMnxIOWsb zRjzqA;_${=09gjFd4ojy#b68ERk|v9Vr??1^vFTNBc@*sZ)X-tzJ#I zz-tS<0#4YAA*|KOR7;{=HJXFQG@nZ@r0#u29xDwCpwC?7~!#GkVK`0SYO;*4+=DHdZ+y%AuPxUh`nc*lt4mq<8cDn6lP<#Eyfj#8Y$dlG(-wVM z@&PN~jM?%6$7&E5)7xb~xJK7E z=j?9bxL4>HUy(zkpl9${BMsjd@Gdx7hU{bml(HyoO}=XuqyKG*9HOPAJ~2s`U#$_< z@}9Ye`I8e#Vz9@8;btQ_PuMN&o!Mq(Y(^SeaWA<7y959)@-%h5B%bWflGT3fbd&tCqJFlO`g^oe+&JhOX+#F9J$^Qt>Q-%vGTEDJ!k#ztNgQsNTs$O=WRILcL3e& z(?pm1?SydiQvK!~Atzrjh9D4)1Kc9Kzlxa)OQJYk#JfT~#{z=xzc;v9ZRs5e4AhW;RNY8FBSeJzdpu>lqhw=NM8cjl`kodu z`G3Wyya2>+Wjf?Y)(XvJD76w79p?X%gvfB}`8L%2I_P#n8~+$XyTm%eLt->R`-jN? zP2epZ9<{h^w(D2vWD@?1Cu5+IeSNwR?}h6`3&pk42tggode!^a-)4Ll9swofYCYwq z{^o=DQyQ-(Q}N<~nef5bAHl))75!K}oMR)Uc4X({@&#vIxz&(ufCPI60Q5X$2Jkwy zt(Lnu#JbF*+je2ma?MfPtCu*EY0$ygI0{S#5THS1z1&(QVZ(sUCxZ~txo#i%q}-fUUt%dflI%~y(Ai&wffZ55WX@aTG_ ze{2AFbEZdy<`%zG`aw+Gn&)K!`t{77P1D;CSYNeOR4Tqy@rlTC;@#NCti>1e^VOEa zf>WJHxst*~3L5dQ-&RVVmkc0EnL@Ku;%a9WCare~FV$DQK&v$(*3sUtHXwA>&Ms9a zOJBMfznh@y?h!lE!S=^suuFZgNZ+x?hyi>VmZ1F!_TTs|aruyQ;M?glwg*_(Qf@vK zD zuM|t4BaxNUXaXZdiv&!^;AL@1e<0D5O6lQ(*#y8MY(FJnBD!Wp6nEOb|45$a`7rtPIMpyiNU~^5ZEmGS5AjZnSiNzf7Qv$_Ac&hSb6c{+Yh- zjq82726tMIKbPFLfhvILc{a^5O2Xj^VCJ1UFy0{FVe?-kul>jB^P?UC_G3!nvX{r- zDo*!Mt6)Zm9wBpj8t+Jj$a`lvDS~{qlkQD7Xo{9D%Igp6!<2L!)_9Ueo2mFQuD~9i zmUXIO^6PVxBlYF=gJ_(xI|(Q-%ldKC-&j4&D${>d z$N%g$I)~9XJPa&oDbJ;;Sp){e^(gtDeJQ_ZWP3A@n$N!<&{I#WRS@O9ltN>L9hnYa z-eHb?BC*{|jAXlSrJLE*^DQ}9Zm?{+>-K(~hM{S)O(*A>4I7MqqlLvmo(1b#HfSu%jJc&?l&#>yE z7fJmdRp#*OCg4g8CG(kE*?TY5-~1Y}e~GID5E*}%Wc!5wdc}g$a6zX*O1xp9M|~uu z2Pn3ZF-w(4FSwyNU9SX-QgIa)iDz(?^xpXfRuRd33{Xps8o< zIE{3(|u$9so|>&fAVs(i3Z|<*~ZNjTeF3NLQ3%eh$0uu>Hik0yXGW}DKx^{%*4HQTGD#erY0EL>SBAF<=*$RAC52xksTm&1dVFvIeS z<5+muNXg$-HI&GE##n^X2A@4aIKv;M9RDdKX}Ww*~E zhf`}k+77;g+|jcv|7>I*LvDmk12j)HovT?Z{u0@oB2$d3*i5xjx+H32<7j{0EXnxL zO+l>DbP=%{Fmy(ryk_Rq9Z+u0HA}&BeirauJ+U@f=PA6R=t{xKNYR@_4 zg5MTx^9G^wn8lw(J{`=e&)5K-ZS@(q-{#v>GZc=A6N|k-X>8W)zfO2 zvoU;0P{NMGcl37+5HipIJpSy63i=Sg)_L_ZY_(KZypBxuqs%!nO`a2P0AJ0Dv{=^l z`JtWo$_RzI;<&ALf?DC0bMS%{&ZHs9mnk9Ci{h7BYt)QROQMsu6V=8sqUmQVL4cj( z^dIx$hQ(DfezH(k%Oa$`7Oc0H@I7aF;a2BCWncFX9&>dims2e7dwmOc9Sz1vWBS_- z0yBRigtN#tlR@KxuV*sLoUa?`C42+AlEx=OpC4GI3lcXhBy6N?n3=71EQ8$p}>^^XSa~n%Ia3U#-DFN4STr5nIJSO9{2{3 z>FIf)%O;guHZ+bN?jnbM4t@kCeA6X2VSeEAgb(ziMqgmYVhSM(Z|jH>;xy1vlds|? ze`|F;3kUx`rQ2(Hfi-U)tO^KDVSGo{*taD>&=_wq*}X<=>uo3-*ay*YC4YZ2IAS=I zkVlv#m(R?UsqF?5Cf1YmKXMm$FEkt-ULXGda|nn8>$dhFV6;5@O&cNWwQT2 zqG#Ix74FzImQ*19?JLKn`(qmzU>G;GKi|BUw@jc=HB^cI=EF*LJ9l}X^}^JraWk`C z*XJ@B_1sf6iMcAx?ZabrTRd*}M@}i_RZG#PunvZ6JR;VR3Il>_9+p0&u_@IxTKEYeM@` zUr=gi4l#f>SC{h))aaQf`$8}V(xuU+PA+6ulSVZB-IlfiiFm@8#t)I7CnurrT-c`q z?T_tLqs!S;i8=b~*c?A0ss;~4nkqsJNX*!8CrP5*_La5{VgCH6%J^_O=h0veyvRh8 zwOBxW@rIr&!!-GbVRD?*$?!*z*m7l<6!0NLddp}+;GpbLf+%4v_y*bFqxI#Mc0`ui zEr0G+>hXpl_wh8ee4P;fMB-)M`nliCQ4TYe{B zjCt`xZu&#t0VWo+?DqECBXtat844!k{fPN{vkCAvwJv>DBazqM3j-mi8XQ(k9Tzqc zvD(|g;X5#fChA5$`&uhoI4^XRw;PS-hE~~Jk3?_5z~dK&`kw;D zebIjt9afPHN32q2=1#a6H4|XMMPxfioIP3)S%!o4{Jm$){9>P5iEl*A`o2~nw{Kai z;!TP?B}8aVKI2+rVW;-IR>CBHe$RUzF1-V-TTFx^m;}zb0`iF{PLp~fiR*2`Cw#C8nO?Nox%2Ko^l8JaS5a;$zapjeFJ#j4c z?dzaOr(nTy?C{st)sGH82H3Nn4<|TNME-lswnx8jEfa8{mk?@Umh@|PeA;XBbkKFH zOZx1*wv=&fxHr;eVeopqk(WSr@f6OiKC)oM6xLw&=X^6twel|u1|QECS018tnrO!3 zeCRY|oAc>T7B^FOH0YHwMALk`)$ui)bNaNaV}wr#2)z&0a2 zB}}Pi`q}fNr`+1y^+na9Pcc;(Y z-Br+l`iUdnNuCqrD|2l@-w;)uWyYv{|YmXF%QyLNo!7QP-(2n%w)?g%lJ^BQa3-1{KuS&^v^c+5{tg@dcs5vto~bwVD( zh}>j?9@|2H-_uW9{JB=;3m7^6GP}{sRqXZK5vTwrjAee*?}<#ts1Hrau~hJHZ)6WO zgR?I=_CBzjmqEfyizP9H)T7|6L;&sBrw-MA>1VeYCHu(PoO4ZF}pmdMoGo@LC3Bl_SmSXBvLWlDLN%qY?K<16k2twjQKXm7j&MpY`y zv|8du6*_=ey_&SRIj$!^l{Ss9MhYlcTYvx{XM zVz$~lEl%85kGi^Z4JHuUO~p7KX!;R`G-2Dd7dd>>Z#Ece*{kt4c5s35q>TBO} zzpXprpIJR{AnTGH7&V@6b8yEvOlg?DP9HYLd)Teg9000!Q8{ohUraQ^0&# zz@j=Vj$VF()=={iqS~HXJroGm9wQ>QuPdeGT5AbsZ%95j@ltX6AW9(0MRrSPR-#4y zAKBD$_$sXdO@1OF1p?GrWmN=$h~rgyf>X^PlO4Cm@yZQ9PhG>`hR2p&ogA+YEaLKZ zE0_1^1E9yr@y2xS%Ff$JMm%TDvim=3DDZe%9k#@`ZVvDf(FuNLiOb(2Lu8X8Lp%BXpifKuJw^ zV)o=IC%DoEiG|nQ2pV~x7d)2epfqgw3QrW4W(D^PQ>&NN8O;Cpa{tU+4qx(Bs~}qs zudbjgV=(-M1EYuJz%yehvpRtDu=g5}ny)L^pjaE@e=5WWI`A_J6+`gi)^%8Y5J^{R zCY-a+Ub0^b<?Pw20(eQgriolzwfU5~?3Xz*1-^juBYSz!{6#KPvG--mYl8ol6L#ACfr#MdV8RwihdQjaTIq9x-4D_2%;)8of>oL- zkCv3%D86NN{aTS;50W;-HrHntXlC7rJ1@t*3_G=*IiVT}6CeuC1Ug6(kO9W5_$ZiVuBt2}Oi64P4Y+P{R6RNFEw z!Sg+Tw7;lwL$3XXSb>-63N%R|d7gs4S||9C;V#g?c4v)&%3?t(sGI0LUn(k?b zZR%>bojKcyPgEk+t8=_PK%(7t!*PR`A=sYM}QMGw^(I!jKKE@pGFG z`zS0!q)U@-KJn*!)=#|#Asbu!(|KlD$)3?Ne0$~(5QIv*3W_v)Qk2(z_!qi$O1CD@ ze^X&=^L@IVS(IVB(i~cHHb3?Ew$CWiU+;!8S$Nzqw$y1Sm>PIbsG809OwqVMT=1yY zfjwyz_88_MGfewzRG#%g7|-gFnK#Ei)z)&Jzfd$(Z%rtQ0qBffJTd2cZN#u18uTg$ zxt2OimY`f5LN&Q>d-+^edUDZsJN~`0_{1z@YH*rcfmoPui!s%f??wjP)9HLU&FB5D z1|6}uhE7B6SkAcvCV#d@%PLdPuDJ%I9D*9Y$dR0Gw&dRAwn)t4jj=(?(PxoBOfiGSFD>t3tB6~eW=W*(9&C8hwpBDk3=3fsDEeRu~(8C>-Pv-W+i z!SkX4!Hl{ai1*Q>&3z`>-Qi{@3(Rv0=~C|`$p(QJN`<7 zesgE39ZOm#t2-|=nCu>rBdPu7cW*to09Rm z!jB>AhG}zx<28RY8K|Enw5A;^OUXHxjK9Pe%A#N86qRXxu=j@+yr)|epHcJtL^uw3 ztgrtBJM|L)=Ysq79_X|n36YU(K?b7Cq%EdYY$`!}Hgi_s#L#n@EWW|XqS7qU$C9US z`Mo}U*9*v{ejgA)H#X1LX3|mn)<{AM@o+Po+l~z znQvLw)WJQRuAKQ>owx;G2N6J>d+GIQX`19`WzvqqZeICGF2SuC=>WZ!fdjpN;1~Z* z+!BZO=H(YG7HAV2d+93sQs_DSrgq2KVq;7&O8X9FCNw<{kuz?yS?3`m;$wS-@AGo! z^3-;s@no!?fLe@STs8cm6JcuIuU)k7VaE860-(I-RY@L`#OoS)M(1yPFMfOnb>YcI zfN7S&Ru}vX*@c`<|BNCHkF2B%2{I7ej(gh#}@P^^WrWe22J}w?0a~NpsIV5EbDN$|pRg?jnF7zIh ziJj1g@m$suR5PX2ksOZ*w<>}4C%tTGn?s(IzS0Ff8iBuZYMQiu+yi8v9T+uyO9XjA ziny{c@XjKr_?m(9&bt9Gnso~3ZS|REM!3S=`CgJ<;ckfA7atbW@%xK|BX)+pQ2}d9 zE#F7&DyLlTFOc|I$4?@-zmNQ4iph=n!*$&0veOMouEZ>^O!7K=a#3`L5Hk)2Njj6a z^9~TW`!B5;I1PGMr}9hb%JjMYVc}IWNtMokg5}EmfXM?;C(S_ZAbewpg*|CrB1W!Q zVBKZo5s+-(`XxZg;XnaX8q)2q$_#p?EJ*P;0}lvQ*_l3 zP|;js!jWphhf71}jnTNB?$vQGlvh^Sg5-Z8gK!2|>h9F5&U}lR7-wUvUb|gmcPY9# zW4zj!PqX@-M(Xl~Hd{F0#Fd07)LcFibFC0MR7uy}sVTB@_{lx+&Rls8!yrUdt!{2i zrO1fJ&2nr}B)|K>>_dHlxzniFmDfkZr)4TuN9JMy`qJPOR?kvw!Bh4L@J$4_rXe^H zg)$>___puX<00aY1$ZC_PwOh1o)x?)yK_z;&)#;`)=fUeT6|r$eM&UScWIpHOSgFe zdwd{-*P}|X|70hU95oy8+`G1g78ma4LAMm@_1J{?7Exyj;9Ff;oD-P%ZXSO~>DT4? zVPwFe%g-y8dAcS@IWSAYoM8#-aUXYKIE8TagI9nK zA9`HWW^U@c^ZvZZRF;jl^Riq>6V7gzMNOt5Ol1|*`UvvWA3?*}Z>}8Km+|#_ zfr8>n|Mm$EhV{JADWBBStUr-gF@!CaEkoCc>8^H4j2&O*XeBkpwPZ+@GFHuSMWhaC z9@qA#)zNKeJP%La@|#|)k^PV8P0intr_q!O3XQ@C)|0*LHy#$Rv07{KaVsS+5LIEC z*!vH(7U?!ynOb;+!j3jW1s?~QUkKu~XtLK_ppxTA3OFNAG}<}^ zTzq_gUrr4vkE_Lg(W31nX%`B6bQ7sibIIbyf^E0stY2SP<`?9jPi>`b5u07wDZ)Hi z5onz`bqEH1muL5sviGGMfS7hP_=WAK^1%ZABt`mFfC>k?IQxz{w=nN^TLx{c~DL8d{8!Q(OE|K4>v6cAcSq;NqgfTeMG7l)uy9GF0F$aG+Ap`zLf^ zIvH<M?@H*(((*F@IuYMqva_4FaV9S{9Fa5 z@YfdXEpsIr@cTs7QNQ=miri0n+saA(YL-J1BD30j*>F2PIq^uin)+Hlckc5FT)P zin&cI(S*z`d^F$esVvPD_Z-}Bu?;BMy2S|`!F57_7w1|(gjnNRjsT|mhxc;RIgngFv&YBQ`# z^cqGpw`g_dfPVKC8`rm0eIk#JwK8FF8kiyYHG!5vRkOra*ZtYTNlc4{zGvkQ`^qGN zrB&ITDTGONHaX?CFncdBW8Ei0|J-`zY%as28>o)+7tC7q{hPeFHF3uJ)pX#Q)tq~Q zuUrKHN&SJ=eAi@4+;IrmNcpIhoM3`QNE5GspQsC_t0g^mSEPWJl32xyTHNH~3jW8^ zS%)?C|L-3}K|w%}P6Y%dr5jYFOHdd+B_t+Y69z*N>8`hQBRPrDFd6}o4(W~VjTj^S zp3m?4{c-+1*RJcFz0T`(Kkxf~bOo|a9JpNFbEL)x%caKHVBQ_Zfw!$xiwK-uITVlT z;k2QRlT{QA>C3*J_dT!5X5til^Pr3`8pcolOF&emT(yTduw?`F%HLIu$`6DFix-~F zxw>}juM}|9VWLYt}b`96q@1$?*vNWGNPdIo}t<6YlgLu)jItdUa;=Wn&U=LDCVCJL38#4#{y#AWB z=&%HPiL$rr2GRT5gh_lh8Jq(4dg5W9RAmi*SDKbhfp1ur?|r9*SBHYTm7?Y|NB`_8 z85zX4{wcYzZzr7LW^D%z<-<>_n?7~a3im}#+}F2U36fC+c;GabYFE4EHRCHQyS z<+-1o_wyI&8sz7{hkpo_z3gv;lIDqAJ?7wQ8kHC-&d_1_M-=D@0QaEx{WS-pPe!cd z-+3(kZ_;zY_pNI0gS65k`-W8YWX>8hrW>5lY|vm-HcD$HQEzO^S4G!&v+_!uu+G=3 z&#GQTDA8=1;52dn(e)&LaO-EIvS{D)4i9Lbo7~KnesCR%0xGW=_O?D~PMuCn3~|w8 zoV*e+Dxc7whsY9o5`5gpz{1#lt@zkd?1O#4kL8!ug$<$wpWWPpIoD+=>xd_05Dl%O z#g0dFlC5s6ZU`RX|44*JlVY>PL}LXaQHj-yFHf3AdzG)a61qz@E{#Ev&&NvVVy`@B zf@j%g42rkU9`wtt1mIJ&Y;MdvgNgSEZ@n5? z)I2~yN0(PV!}MyK_;IkuU-nq}Ycl#%bsz~HCZ|_~HaP{;Zu?K;^%t(tc2A(Jo-rs3 z{Jv$~&#Qx9=<@J2WoZS?g5k2)F0Sgs(Uc2`L}z;fy{;z-w{%&%DiVo1RvpI+Dxv*~P2Y(&rypvu+Fd0esmXXGsMU%;upcjX6T`2!nee|xg%MFG9kFkQc z&1#C_zzN13o3^)%%fPQ*Bara>Fd@_SPMh*NO-~PP_1hZm-1|gNiQ>}O@Cf~9`I{D) zVg4WJ{pxLSjmXtQaSWpO;~0Pj1HX-h2<#?#|IlSkH8;6<@7#Bf%@e$c+UviRATZ+X z2w*$xWv34s7M`praKAO0R9f@^GJ>)Z`74iZ*vrB8&2(|Fyi~mDm`QU{{;V*24g?Qh z&b2R0*u{RlE3of&lF98T-+sw?C#Hu#@p`V#_*bVh$LA!bbI1lF22M*BGN(*R(35PQ zLA={DbW6p>wF!pR??TQI(fl^gNDy zygvUzY3I>DB83?+NnOcspstNt8vllWx=|_IN1laYhP>gHsjdmmKFUjo>@`q!%GIdt zFE1hdXOoE7VehOe+F9tVEH(lxw@bEA&^(OR6vSeT#O{IWm3p6@n>;`|8GM zB)p)ZM)PGF!#hy79h$#YlOho40q3o-%zZ?;!b`L=EqKSd!m*UT!iK{_pi|})p;T!V zW+&&P5Yi1;BU)K1Sc-pQ1R$w@)|uV}P$PYTtcoS}X5JufQ0a4o*b;~@^1@XGYur+` zg`3?KS?~PP1VZ}h#b9%Eu<_2;OVi9=Q&CScZ;s&&h;hJ7j!cq%(;j{`t*!rH*Jm6R zw3nt=-|z(14_*Qk@Q5L@h`QbHNJ~A(xp!>J+@vTlj2l~tyygPaEV|SjFV^>Sg+B3T zBe#zPcCtVR&W@k93LwmTpC1)4mtG7@?7QJX(w|EdcztD&CB7ZAMnsfOK8rJ(VxW9=r_1x`Q zsUb@2{i;+2tP*_<>1Hq{o*!JXr<{?@@LmOz{~1?N;P`l`mvSn!yvK>y5iIUE~!wW@M0m63i&x|m9wMVk`#wjrm*1I>i&I1cQuPg*y@K_ z-I-ooLq_axM6bf&A`5 zE#DFV`Kv^AG74)E8UHPl!1)LmfOP*ms*PxE$j{ViAJo#lE@aP1sv zL*Z^}xo7e3Dkj_aSB0SS6Z3A*A~5Ogf)X7!)xh+?fn}_ra)k12L>~4qhW*r%>Ah{{ z02BU7pgGI-MAlB@rn^!^xGrCtFN20HZ~x>Ukqja^P$_WmhJ-Nw0T&ipob(E(mc^H4 zU;3AmYh%f1?ZHs6=)Q3v@y+xyWxI@nsFM{-YpmP5k76&pm9;K>_k84@h;R*m{i)||r>e|oXz98V9_U6qE z&>F5QzM|Z(JbT&ah}|!8b?5wn3 z6LU!SA94Q*^-sjuTK2NROV9g}gWZopIxpT}rlWhyX@|YSN_WSb=DTA0%~IFr3I6V; zNje3wHz`sqq9Q>8i2`x$6RXLzWM&5&uf4nP8w>svV56FH&^e#zQ>Hqp4CFgrWeq>U zpp*i60!Py0!Vcy^*a1+36 z&2OCQ&u?fd68a9wS=HjdKPX*H)i3YAxE9!pi$vWz0n@%Z5;W&wXY#%=!nU`e;mlX! z&F&J;-nKEZ^5h$C4m;6FaIYH zE=!2vb#F4!RquRk3ZOo+f~N#|;?Nt3M__0eIKg&ugFP@XA=n?T-&dQN)6G5zyGBGdyv}sVeUB_oGxa0qn~-7FmpnQ-9=l)U^hE`04H&-$qF(C z^yMsr9r6oh9rw3y1Puh3#u4H3 z^L#)?;hSCx!q29m!L(C&WPZHtZTmb8xlh~gzEo}9aFCk|rc`f4A!DiZa}DEyKGHIj z;xvEyVamy4lQ%x)qbk|%2KlaTj75%IE34y4HqqL5#iU}VeC`n0F#q7D`L=JS(a#@3 zHdpg$%iwjFBFf7+S~Hof1Zn!^fl8)zC`F6H`*%`K33tOcADxt-cYKLras&89g!P>= zQf&TB-H`Mv@=F?0=?eR}>*wBzeER<`%1Uvg#tA#^3{!GBwrF>--xbYuvCB#knHFvQ zF-GJ43$m*VP}j%L@34q;fkmZSlGqq;my$N1QmBE2Eb2~kb>ha-t0H~evu%QP`#M1t zC$|8uPYMj4d!`5SWl~DPZFZSMv=1Lj72u&&c1bQK(b4{(@jyi^todUOXXIW!c(Kuu z1}YEvUBH_rx~0cX-Gqtzt(vOza?Mz(6rGW!GMqi_$(nFf1s#UYbmv6;v@o5o_?>6R zYUH;Pc_BQvT@+zU5j3?39cC)T3>xFjQl?|Ba{>ozLi^TFdq2X&u>}ODoh`^i=7d=~ z_VS7jd6QFm6YgGl-PV0T;58QO?yb9IQL9L--k(lS_TVM%S}pQy&Po{`)MkF1s>eu@ zTXl=l*p7Lzqq5wUp|bjQ=F!5_(@I&thrd1Qb0|*aDJ^#0=B?LbwS9S87(TNg%f%3ZHxIHVdVEz zAU2?2UC%w<`V3A4PJ9z~+djmKT{*l;>B&4}ZR?S?+nY?#%VL7ns#?9d`TKFe?rI*L z?qqc)A{<mLQCJ z(Edi8j%I$ieHzJh`<+NdkikD99dDZv`q??C;Rc38XixpNt^W*n@4$6Z(TDLJzs;U3 zxdU|jJX#yg&%Yl09KqI1h@o@qCv9ay)`Z^n?R8IIYzS{vhHq|DjrFNX0hAYtJSR9d zes{QFr@Y4QPUE3PQYFI1VO+fcn7h@{q3j?+PrtrGiNf0~)w{s-gt?7%?{Gt-XvD$L zt-RU=Q0l_XP;*WXyPBz&?ko9Mgt;p5ez?}>ah{6(_v~wk3tMY$OsrdZypCOVlU${+Ts8w0ihs_Z0<9en#0FUb3Ce4Q=Uj zU2`0`sS-q&aQ_6lMB7s>s@0>I#V1>JF}t!S+um(Ht~Czzn~78$9;mK%;Az67n@jqm zI!g)Pz*Of@P&V5cdFMT+3Z%W`>S2^zT8}K6V1upU{OKT-^Yk6-OOoj7v#O}> z|5;u*ODa>d#l_*3@uxO798*Ah05EK(lyJOoIB#w_vjG*g4pf%DHBb}JIuRV=*7#hW z?vkTa;8BlV@fGwV;;;H{=lMf_t?vc9rXtZPf{%d2`OT}=0ut}DBmIYe0Ng6onK z#SC}}2%O(?H+~!{-{}GSE$Z$je@X2`Xx_n?+KK)7%d?iWRHcyE=xF=ozaz~u(DTBC zTmOijR{0F;L`C51*JMeNUeEisME4EUon1PZemycSs{fT)bKgBOA^&=6^3b2|SKteu zl>HEMIrcH+H218MWe4!Eo*(UfXDti)#WA(Z@ROjgU(fPYjF~6JCn{mE-6oI8DMs{wl4H3~ebLbkgWr_gCR9X*A#M(+yrt9Qrr-m13~UTITMQ11`Fa5)Zt z$Le{402FdoN_9|_<@Bq<@UVf1aLPCfqbMO(JtWq$sU}L=GN_-(hX^as4U=9`rrkx+ zxk|QN?*D!BlcbTKbVr%m372lDyELtcgdP>fi4iU_F)FZIdjhCy z!1rZ8j|%_Uo4+dsymr=(=4<{5q2l!(W&J-oD)s50>RlWR8T*F!Ts9j;?(}xwzcb>k z40Jw@Rn$=mbdu;`|CA`uT9za$^}&67vYQlvejiKG9*;L~O7ceD-Y+swhmR{N_bENc zZiZY@0Hzt=ueG+dFR+JEUl>Tom6Zh6L$R-oFLWyx)`g#hM=+(%(ZO5B+lr4>s~cmr z7hNH_^6i1du4d;Pnn-6zPZ}PS*1z}MyWW7>qJ622dReCM8+^gw|@zotyDx+5hxT*t;$X$yHRZ%uK z>47%zp$Bvo+xr2ET4}q*1{&hOpAc*snpdi0W2Z>{*u;PlLlr z>+rKj)o+n{0ev#u#R7j_>jEgy)4}ziTdcnQL)E@ABRdc+lhWR&ou8heKk%+|AAjt} zyI>I^d%V+<%)Sd^y@Cc0;8O#k-pou-G6=#IA|Jp`IPBB zovkMjQbytUq7~#ZtdL5o-s#VR`NTXQHoi;8W`yZjmtb?gNyLS7uzj0Gzl!p$uoC>wvY^xsI4DthQJLhc6P?}`yJhRl~h;<0wSrJ}XJT4rHN1QMAlkxZ&hLJ_hnPI* zR}Rv*5Al_eC2KiYLEBpZS}k5O@5t~XiGHqr>ij?9sZSTazbZa38|CoOOsCuW7{^nV>| z_Q@`g=jfgGno`pB)%$(3(a2{t6Gw)W$d1@-TOdcGOUAzy**l8NaW}u3Uwoa&aIunk z8F{j*q>&8?lUIlWKn3YvpSKd5JmXohxzpC-ABh__eB9!9au|7%@yqtew)668qj8`z z!ZK$w@@Pl1-mfk-X*#%|~_s6B4@o`tv zAFKEI*T95+lw_>!g&lqw-9$A29NZHnVL~S^L~UAfOOjn$-#m~gmS?@t^AE6S|G1&r zk$$+v_h$N2Au`*JwNX~)lnKEAGzefQxd=|#j0Df?+5oy8rFp^XJEo#QC#TZR>MZCl z$V!?pU!me^w}QIAood$mdkLH&wBUKXm5{r?<95d-Z`h&N$$E1z!=DY-1C0XLVbwwk zccka~ikF*TUVb&XU}oihOlE;WIVsx!{*_KtEj*NS?@iA|d;U$p`Y%q2*rl70p9ucz zoEv%|T5$&eQQRx(xi|3^XO%}=G&TCBEW9;*)!^(6$Kr_OX;liMS0cmO=Bj+ZrnO!M zx_G5Sh@O}5UginB9l&ZYHo!+l-vM*D&XrIOUSgCcx#oM@$i-@I((d>qrP1h(@-Ku4 z;R_C-La;zc@$TFc$j%S zkt=(sSvxtWD6gr)kY;}tVC#CwX;$e@S=rWzZU1&2`LEUXy)F zV0XLX7q>_0v)z~@c9X2E_-ONe4GKKh@51c>4Uy#)A-HRFQTGA!$h*^Czx|S+x^il= zIJTY37hhvKfRyu(hQM1aOX+jwyklHPQ|H>3-kdb1R}%W6OAHBKIK}b8vygRG&xyI# zbnhbL{3u0EJEpEp*F0$4>nX3lb=OKX+^ZC|>>8`-TbQ~Gz3~FMmYajeXJq<=t#K>H zVgHE8mpi4>T^gP@HO-0hXU+Fdw-s2-)SU|vJsY?=qu<0pG)L0%cw| z71n#csK>ul8!&Co`LJ*G3|D~9BiLOkLA`}xpa8bxmfI!+;J zM2`o`Ue~|>AT1Q`m_td}d@du_5RS696fN`h>80pLg0b->~>c;q3$X;x{T?Iea3rO*xKj(=Q)k?l0o`wVc}E65(&s5y0*)V^~0M84u=F$!P@i(^l;~r}jHpC6mBE zy3Y@^^^Yi4N!|)@@T=N6;>)ArlG%dFbKqFS$YBHu=hLe142bW2Y7kyG-eWH3Rv!*b zO%Z76tX|mbZERT97K%nf$X!2;r7)5pG$C19zLkA<%G1ll+~Rqs;!A`M?rblZRW)+A zlYGbck>1j}YG5E?w+CPaI{P#$N4!s9Vh^gx^!K!k56=E6%Z|ozSKtmirrQ%v+385h za?;vhz#Y>Y!t5cC@W`@ESXY~pax4eexc=X%X^jk~&B!g0R`=pvQ7)&SWFQg+-^#oZ zc9!R9JpXg^pu_s~*{+`8ZkwG2BZ}%0x$YUs824KXTrgk9!&iQ^v*S>ITg7o42E{u; zp|sL8O^A&WKzngW{#C{z1(%%n+Q@<1ehCZ)u_+V6RVcJtA0s89qNz@+? z@n3>YIxnSe=5nZ&q@fU3U6cXE zhB_ClR{hbI(YL$(NZ8`a;anX4zw2mkBzC!vf;ESZproAQGNx^5{^RECtWQz)VrJ%o zq0QNoYst~@+yf!=2cQRgqEX@LzIgqp$}!sy4WHQ|Chw8~`bqxx6Xnk3IchYxf0?2e zlM}|03Hqr)YGl5tb|NrIv!&|SI$HiA4GAPv%JHvf+Ua)HF(P~Cr9=74Ew1tAne%#N zFS)#wt?mMjJJw6MV-lqL>iKo9P)7H~2CT8l)=4Qi{Kd9+j%xnn?8|eKYf_5laKSxQ+FN5d7Gm& z3@|bEg!s&jgs|>S1e^R=VZr)~usB(9w%qjKqIsD?anDaAFGVUG+$>2Fz|XIA2~Y9~ zWpf$_`+fRq2LksD_MBPTMv|QmR@%b#T{@G`=#^pQM*w)7K-SNB?~r9v$Hv>5dSU0q zNv|W+razT~i;L%aa*`02}z_Ugv+mXFSS%>N>;Nj)1t6P%_ zp5;S0&wdC#IM#}5RhH)nE_bytbs9B7<3v$9^pKYIS4~6zqFx8_dHPKX+zx&h5e&D# z<{d}-{GN^wY_&;a{t;Pck-ay5=A=lSTX)`DXr&q}KEW<9F1X5S$j25hZe?F!uKxMf zPv^q3s--UT7+_O05liSeH=#;NlK2RY)sdP~w(a{^|c{A?g0S!U(g$rWX+6o&vRV$wf$R{Yqq2cD|NYf#whjduw&m2m5xcX0llt7c9SC&rfc&G3Iqhs2xysX8IW=tF3@Oqv`AI67Riv?j^I@ z=6<9$ruVDcg<5MU`=Bo}E77L<=)O<+7&eSvKYw+)*w$6nK%a>;r7z|eP@SblOVyoV z(vXG~8hqWRF8L#^#knX}q?J@zZnZV(tPlFbpvfaOvL8bAW66qtJ!a3);%iDUe)CI+ z#iJJ0b5H%tkL&WX64(Ewn6vYRQ)SWRv&&xffH8ey0d1vbINOFd-Mmfrgzc?cX6`^S zNVK)hZTCo7!NfcAHJDsMDkZZ9cL`(1hL3Qql1(-n`Xb``_Qx*mXKgPFMXJF%a|gEE zL_i8Nh|b-Zc6xp-mdzHOEr~jH zCEpRIAsx2>TJ>IH;5vXvMk0aX#3&j4KWX%!@t3EN&ng9ND z#1l{5$$7t39T=8dbqJkpk0}v6Jj~h=qu!rgR*nun97VdW z*i!Y~d{>WrBGZ9sb$GL$lCUe+Iw0ku={E2fUsh|S`r!IFbVLVOl*)TqkkRG7C0Yu3 zXQ{1kjdA)1q7s)g&hiPXWp*uegoX}{t$?$54|WUk7Sa8#(G?uyO;S7AdA{bE&E zi#sOE1s5%6R;#A&w5=dp0`YwyjX&DgD>rTO(r7%-o~nRZ)lVI;^o^OP^r5?E1j?LJ zoB6&Sh;S1}J(gjobTiBJPunx|2<7Ziwcl|S8jf;Ik{)Vs9Y_2{GW?}7`^zCz40|(u zGSvCQWQhs;cRpJwla($uNGS?)SdG2s}EQX!?rAM^aG=5BiG!Y6Uhp z{eiNF({7vfwO@c^*F@E&FLJxdmF12~G5macfB*6uG>9p`1mlZwvF$fHg10!H-!U8y z6l?We?KW$^*R$U;sE{2X09V1nyBe9AExCYt6ilmU4~yVjSX|+x0<%kEhQkZ7q#*ad z5UoFTMhE;_)GB*Vl^(&Q#xM~17O4pQks_rZ_^YJ;Ey5i7(}`zJ8sByL6&iQDU8MF1 zaq7?ey_k4#GgjUfR00zs3>6S8=LWTX>crSK8}TDWulX&1)H$H-e#eapfmTp}-CyVt zYjuAStQtpwhvMj~YAH9r<{57udOeC;x2USUdy~P@j}x&(NxY{{Q1pH~4AiqJjfCs!i?w5t+&; z%3HVFT^FL9EbV&@2lBSh1kYuBM&M3_#|IyqJ$W_-10Xv`(Rn;&MpohHSKVSQZpE%( z2;*GL{4!X}F6EE5G|#B(YJJ#n{2IQ3w8ZqbLTg8JzkB=-bc594$p$53DCX#RoIF^t&57?-#W+nnT`xXG5z7aAHAdvWP2#3T=nS zQd?h{t1JVzZg!^WclyZpmMPWV!m8DyXITf3{sH0|LCK&w57DObL)TC^_?^EHE%Bgds}e?|wa!7TuiH+o{Jwb@B|pXw*_>+6E1bv)J0 z=`pmz9vPEg5uo;vY%gnVgr6{gVY(t)`HX~+9QDz^B};KNTW=L#ot2Cu-)_*D7jX@= z=<};)9{~<|?6?t(R9-1`X$DhAqFPxBML)w`h3IH$3yo| zB)4y9u}dKG?fsEMLa$3`xW6fq~`xjl8 z?B@Uj6LPJtT3XJ!G<#j`0oAPwlwIZb6`XHW)f#raFO%;f6JDd>?1Nlpy$rr0hrG97 zTi{OU`dO0D^0W8K;n_{D+hFr8+698}pJla>p+x!GP;dEq*DaKuKxA2Mvm&`zbM>!T zXgB&WNNGSM%H>0_Kh1x}$|R2 zhYwclF4y`7OYRQk&_NnV{qvW%x_usSG-29n+p*H!%J=?mFVQwltX|6ZRsXsT{WC=- z-?q!?c=zwAB;Lc%(dHQV9(IdFsQ2KN=o+W2>&Fa<<(7DE_j~g~jgw0GIHimHK;f;v z6$E?3xEE7iwsq5(W^c&HkgtCVR7x-$-yP*YsTej+wc{G~OeXW>BgQ`o2YG z_si%F{V}*ZCtR*5XbMKpmpbFwdv^kp618s|K|mpOdnci2i5nGg!Gkdac|qQSUV$rESI>~Z_%mv6RF6bg*qzUZM}CvEx$$HI@f zDLv`F5Lp1q(bFMe@$EU(9ZpPr9uBl>oUnQrX($PlJC3pr5e8z_$X}3A^noVH+sX(b zz5j?<+n+2zuO?36d!|eh zX%0zd`V%9j**dM}X06k5>50{@06)b@|zTIa3%uIb9J z!Mc9Akdqb*`K4@~Gn+nxe@6gIML+;wqXR@P>m)s9@=wOQ)?CsT_xK!0qO*~w$j#8N zfxIjo4KewRuOnZIJd$`_BNiKy`1!&Q^lK(MpkRIWBAqsH_?e3ri%)#2Z<_er?{a)D zh4AkWU8}zAo=Xid;wqD!`WY_LBN6VTp;X={Ok%#|9c-Kiz4i+3=Q+T6-vib{M#8D@ zGP&zm5pz*i=QF`Tu^}_-@wrnbKcn_uc{#gJ!DqbNuJTf!2bHLmC)=2cKR&~jrbTI! zBu@#oYS~_?l;Rgx(&rS>N$Gm zn$GssZqFw>j9vA;)emxjO!be*^Y_T&zC|Dc{?Ij?Chp~=KZx$nFE-9_Z~6#U-^IXt zCz9(&1RGF0qoJjWJA3|pdG+8L?%3T0>hXqFP`UZmsH!AJqGfsT&0QjKuTLDgexN_~ za#u=9IQHw}?*V0h=S?xPsUq>e{poH$c&T86_yEj>hBVTp@{x+&CV z<`_}7o2wAUOP>Jx9WDLw!`Xjlip}H~dT5&s)BggLyV{5N2CtI?ii{2e)%!MOOOSzm zaa|6oKNJdvlaCE|l%(sh#VNwIKN@52f_MNMu*L6{`L44h_JmhOSuJEJ zx9?ed`C$7?hcenk4!$>nxmH0hw%58k9k>*-R-e_!W|FU!zdew1oUjL7k~y;2M__W^ zCXTBDjX-uYRn#tu;8%8U!Jo; z10FnwIL3yMO3pnNS{+It26*Xrn>$x=_w?-_f-i3gU)V|3D@_}VQp3Q1L@okdr|f)s zwNsmsy;}LzOfJZ^S9L4rshgkj4vw;ki>`zw6l8Nd^S5>ZqY@5T9TQ$v@xGd3pui^`7zd0{fR{{EaOA?HsYAL@R368y}=J+?sd;_UKq z;P1JdpmwJE7|gEjvSS}??Y1pwM!5#Ght7kRTgCN)7@lzd7U%ubfi;NkbL^i^HS%EhcJVU>p!#NjMWb{=VM4b^!P`_ zUkXP5TIHC+#(f2}4S38l_GcQpzekrv-IV9SJq9VutVWqm`s-T+d$eLnx1#!+37918 zOip!IQ{W{RV>c=fpkt?oVBtE1s8UKJi6MWlhLm;!@r{*uw9&-x$Mi!o$HKv-As(>N zl(@{*saAM_Q+@lRao%)i^6A+K?lk6)Bo{_QL(ED-*QpmWI@8BCzu$|0z3pz5ggfp8 z)3iQD&-V}l>|FzoV@U?$p80VmQ$bp;zk8YQ3!v#!`q~0+S=!JFYSEK$bS0)Ie@^gO z8$A&s>y``5qVqYUsW6_#x!j8>BvLjb0IOimtsA;4HQd>ipswP2a>ri9r*-~{<^4OW zk(d5GW0t!YB6yyM02jsP8Ph_v=yc;gwHmghONfh;WOi1lVusevO`6eWV6zB9o@(au z9_?Gz9}G4HJ0|w#b<}D@Xj7wY6sZ0p_-A0A*b+5}xd^9Nw>RAFQGI;H0{jrBT}=onHP3HrDK*k>OcV)dd#0XxgYRvb zUsT#7*sLNRIy8H+(50dr+@70$GfHZZ%)@}ca^6)Eg$ADK5UOqKeeF!Txt?3MoilrH!RM04ktZl`SlKQpQ%mSfEUQBL?qOQZUWEF0mpw zdkOF7Kk2#eHm`mL=|-Vn!Hh#xY8qRk=ZBt=3_VIjncYMxXqOS3EA%Su`8RrKallyP z1QFC~fBBLCH^wCcgKb*HrcY8+#QLi@^jikKe~b;)v))org{@gtt^}y7&e+vYz8m;S z%$M?K`A1dQE9~z#O>u$F8!LD~5xo9Ja9=?LxAWjsH!?~(H!Uq;6SQKafEXPdI18@j zyW$t2Z0O}ADbTjNH$2?9^>Dg{-l1Wr`uCPZRiHm~ex)S5Ro=G!(ZquVQ_Vl?H!K@K zE1xm&8@nt$L}&=FcClqbE#R?LM~y0B~^znjaaVYQeQVqgCZ_EFDU_0T;+s1MoJV4?oF)xhGHp;&b+7C z8#qD$tQ`%wC+!7H-imCX^D8HJLKxaUMq!`9^wYtowZ(_AcRO#4u2q2>$iY|g+8GHW zXn8mh0l#Jr&F(c8V=XLuPC_L*?Aw*^u$Cgv*)sRB?T5q4)9d|~`*aG?0RkfZ%A`Pr z^T#pU+NPRd%}jWCJQa2Io~j3nPYZWXblC5(x#n<}%6UeH3ZrXvQpD)jkrb~0b$sk~ ztT#eL+qBhSWdu_ff6&+fzWCU4)#1kxK<#w>nDFdU9VSWG&aKKr3-mJHI~szB3<&DU zl|NgfIYG$R;K-Ne;P!;##}Yiwl(ADs@Lu@eSZ3rh!E~8SWLa%`mHDB$8!LOEUbH~5 zTR+zl4C)*EoknpCP0mF@)tN0m z^xs{@oD4|0NuwhB0v2nsZ4NDvr-_5niIF>1A@c*-U1eIh1>SKYi5_T4_!Uiwle^e3 z)BAvOG<6hbt(^A8PBuI5G>pRrgL>mqVa}{S*u<`)8w-s#e7cF$$^EXS`w7m!%&Kg+ z59wwJ*q#t#asG(6J_1`drTkos&uJDt_d#Sxju3aQC{y+4J1z@8Abv7bvoa!GrAWr~ zzXSZKFOG2qpMo)Gj5INdBEQNfoL@c~a;KWOmwG4Q5>%xqkXp?7ePJgwK=3o@igx1W z-l_McI`j{a5}D~K*Ue@h&rkTo^|ww})WT(Z401xE(eRCvY{1SbL*ctUa#lI0V_lTs zjF_aJ)ds3(7znz>SCUu()sjU=H$kzvc3RyI54Qll&xire2#=*J*6UiBBbKwrexMr1 ztn_HK5NIXP zf87h^i~Qugo?K1uw)cAMiYsuu{az!MuWuOMeJs3~rZ=bp+S9+HA6ew#EZ&Q~F0X8RD3k16jaqqT9c3I#QAN|p2hb~6 zp!ym=$Au-oWzkP?7=B~Bp~vJnCn|)Z+ZxM2)I0-ePuh#C@w!9bu=Ua+f|#Q1a0Koo`z=!7g`)B0>m?#zQgg- z8lhQ}0^#oQ2J4zvq)x0*>p7*AN&x1jD8MY9Ps8_3{2mb%KL$1MO~F0R3?YC1-~g=k zF}=%Fp*9FSgR~x-V6^VE7Yux%u9MK`HT37+r@}ZH3-!()YpvgX1@nM1QN&-$=CAe#6ok6XGv>iEiqAjoaI#rODtPNH# z96h;r!TIu4p@o+sXV+-2BM~d9K40YgRO<^HPWuL(6)zwdfwqHebt9E}@7F{ekS;C~ zpHx0pe@$CzGv3NwxaC2xIy1%H-lrSOd#=`a{-0m=)Nx68m2h#J`H=A=Dkc)EjM7~J z=lA%QAP``giP41K+3gXPuv*;e=^Lwl?w7rmO^w0@MlqxX+9Z;_tbavp2Xr7cFYjGr zm%Ba7q2JcesQ+78J)!S3gMFB)3$U!*r%)^99Sw_3IiZTWuEoW6nbVLLqwAVhqy1*$ zhV)!j%jA8S2mI;2%nM5Y>Vm={_rWP zS0PrMxHLs19cxA`C{2X7*yXk6!ro)p;>#1PL2<#~sX=rP1E1 z90e#KuTLWlKz98tO8KLEFvfZQAo#4%NdG zgb*PC=jW`qC?UlBGbj-FJ{Q4o60u-++2Mdd%_I+1<0|A&4)oJmt)bvbkDt7?IC)cP#@qcL!x~gMf}4;DH5&7d-l&MQe?-nLuKdGP^e;Nd z0f=gw(w`A6q&e&DyJhmeWeKXz#dGQ+3=3OEfj&$<16hnhMiJF-c1U zL9e)?Te2hX7**H#w+_T7Z0TBv&V~E49HiPJTtepjsS-6?;>S5 zyRGP)tEqV6y`9djA)SJuVSLlgcUL~&J52v3#36ggFsyOsGCR0wou^LnjetX1VqN}o z$mn}$mgi(h+@a+W&Ue-_y8*)y7OQD-T&l@+k6e}AyJKKOg8E)fH- zv&g~HR~K7;KXu@00iY#wk$$!~^EvR}hw6}_Tq?SxQ$+!rmHr?X*ymN-T9d0S8)ILK z)$K?=1CM&VN{Q-mpFrsribRPs#<#CMu!!qvELfOlglH2n>VueUiMpofYNxBmrXoD5 z!3%@ayX)-H8l2D0w!z601Wqf0bXCT+iodSJr8E7#KXjXX^nVFq7~Y~Egrnn2V@G>< zwHh8U#nXx3fAHdDFQB}?*U)?C*)#uo#R?&l(J%qCg-h`+#_Am<*7W5bYv-4tzv>eW zQe+qB7eM}K5lZmxjd&Z)vaUNriaeO%l#C=FmWReMF3uZ$%qpq5ZQN*XUz+m$49^S>64#H+}0d4u%xz@{^#8HIroD+$+Mg!pYQu~ zy|3$ab@A2~qDSMMjV)4nr988L&u6S0)}6026;+!!!P&DR0~am8=qkzlyb$k7x_Wm z9KHM^KWw=xWt{6TA<(pJH7iksYyEN7A+D4NsvBul zyPbg2(9f6LofQQP&d&u4mx?hIod=|of zCgV&EBUJ5le}+&ADahLUS6%0hwVD4I>6fMYLku>e&I^RbuR8|`-(?}SVVA@%9@FYB zZ~=B!?gxMdP*RCf)rj#E#L+_mN=8y)nUPGwtGtg!F)P%O(bAGlGQuqVa+1G)a@W)- zGfp{rYkcIFKeu8{)-6JHB}}<>G2xTVh@%+0#@TZ@9m4G;sf}bS_f`s0Fse(vuAaVW zHq{o2>NNWw^1VL(x2UZMaeI+r{P78a%A)$f6w-s4i#xhwKW=S zb|%&CnN9@<`Fj>R)Ky4xyXeQEq)k;+WJq z0dlpoo#ub_l~;Z7q^udlHY65@N1&s7Rygo(n;?drz>n#gUt6jc3#3Pk6v%rn0tkSa zUV~)OhMKa9jddp-Nu#8yU)Nv^cUBP|j9~Flm|8~dJ%2p=)$`jT4-Y&tS&{6HcS3SW zPy)$7L`R%}zKH10DU1216F55$;(so6--SYD=|&45k4VTRc#F2bo7EAe(W6Oyu%Cuc zw5PD)wBMvb{Ci8T7j_LeO8%77iZ{bJ-3Tcjw^|3JJ{}zTtA4wM+~0i5Z$fNH&JXsa zi9)@&EToYLPd1}9(aN)OJv%)HDg*697Z9uV{T9r-zi~tIxn>GuC(AZuu*aRG64B5Z z0%;$2%gK~{s5>lBkZ<=chpf2Tt#H5r~|g;O*8iV4Y(f#2C=q8Ws%?yB!3DZ_e`7t;GkwzF?n z%3A~U!!;YhwZu2nS;Ze+5cN#_ z{hv`Bm%7w%7Y7C!eMW_A@?YroPLSM=L}}<0b0~F_BA~bZP66(3XXWoIrXOj{GO9QD zlOO6cKHU%1`doWqEfI33a%myiBY0$Xn-&}{T#5hMa5OKp7k0cZg0h{q(kaDQ!Ef74DY#3Q^N-B{1Rp93PVFT z_z<+0y%T;)GyMVK&i%G->@4z{Jnb8iHu%v26eQZ^cU9GhRanx1-#=!zPu6h0ejr1z zqCdLz)Ya*A;=n%*28|9EbL=K5jhJY=0Ozyw9#aAYzr};s5(VA42-++lrk@u^_4F_E zUk-Z2biO>}?VQE|uD`-T<#?nKegpj>ZBc8+-YijC7pf4`OeS+C^NZ`J1+4#Q!BRA# zo9;lteSI+mWtp(DT_?88m_$SJ@W~L_uXP)zU45KiH8d)7>OD2o8zO_jhQ&ATT{+g6 z9Dm)k^WT4_NSDt4p!Z%Z<5Rs)Ol=7#rNBD?! z<-!H+;%x>YYZ<~~$Io-?1P?Q=9tA;?Q7Z^M@h2z24Vhg+E@11)(+C-KzaJ3V*AM{K zEU;z9UxiG0YV|w zVwD6q>BiLivMN2n?~-$K)u%LP*oa6k>*RmP4rKqmo;hAz*!>6w($jbK)IeY27bR*N~6L9o~ZFyc!c1j4Gr zLj233DAUEgfE^GOf;v7j=t)-4>dJlXf2gq|pRk@%Ds6CWRfFNTt$=ujGc| zKfR6EiQ_Jl1$Sp*lGT|7?wELWySqC1Lqy;>oPg^Vdw1d(U{XALdwj zK9Tcd+BI#+J}7EVk~A`8L)?-)5sf}AU@<32vPLs!;Dyq1W7qG9^T4i<-TWK|u8?&z zgDEMqbUI37pIc3yeo7CLomK~KJC)dM9%!unX_330 zEfYPWH>Biq`*Z^ZzZzU2mq6|Zago1HJb0A0W9t*KL4r?J9SlvaFE=IuSl&+Ro5ifh z_*5n2`KoTV|5RYGz%Y#`C8(#0j`v#kZ$1o~rP_PuYJYQ2D%{VK371e40+Pz-Ynkuy zfqm=Z58yHBg3~)nqh}SP$MlN|-UNaz1W~&-BT2M_UCe*ZYVOXGdZYC&j(gK3-r%kC zS3p`9hf{Ygct5(E3`}c_@O`n$2-``<)P}yfE>r+nVJnoV0<8PYVrmSoMXnwSK0P>t z6%!ntZS%P@al;B*9v-3bP(X+6hZdlif-jmjj&v>%1vOT?DOWhSfVv4n_+4E&@I;Pi zyuxctSug0vayRO+9?4xL8$z0QHM#HYc2_%PC0!H6th+TG(M)hnpe3g^!={qvWLPINm9 zZP98_#eKeDq;3t>gD?kl4B&LoL5s_cCXxDCzWa3=(zQVgH3Y*OS zk9-Mo!htWsOIQ~;pJz3?`YA#>yZsgFKTsUhEQULzf!{*b-30#rD?h~US+WlrB}Pl+~HK2rw$=|a5yl@mc^uIpCt$9zx}%H`7U+ap?@tZ$L6JE*SG>#yCe zW?UAo1S)LOlA7Fm4gLr+QjB4R?ev-l-M9&;x}CE-yFw=h`O+l=8|8@x{*0=@M!%gS z@Rg%fa%v*lmhMt|W(<)5BYlQEsValOpFBxt1%!m&qW!U`Yt6>uK4%t#=_)Tc0y z6MXi}y6K2?!|O(wSE?z(zlP~>N_{OQR;G6EU>Kuw`i()ksI zC!NASnx@OLLJO*2xBnyay$0ZY@f6q>BuTAdYk-Sq5Lb!mo2r(cJV}iHL#ss`g3<)* zOesCYt0M;GUWIGc>KTg{=viRXoVGW6!=`bM;?(C`;vY;3qS~we4q)$g+$WQOrV-9WcxpAf2n5?0rg@OSril8_{ zQ@92e@XeTJ5hJDf6_!r95v0O}38%?I=F1es`yfL{;dJ5j&A#=*(;xI6MoIk*L7A>f zPaN_^W*_~$f-Ib|sjGE#(Q;?($$=6^KsY9=4Ec}iIfS3r6-pGlQ@p_%7@8ta$B))u zV+-Bhtry-Z$4Q=)?JtlJ5wrz8fh&YZ-Hnk6shY5j{_^rMTOW>O8?$Nn^&2NO4Vz1* z@T-Fd&*2Ycc9OCyk5zdGH(^dEJkgfoB*u%6-CSeUU~BlAs><7c@2>%<1ED7^jdV`8 zNrs?CdsnvfWSe_h`l2h)rd&n1w-=!L&*_TPk`DKCxTkq&h7jcW!MhK(%O+`ECgx%0 zG~KIJAx88K>a@*(*@F{V4x+(Srn5yEc_9r7UW1D~o^8mkI6^4lv| zcYP%B3=&xeq>x-=d;&5u6-MoT^Bs-V&^Ls;_%jxgbAzV#8@eNOzktc)yV9f@1%>SL! zu=8`OjOw@6Dv!de@7|!sSLFEtHSrF5tm%Vx0=CY&a&)FWOf95lGQB_i!Vk?oOXu3K{p8>#O&$Rt3S@>dw9ci78?iSw9_T4 zarPnesWzFumz{%eZ4Ud;%+dTn6IZJP*W^NQK=Z(5?NK~3ziR#8PWW7v2-k%QmJgeb z)9L56#3GuID6Smxg%vKRv}H}ZV?wjv85aOVq)Tbx*`8wJ{uWT%K;;~w#FX^>#p!0v z$TJ5!*4IacHXwHakg^Oa5S_h0C+->#IIaBGcu420D>Hj9S;nciw#@KUGtNYKU(Zst zpv*Ml${Kmq05vo^P}sb`Tv?W)=rpA+^5(w@;;ro;ab!>b?~A-dt5Uffp=U60Lye>s zlU}z|L$GGEKUt%h(&{Vo;_CSAR^O60lbTO!Y6EBul8$sTKEF>KF4%yyg#KhX@GROh z)i>K^xFxDm9SA-_xHG2L-R!EtQ>`UJkBq5H=@U8;O#Sh_Dh=HREfjGX(r(eGg|6$` z?o7*k^Kuz7>!{A=P0iAu+7_#fI^80G*?o_bY;g*$%u-MgIRT=ED0x)jgX?-i1f$u`^r#y-;MJL>CyMt!%9A$RjTKY8q`w!+J%Y)92th(MGG^( zKc@%0;G9FW)H%O!2w!xTZGCf19U@wYSC(uur9!xi-jX>oMHPgrN0yCNT2>J%PzIk#r~9>|?h-$j0%pb(_L;aqC-^J4*+BVaD!0{xOe6HJ8W$ zIau06G^X(CZl=<7o!_=PQw`|Q-qPr)_`HUNDwP2}ncxYDf`rBjzZzLq&78m^s; z_*B?p3lf(|&a*K+Y55n^$)?&-$$h{6@?!a}%Zv&#RFNb_b*|Q4ZBkN{V^_dUN-wt@+Hbs zhGv79;8r%zR0e++N5M-@oJ< zD*;PVg6JX)wFm7*0#--zl!v z$?b6Hj6P1s#b`$I!)J~FBwv*wm%xDc&arL1U|@v#QiDWtTg4~bpx;w~G1}W-(cs;z z%h`d<0%vt6cv^C+%3CtsMPEHIX=WRGAJf=A`%QSfe8sbsU+)~KB}{6pSsZAQj#jHn z1Cb~QFftp*)9NA_b!^wrA%o*dHHZVhM2#hH;QF+t4jz9z&j6`cIjtS~)2n{8cIWs533EfW z3Aj)Hm1x)>5UM-r)+Oh3S+)3)8GF~@Rzdss8%cSykcePF5z5JN{VF@-uzJ8tsQk)E z9Wb|ZV5y9yMVQw7T-5&lvp?`druK05P3`*C`=K;|-;o^SLj{KC++QkDbQx;(XOFO1 z(ZQm)agJVp&#@?d`=8`nYPa~4cmBeWeSZbg2b6f*=?9w!{jwvH{{}@#u7%!*GU2Pt z{wxllaxcU}t}ia~*e@fLyC!P|AT&Et_oZbPm6m`-(3=e~b|2Wij_zNJq&Q&Xzv_0r zZg+;20I+(F1;syY2ENzqcm5@P4l-O67Dw-HzMGQznDa%`PMplctW@dK998A8QrMI4 z!(uso7TB1H@DgyhB5ipM-`yWa+-E%C8%VU%YM{ zjoWFPG)^{YKLzke1&d(?VkAL{=Qf5Z6c3F=dEP42CkR9Hca#LvO(ydJ*&4&2&F|Y< zcky9Se3$jQq2nBV*X5!4nZktp?`u zLuVSA^oY0QX1RpM!yYGp`8Ui!7WJs*_UffC^${dLUz$YHr@fCaA~GzCGq^I1eLy*Y zWDQ6M%TBV^T4Q&zAGa(%ObUk8A9j^!6V6=TXugx97!lrcy*kf^Tr;U=+&U7jepmlz z{V=}-J5ACZFm<)T&+HvPau@kmEM0%2t_t5{kWv``?!gW(L(lc{Fd8q<2iTakD+?JF zqFaFv(p>*6SGMB%8ef+G2`u=BM#9!0hFJ;jQq7AYYL-|h(nlE`ciK8#JMbXZx z34Nl^!NPtYi@8 z+)8n$dx2=q|frh5rzN9hQtkoh$Z^NU9 z?i?~rD)7qCebokp9V^OAZg$IFsIGP=gfbA>1BQ=8?Y|z{+uqh4PpKr;WRo30rdoQD zBy)Q|>8EO0AU)WcKjQfQ?zR~$JC24-#5)5y9Y^*$*ZaoV%7^av_{%JAWFh__LN|)0|&Z`<>|LBT;^A4eFmGqY{&a0q;*QG=jya|8f3~ce>c>&WoV}v16$#f3F;+mw`qa+_Er)JE>Lh&2{lYmoFjl!MDmaFl|3t z0{&R)Z9@yif;S&#M|R1t=z$vjrxqI zn#j?-Ah6y#Pt*#H68C}*nxx6cmTUN))D%KGJj$N_s=RCRa~Ou~zPBg7N0gZ*SXAI$ z8y7?FEcLsWJsFKzXeFt3njBdCC0LkE9h?xV+TJ96yFj_%HuN^ZsS95NK;fhHl14%K z+r8Gv7}vDdtb8pGy_BHjl9v{%00;$iJkYfm!l)#$vS;v=8W)t#y;R?G!Dq4Ydd;5` zyiQ8*_+NPttlHoczz-y`a955gDhKJ($1?hwzTQaWD4dX_sQ{I%#}uss4+{LT9Q{OI zY*(zmU6A|nuEDq<3%tR|qR=`t>)iNk7-n6|?Nz#LnriA=f}~`P5rj8ZjLw>#!lw4YZoDvm zJx@HLd0|Ogv*)>JLCpXfOP8JhYjP20)d#!M_>YW>b8hZQnv%-vX6c((%s^h!cvN%1C-!r}O+vz6!B9}z~6s_i1YhTY{{r_-Exux4vP z4>xP>$n-&!WY9KpH0(bzAzt7z*EL8oohGZ$k*=gPJDghXmutA>VJJN0buGo~*#g1& zz0ZbEExrX=2={1rG(A}C`yubZ_+BvS6O<`Ka@+pzvh%mr8QWI*rS~)u6Yr>$h#*#& ze#YZ_#Yt?)ugkepf88N30mpl|+3QNzU_}Q~NQJ;BvX^yXYj?Jj{2lh@8U}Qt7G2|s zdNSDX6p<#&J!tTR1t*M^R&;-ymivDL!w(- z1hW;Jjh8Gs_6OH^$qFyG=I5R}a;tK1aM`8Q^Ug zrJcSZxxiryT~USW6L%Fl*Dgr@gObYgeZW|H z)_|X>Jg5CWA4Yt1Ae0H_-Xsq@c=mbq#f{@i0m{38bG~)Aur!DnC38-yEB_u-Yebnl-FkpxWT~X$S=%qOY$h67ctSN2|mkWRk-j|$Qj8Wd~2Je<2zxJ z&LXjw%eB)p{=w;6G63F8E*M_i9A{iaHFU=(tKGJ+K}~}xy}lP+!W$>1$|nlQiP+cX zW6_J-Q9>y2z;}yUONHbS!|#+wmP~(YkQ$<8YH8nLF8;tu&Axy?39j1Y#hq2WT;BL4 zH7oeY#CAPmD*OlXGeH{Tu3L{~?m&+kBT(mU|7-yB?}+r_p)B|~9!iV8=JPitf21jn zFTqpbx^m*QEywFsbe9PB$=6DhJRYWO?JG||`Ra8KN5`BU#ZrZa!Dc9m50~0-o!2EU z(0@GGg7?gw_WU=0{Hm#^6VMcE67xU1VOeo6o+&EW#-Bhb`ah|na{ z)IYXyw9?N1kyYuy6KG8do7X={XUczMQ7b%&S^YJF`wR0UgGvxw04EqmsaGJ%yXYNP zQ+23d@R?6a;AWdG5v1xL%fSc#O~_sH_E&8h_*9g3lH*VA}s*oq0B12iFdPkWc8 zgJmqn=N5N?7NG~fxC$^^Ny?G>`hB48LfJkAW;IJP4s$!s{pFr$Y#Y@GOtbNcIC^NB zVCd$cb|y-#ovFhE#RGda%%)sVx2=t9Z)=rwq?_lLUGeQgsC=sm;-PdY*Uk7_a-t7c z!F=A>p}8LBBJR>No?jCT(+p8EM-6{D9qev3*g`?7g&%BX=K^2 zK!f6-58LqR-cC4at%5dNi#A_2SgaCFz`AiV+P9_ui04r=yfN>Q_Yk;D@>}^2}uRE{71HLUv3&O0%Rk|MA*%{E3w3o8t@|Gj{1ze&jngklj+pd zr=m7t2+|Bj!V1ELB&ii1u5hDOusyI65*9hxBFFa{oIE#g1w!^bqm)N0>M!Mez--WhN#0QOOIc^glW{}98Q$mUFi7b34lJj zx_d>kQm%h!El|$}*!>_HPMfP7^EADL{-}3*Xa49)6IXiIaG|NaKe)jJbo{Ff?`^4v z^~Sm4?L$_nj+%E;mTRUsZx(qd4Rui18XN9g%iQfWyK)9$%fhUtnqAtOFN~AELa-Ii zl_H{T_;}1YT67?JzGEQOsY)S>n=#2E%~Ut^PsP6nL1aK0z$HCKW5_}Pog6MCIWALG zv*fgvUdC4vD%*`Q6y@H@e}MNEiR6(6^8hM+d)NOXtFa`FiL=RbT9Emf-~8PE zxAiza@+7IiSNN|L&IT>f3Ic8t6ZrVCg7Zar+dGSS;O&Ba(x`mL1n+sJ{?U}Xbg$Sc zhU}O#EE9JSxBbPA;m!zhF2bIbqwv7wxbc^?v}7M1gb;HUN8Bkp()P9bH5h2I^8Db7 z&5p_73FErW*RT3-jg1s$hXu4|^PbBl{rvakE{VSpV;HmAR6x6E@#ysfp1v*_{Ld>n zNA72dhps6akI28nIuE~UB_uiLl6cmwF{|Dx6B=t>Wosl|o#KdJGX?1Ydfv@&{nu1zfhZAzFj~{d*ewOdCo>*+MrhA0cv_ETd}Xsi4{ND;zcQq8 zq;zUu$OWJ<@RZcw1d{f!C?w7XE4qKaz85cRWj|HR+SG`@@CahRxXmlZCNmiN0m!99 z5umybi8xevfi^bmub9Vga5Muh{7E5lyA=*1Qb{Ot&LFIGYB@H1Lt`|l=Th(ci;ZVi zccMm8A7;6|9DFl-^(Aid=Lh2inl^+MxZ#S7KY_Yu>-!)iOjXRq2mO`PwuP(8lyOM8B3)4MDwfkp%`)0SrZ~a7x#sr1 zIslMcVvUk;S{B@e$#-YiD|5)()K&OPh}>XB7f-S~iMKL+^LwTBa$O?P%hRNVK59>e zBakte@EXo7$S0c-t~Of^dNPE{GfGU%Lj$GUUQ9JU*Ozpi9^UG-bezDflQX5mt8+n4 zCLfNp2A|y-3zZ^1S2O4-uN5f{zQY6=&z<_3*D~?n}q1z6XrJ>DKPK=u146?coHYcIb)FU@yBm*g;l1@Sjc=dyStpaT-e$+ABY>2W)FMRfc)ROGhNe&GuHuHS)*K15 z5Azy-ZxvpG>m1qLax3PkLe!WdXwLeDcpaBcnOxI=03SBDV8yEPE*m!W-9YP36p-_F z^ghm8L~}m~{FCtJ`kR4qQ|5^#kLZ=6TCkz#s-l0d=~oNYpjQF6lQPzv1 z@N}?A_U-=V{aVH@5RLfeEqW)~-O6&T#AaN(u)Y@GZt+4+J?h8$+tv%O?*4s1YQ7op zrut=6@t$XumQoj_?zD8-(s70MpkRxIy}BrqxEVxS;zeY~Y#hbHben41p8NKPvXCxM zb@)zd`nqKXm6>3x3NDDT%|CmEa~#vbu2uN5t8mXd<*+2d(o;3#_Z)NqWS+W4NG3RM zs&MApZ%r_JcaFpS)%`fJOq;0+Qe?A02ghWOZx!;w*;)Pd&8L*Y zn*yf7KkJ>A#VT>n2W%N*h=pszd>XS^!(TbJ>{8|GaHq8(G_2Rcdky|}2ld%e-r|>$ zl-@{c5|DipE*)A5fT%P3nZ8kW?-10X8xli8?JknOToGtwk8k)%LBxF8wW$PmmOOXMIZP%(eEzKP!82#BFO2d{KD1v{^T z;&O8KrXc}8E=rf##(vrn?XJ32w}ZgB_&Jkk2-f%SkM!Kn8TizwSwT5l8hrKZNnCff zHcKyQ1sc_35Gi8jG+XxDTI@^!{8f?p*JX$4bs?Ye%l24NZo8ZAf-13s3GX$F*kD|E zcXu=`Jwd&t8PmBkUt3%1&G9SXt(OOAI&SqemrC2Y+-^n>UfXI;j^pEF!a@2Hz4iv7 zBCXw+!&H}t4`NE6N)h%5?@4N%Vo#Z%MPDkCOL;w-#^XS@`ZOTqTWpqJWEf&{*u%T2rn z;5d>5TowJ)7(H_&F>XEOU7;rQVKAKVxmML)!FvdBgrs%iX2KkOMy*5sj4|w-2u-ln zmHJy!+O(9t0gvV1!y?1r8Wj<#{-6uz>d(regouxWb@34;PDYt3NL6W3 zulJDykKS?{hkeorR~|oUaad0kpO1{;zjb{`_RofMx(uIN*~%`ylGaGcCb||6`C4H$ejwd!MdAa0Rg{O+Cj9GuqR7fhvmq(G$eu%V*h*8OulJJ%`DR0+ZDLe-bb|(Zz_Y2d|QtpuL z=8QenR&!I@4sKkL%k2>$P`ei{(5kq~)|v;Qn;-w+orv+a49|!25Ih z^X1??k4tCDo#kC*lF}z|X#mGtyysd>Eo3OLTFSrM_9O2{Yi@HZ4cf4IaQXqGVG+mA z`Oh3P9U_PUZWeq98WQtYF!}cA{oC!=KY+{6ELBnoe3F;LSHST~W4zpn7WVVAyBY2; z46vn~wU9Of|W;O79qiY!eQE1zK17kQfry zN8u4#^ngm(@Jt@RetcLX{QZPCT&!1E`bKd5F!HTf9FhSERfCP%xX72eIh!3S@*sdb z3b(#5g!bvJQ&|3eAT6Z7HbZvt4m*S+R6j`duQ3hU*D!1 zuwodwC&0&wc>fU=h+jYhBlUMkwqYy~lw`qnqL?t9EVPurB8YAD088%i+*A^KU9o52 zWJ(*Of8b_Kne)ns_&Q>kXIU|uEEVC9CUvmwR?~9Rl;3WZaBCi;OccK558eq7qj2;f`XS<_ThhI22JfD ztbt%8DS=)@0{v#8%c7tZ!OR^U#L1b*aY13WJ@M8o#d%DEA#&E;m7oP$+@lF1T>uZL z4Lb<;4;+5=5#k`&VDs%eDH3y5HPO4up(D-RG+kUZ z`^|gQGrx>P*3ZAm|GOUo*c0V^jk_b~n@VQ06dgl{VUPV2(DaAw3nnO)EPnT&OJ))jc{YDvDz% zn6ky`)1e%hJ;N=#UXT6x14|0h=M0*J(%^N?+8#ri+R+k7pg9=>t3*3cgS3tK{2% z{YS=|p5JsTRe;&k#Xu{Zxl2R+u(kcfNm#BDxA`!6oNEmWrPakk(E1L<7O#Y`z$c>x zzC22!QhOraAHFLCAvOh^ak%XteAd^PhR}4e?o64g(M)n`Y}s#X!Ybhav1w*y8wovC zw}iWd-~k#jh;T3$W-bZ-Vyz0vLNMz(9_?WK6f&cH{&Lmtw3ZxTW!Kki7OyP#AokrXS>(kKj(+e947c!bfUbb&dZm~XTgK1wjA%Ck^sKwDkb|- zh9BP)MUah!*bf;&!6<0Hw%m<0a@Ja;PS|g0p96?w@ZCSYimE!&?E88Ec{{ChrnAA6 zD2e4A6RmIIJ4~ir5&1He;t0OV+ZIH3O2OHxDq~e=fvUC%>v43y;@?vL$D<9S$D7GPh~c%NY&KUQEE6 zfkG>-@aC^sqew$iz}@LDDkOa9UCjMEz>zK{ zjC&-zu+zyO;&#QYarFv{L?H%1H$^{*NJ`g*ze(F0 z!tM-0orz>oOh4pOqSSS@n&XuUKxNg!av{!EPTRA~3d%-C% zkMf6Q&-)t?i(uKw1BsQ}h-%07In;Uz8IP=PAb697z{K%78HE37!*4V#ocBP;-eKe> z*=itoqM)ACm6#2^VGc3ZwQFhNclmOSBsMvM@8@$%>LrNrbo1r*f39ER-O8p$y_6+G>c;5G<;@=(;$VV8O{ey-47Vn zErNQW(kFb~)3}CG&*S{h;9c_-nuCKTcf6{CCq43g(KWmMRZrlDZaLs;K9EX;pm?S0 z9QwfTLhZA>v(0q?59#YasK-Fi z0=*FW@pVx$9^k8dHNyli!(^*7SUi!Jq`YcGj+YJWMII9;g$&M+fH-0-@x?arb|*$> zrYu1u)YswaA9#yH_7c1a8>}M5N<$gC$UwdroO|P<f{@L0iAb^8g5m5p|AZ<7f#tEna5HedAeg>e+9~(B2ZCK&^nB*@JdYRePWcl9PKY3SYj~G6|Yk_L$DQ1Iy2Vl zMRecw3p65YD$P0B%OuG#eSBMBamD@t2weN%f|C9I7Bp3=}GV%tQA(YH-1Ox$1;5JR(A1a z(#)K0rN7KW6J^c&2U49_kN|d!74L9(1t6kOd26IMPwGl{lt!A75TDk6NL) z`c1F8OREYgXKvDF&))3v7il{NPImTLK7TFOV`qZi% z&l+-MCHNKcv(%^ix;)V-ZLKj=)leyyrrf`eJU5F=7(gts`?1?2AP>5zb}V@#|FRnL z_a4OI<%dW-)z_<|nZ|3vcR%Wjmv(o`Ko!wiTA?8nj;Jp>uAvVeo)7AvjMz2M z8=EO^(o{ar$ABBT9zaS=avYEfE{Ww+Dd>laHJI3>YOT$cO?AH~@)BXKAnN}|=7^l98eE}*ZDLvaw+r;9 z{F%kdM~8ql(2!-+Yo~F>UwYHw)z6h-2k8TUZ#?@j{fhZ=IQp+%9<3*;TZFVmI+~a7 zA;kk8W`Qw3L~<)qbPXbH4h3y%6ZrrRq0(uDb-si@eygj=TIaO!p*NIx3G6ff>ace4 z)nwm(-Z*|v%s@|+|Gl^EWbP7X+~Ll)+f*#)GpIp-+fs1k3GvQ*SFY2?w-f~@SMPfc z8V)C1zH@yhdAGE)>ZyxZnN~GogDD|tvEt2Eq2kjGu3_6?TlU|Xy7wgcM$#qdB;!Rr{5^H9v<+3(0tThWab?zSq5n<&KMggnGC+dt>FpE!Pvlyr z|3mn`758|0@m&!miwvTcSX&BAwixbKwAv$O!~N0xih#v6-^Q9!wM~r#R|Idj1l$B2;%*7IpZWJ;iv#2WKHEcwM&;At3ya{iH$#ft z#KOj*?8F&h{fCmm6>}#dcgxdvTkkp9jUD=OQDb712Bth;XrIyM%?7t9Q*Z$^2HHvG z7g4pgEq~Br;`B5jRlkZm{=f~&Lv42;g0yGJ^P-*pvQ?avLiOcNmq{%_*AZ)zZ3R)# zPqY{&QqQp?gtnUITPZq3Mfqwt1lVIhP=)PqlapfJ^$H4o+7;wSkGVpT-)mp=5!u>i zYJp8>!5IP`)C9=2)1ZiJAaf%>W+kdNSk?QwsfPJ@7zrd<;Adg)@9?Qb(KS7If5UKQ zou^&7NrOzkrOA~wW%RpePRDJ%VjX|4US8!iP!l-O+SnAm7OozV<~*q2x^B{>PW(MB zo;i%*8GPO0A`$;r2D_9~#U-s}ojH0rmt9$H<9PH+jM?JP4x9FSzv^0jRT8JEI(GpM z>*%d{Z77uPlYP+v2J*-9FRaEfuzeLCsLeHLt@cwf^Y!t^AUFkw|2ELi)Y=)Z->zT1 z0j6hXYjhnq%uCBSN+BX4blvp^11u>uO}RI+Y|d}&q{xAzmlmb-XC3?-#4P2}rA$_$ zkx4!qOh*7#Hwqahr{5?B@sA|_AD_%=6@wXv$`4D59fO+44S0p{z?<@Um-TRf_WUqk z&16%R1yKK_#3=vobCsbjR6FFRW6Ew$M69uT^yb#Bu;1jzwMFXqk88_Lt7^`^EmA}) zo;L`=sTX|c1SMgohZmB^^$#+vSs+7gcPFO_wN3A|eQEYw|EU z)P>)Hp!niNz(VHgjg`2~3|&VhJ3pUF^OSS0v=>B9d?Xkh(l++8C$u3n)ygwS$`ddh?P2-h<$vQx7bE!B$5sdyuO*iPUi1M`{=&@FKSt=KE~Yp1K>vrN z^KfVLecQNJ?NuuFXlbh`s`gA(?V_bNNlVo#YR9fBYNl$nX4R@0J9cU-wf7c#g~W&; z@_nD*@%{rjawO0F+}CxT=k+<=dzx|{+WCGcxR1G&Ip!^9;3(8I{zAL*frHYgs{{F+ zq!WZakdpS@JJ?xpjiEjN%k%m*LObU=uv{Y=HYP)XJ@`E`Ptg?d#eV70O^VNu$6_k-q=>~t**OuA)t$f&bAOgHOSn6J4%;5vdS1GqBuyqO1>}`Ug(aE8?t+KDLPmslt0H&sGJj zF_DY94-wy)uwT`BmIBDf8i(hSR%Zwit9ncv=GLusgRKq7Z&Qm{<}X99PA_84wVqqK zb6E%H^>BT!dNuq*^v#bEyy$WffT?Tm@IGHJ-}axZswq-6SGw2_nI=SaK_XLf@ZPv6Q7qd!deZB7DxKBV+-iPUBVHE;AjM z_wz01{e5|E>;t_@mjSko&abVF-OP}NeS1aUg{6~SNm`h9VXFw%9ZSA6K+V8Qm7g$2 z9IwVG*dE2crnm;cD(`D+j1`B-rKgM)`=rqK_-vmNvNLf*Lf7!_qn>JMFs<;Ut7GLQz_fNiWa zoU8%w`ltenm!BL-f6_zuf%qD_ zP;S@}s2OLnOPT7xDOpS+3(W#3z~vFCX>s(`YkVpFR*<U_m) zm8lE-CC%)Ns%J?E5VjhcJnPw1o*j8p@gMg~{YMteRfUqDZiZ{ujyYhZP}Wg z=6E(;O}!TPHfiJ&`ub0&Q=#Gqd&^)^bv ziUfGiewI#XbgOH(46qb>CUE9Z7E8?sN;U0xFcfIE z0T?7Ntq6V{q0HW`+-kL?rRqzwS%l^FX4lK0*%JhcBuH3RA3fo$#Wy7=Km>}J-G4Of z^p2r`;0-RA%PI+RskvwA@R1&CwdgD%s%d`y%wZV;@`o_W6fV?j$C>yW#cY5Pz>~fV zB8uTSI=xnA4QKaNEnbPUD|_TPU#PVS&Ow($?VY}eGe53TtiSFt4m$qo^T($7i#qy`XQNPQJG!^r2x*i1E z;OhC=hz$s1tLs^Cp(}54%pb3MqmYu`1qh8iJtCsPt?c)?ZEeufgU!zEj{* zxP0NF3MQD8!q~gK#pytMtuYN}a`vi-0k?;|5k z_lN%bX;p_!V)gF@hR-y-Z~m#0J=C?4{acP1yYuH|%q*M2wrGKdU{-b7OjqnzDHEZ} zFQfyX`JHa>k)30qf92hVA}ksAUcw8^gxQrhZ5Wq4HmxKr%#h`tX>86FgU!35V->YO zP0|(UfyIB%HN&~tH|aw!_g&aN9On0^di<4%4g8gl?^Rah!Ry0bd`VSQp1NfIbA=|ymgbsD=NE{@ zr^r2++O)kGH$*?_ zmZW^&n_kdaS_dkUj~T04WSg61L{q z%tO}fpI4%@d8Wy&TS#@9vFvDm2MwG=w=B4N+&Sbf|IVmdk~iwnivXU`hybPz=&f%5 zl?;7%NF95U^n*gjaf-0RJsZG)7R0|IMz`XYluJF+I@WvBc*tW{MRX0&`&|*V1T8## z6yRux-7>JqZPrKJg>`^0mU|EZtK=@THvdoP7zmt0sOH07|lkL~L5@h@&}%29 zU;bX}S!wG*sdG`Ji+zhPhd5k5gmHn%iF~6cmkss*D(`Qc$dghI=Cacf_fHOO4*OPp zm3uZhy~pe^)(4FqtW8#Ow9o=y%Te+MPK+nlHRp%nFEs*mtNk1QUSk z)uBxiU%joPh;fs%n`{oCJ$0WwTWKONbxK+x^cJC{h1LOstreu$3 zWR6Ayvf;5+g4P>7wOx1)z!{)>hQmazhr3Q#=M~WxO)w8y5qxmtfM<0d>7B~=D8V(b zbH1wK_^yaN(Pi64>E*Yc6)+eIVQ0HnMU>4P z!qhz0oI!UbYE`4RDE=y&lhWktI`gYKI*nJq*5|9*bzvnDwweB3*J5THtL!P(%5(%j zEX4S7y?fvEYHGOcCNGn_^AQRZBtJ?tk$P^&41RtM0$^xaG2iL{|KaUsIwWmx>lfbBu%)f&3p%9W$~&%SLa^m70$sj$Tg+uPN4lW{H})M+O#HLQ0ieExs_ zV)nolj9}Lb?F;6CxM;Dvmuow_B9_%hJT~yv-9ya@?~LAYA9Kqk*hO9(bIk8Ne!mz} z3P~xXd+7!t*HtdV?IGfi(mj(oHAIhLY)*9%yp3@P=gjsF-^KE5yOpZd{=uo1hQj@z zlGi#(BNrw#t`KoatK*rv9}*lpb5&cdwvW~d-Xdk4`?^)#4t2lYIW(4{vQA)l8qN?? zqh$WJ676Y7S^2gyOz3!x|53#!ix0AJ-Jx$wt8T_|A+JVRVrJd1N*B+pNlWXU+Ub(&BNSN2AlYVDgYiB6q-KH>d8ibD<;+nS0lq zNFPe8iR<}4ug8dVX;Ys^keQX@O@~FXB9j%_ZdSA{X25|&5B3xY z*sGqBd1W(T?;SLIf9;B*D((($#*X9tTAQ;KxTJcd5^bV7C87Y{L@qrA)nHiql@t~l z`ThhW_Rf30mp|o=k((0ltDRD3e_+|*ldq*oz6UGCkV1E`1be3yr>m;=lOQ`@#aD_x zr}=KbRbSLi4zFHP1F5oYZZl=PL*Mh^Im$XewF>%aE5@R3nAu>rZuWZNWH-?=rgr3w+6rv(turOJcCY7py+eu93D^6?AK|QL;*$?yr zwKD{7(3_227=~-)M1Ai6Fs}eQ#^<3`jJpCxYhn+!oC+wj3*_gY!44yQT6KplCEj=Y z@l5B!=AVx%+ZS9Vna2lBS+n(0w(hB4Wq<3x-hx$*^|=vVr#?z=O2896B4g)n>Vp@d))RD<)BAe!NX2% z*a^r@-_r_)3IrmuKB7=^Nz&i=nA9-3nfxz@UT&MUv}QXjY@u}RTbDY+2Oa*cC&nZw zC$)Ebn{RC@avqq%fL-fpqJ;Ka+*IbV7ps@BM;y>(NeA~es`P?(vU#d;OrJ2WL??Il zf6xt6&b2x_lfZaC={%g!D1RyVrnCe>O<5LL9h4!#RdV&dPzfd;Z3MTv;M#V6UA~|8 zT^6aSdhUAz=uKMgTUacy9QEq*;v{rb;eEMIv-BIU2FVXR%QM}wq&)qtl7{bwo{3!rV?VeX4fn-kjJ8h^8#=Lu`-z4jADO}*DTGpOp2ZtlX-3*$U zkK;ri|B9npoR7h@@XNk0J6H)HxcQAi8H}iVE!Q>7zT{l?g|bG8p|`VsK-y$4}b7<5UGJdN)(I zDLaeldyqqsC4sT`^68@Krb2F@zg!%v$C*BiD!#=ccr{wEZcI(PJeShiyN@Sl2hxjk z4mRu!c@#{i5h}}c&*bLmKL;w+(^amaG>MO-$^2bvBY)#J@l7jd2zMG#e`cFtR}TcE zrJb#R_uj`3vtkngW;!dt9k#dhcv`HM_qxR6%?-R}9=XtXCT&jph1fyCZ`AD1W8(L0 z(;F){UVaM|PdPD71D0yGeYi8Y1>MD>W|uK9+iocR37)xN&e1(BQng{apCUSgDh=rm zWfdd0f+61Dntcy{xAKqNo5urQ5YXE>{7DfWaqwd^t03~;c}vxC(4s%)&uZIGLNJdX z+rMljRIrv=3^xqppcCr7s+{J)t0PF;+8|T&d%=m1FMH#Z4k5rjT5}iuDk0PRl(Ux; zalI~JuJA-PGM-1-N6+}jIFY<)&o5Q(TT?-Np5XV&N8O6#dl0J2moTN5ILbV}`Z&Qj zma0Du)a*aY=MC9Q39I*!jj)He){ly5c!m$n0&8U`rZFBI|A85$dCZ!atF3R;lXjaDDz($MtRl{@$k2yimuM5Gw zAhR(+?jIWw^GJb=am8#)sB1WZPw47bi@ro8=B|~is*7LwZRD{aMaa=w9SqSe5wRqQ4H}Jy z5cPb(;$DCL>`p%YkL(#CqDaGOZ7*ZSn@+FqUzK6eC)-qWk~Ihdf`>XE%7B>&9hW+l zxEsgyh^rd9fq*n-%+d7^oCcv^yg>*t?csD|{eTd5$*n^4UoKx_+uBC?Ekj(+ms3zB zcHyzLq11abwf0`#0J9w%#J(3AOhM!WY{8wYwC7Fv*y14mQw1K}9z=fyC=PiDUg+dx zrx)4-N$$e$SrraMcMyyr6MQU$b$lD$q%HEJc7Hqziv^Cvm%`Da@eUYA#_TKKqZOWW z!<>MH4?}ohD9fpL86Bx!%Cu_(%ad2CbvrerAut#5Whe)Z4=$FR)2!Yk@Te$a73EL7 zg+1#Gkgc&R7HjHubr)cnmx{RHx`GgdE|qtHta!jYz8LV=7*LO2n?kdqZn%NRQrRU- zEuBRa#S3RLD(^dIh+(EGiTAyXd^`!Zcr4!A$Y!QzC4N{QxbS>7!b05C zRIr;W%ZSj{>;EHid;*H$pUl?hL&56w?-eKdwtft2B*Hh$z}4-NVJl_eQ-=bsT$r#JW!r`ecbCwtXsxcC2Udb3k6# zt)jv7Rm44h@7*HCqIBoE*0Y!vgE>fy?jOUU6(5xgKg91go*=P?@q@fc_xZlsx*M3@ zW%=FE0)7frmhK|Wrqo}#Kn4LXunG>F?o#9rYZ3A(-KA?4vwLU@DFSUwmEr+-FjYIa z4v7)=zy|bhZW+##qMf!n5JyoSMAejYe!x~oq77Ru#8y4R>GF?DTMCJCq#{^sTR)S#rozJ>bzKym>h#lvqt9}kg^Rh z``89PrL6213MXFk-pDEX8gX+-HRXc*l7rMUC)Cc*Sk!gMQRe4qJ?}rdu;2a?sU*1v zRSI&vN+YV55Zy9X&!RR0bvC*#Jb$qW&M94Z=&x>jd1|3GG$SAePAY~&7!Q;SG&X>- zW<|_0a4$oj(sOJUMgETpY48z9)quu9*`yDdw?mnfwSZ;l87rF}E{R9P1~3|06Hpiq zplVJyya}5*7E?trI^wS5S3C_iWfe*u(A%&>JU5{gK*SENmYZo_%2cQNCJ3lHzze}k z%DO?6urg;S?)36Wcr@uY z@yVhmK9|5bBWaHO8XHs|d&%8GNpW{*rW}INw9-=(SP**GFP8T$ExD15X zzRIB{-pNNq#&*1p{rZDb3?k0WWZ9Cr@zV1|g{$t4u84?YPBuS#2+y9SYT{KZ8vARV zXA8Sl5H$Il;J!*f(usiOm(RvOzEuc>FQwr$<5z~+XIm@2@A0wAE7KCguF8~e;l8b! z*FvMrI;8IPN08Z@up@NwzsFPL_Sa7&<4t6KDI>Z=*<@VZS7pOjRCN3GtnNak`>3w< zatwwT!o=?s0khF3rwFFqbX%E``+<7hm;FVcOEI**o(VP~mnx!7@SH1aYc@K;#9Hl^&z;*T=a4m+$4f2Ez6 zZJXQoXD?=pqzt7F;aqk}mwKN{;?HDMV}S2KS&44AwpKKXZa0yTy5x9gya!;sP4^Z# zA4z;%u01Hf)FTQ_pUk_^JhZFnlAUG=o~BFCIx2JER1Ct8va|E~-0C5*1>J0&REA#t zSW`)}yqiQikbOeEJS2L0s|!q1yBx*l+jbk`6S}Fd?6z|Bp;}wMb6k{!q-6ZXaQ-XEkxhH^jrJJob9) zLf8P2d`^80-ZOyWoumZNgT@q~#7o|33H8I&4ewqDX)< zd}3(6DZw|XyJuqMo}nU|lZ`XAmT+D_&3Y@y%Ih+w^|c39WKj=W+UCvGG#oJ-^h>|D zoKR<)y(ZAl$Kk8eK~k|gsyH+W91I|di5gxDuV*UaH5?7idQV7hN6+8DcEkIJ8r`k= zP+4A+DOX+IMlq0MWg{`hX|t)CNB@y&*2`;<*i;)%P7t~Ylb_r zUSYhLn@przX4q7lFQNES8md(c)KL~I3YI-MRmY9bD?9;fbv~V(Z09Z#KL#F3E#)kP zWI0P>qW*ox?O0Hb55YaS^+UX1?D%|K&PTV~0A1IS=Gh%Kj8=c37xbs41iVFXm4QXY zOuH!omZeFXzEhVn)%)0{Kj}{&0b@}fD(zmI)a~BwKOZ8q!%zO1zQamWYZ?GcM# zlZAEksK%!Aw1U?zc;6jCAYW`WaeS=UMx!#Qg-cIo#@-rtXwl?YryBLmrH#FQVnCbB zOyl(=l>927Rg`D{C6L0aMxa4`ke0n~w3$ zf_9$S84x0zArj5JC+f&n|C|^S@6H826~MmnUS;N8Yj_iL<&857dtWCiEanExPIzAD z(Grq+z}HWQ8T9YhVV9nFPwB~R7>?B>8y`tqdXEq(ukuhUIjNSYj8%!qnc9Ypbfy*e z&bkm)*dh8briJFWPClWq8?(sxSgSuI5>@peLwnlIAWM;(L5-1*26h7&Ta!R5pbBYO?y3zANE zb?9KO&wcZEM)e|wxYjZ*adO$|6v(YE@S%V0MhI);wW0Om`{T87wMX$Hb1yS$uzB65^3~lK}oCb z(&WXe=274EveIX_HhCsFJIIq)VjMZFCB8}$Oe7?)ToA44V`o~CivmYmhHZgdo21Tt zk))%?j{i*nQD2t~g7iGJ=^q7VoHaN&3Q8YY>Bv1$d_gV{>o?ZS%iVd#6-L35QRu{? z^Eisf1L_j=2rHZ9^@g?kRn3ZbaZU9qu_bus!%e5n6I!Gx+GhRF3X-KmVMkry?&H9b*7DsXRrb5yH zF9oxDAn)0yI-$xItB;t^-ap53=+U`nt8`9ejv z-ic7g(;GG>Lh|R5iSwW{{zxA&4r_HEniqyIT#mIm1M)10yO+8+s5-%=+x>QefT!9T zXwXP*X17SX7W}fD_{Qb8)32e2?mNBDUnxRzC^O{db;+0%x`TRG;@$Vp5*LRqe{u97 zibIj_Ppz2OdFXvAEOh61C1aE>-Pvac-0|8y29je{_p3gg$Cmh1h)e+vm&zb_(Ddxy z6y$|cSbwGu-{`KzNMNDmy%xZfB`w(pCcKfkI3ujHg}bzWKSnPWHF>~4IE zLG`05_1t4by*m~j$!vw*3gIF+tokO2EOdp^7Dd>b52G9j>%#kKi_qzBv>#r`ZFF!zR(XGmLGO+|HOuZYcoA-nF z%MVG8H4EMy;xyWrIaVz8@lM@2eBRi6^ZxE;%bJHo`c?7o@Aa0=**dG|5ni6z)ZK2@ zy7#&~Ol2kn_>;XXpG6w2hE09@ZeD8p`%|ZX(#@Y&a~HQdAktBP>qkysUbbAzt~=g; zc#T4|p>}r5sviY&RdV4P(RX%zY9R4+Ioih%Nl4YMJ?%KpFy=9vg7CGZsbOnocRcSR z%{3kKNp~;@qJy=D)br3P9M_XvJn7&bFNa@+mMIEH5mAiFmW}&ESd8OZ$uJQEjhqx0 z=^h0e{#0dg3E`b^l?e_sZI0=85inO<(h@v3g+x^>!j8?-US;2Ne9u7vnN`6-g&o;HAGt-`%iL2tEF9|TyE1S{(wV+=#+eDrJvG4FeakaMX!FDV* z>A6*g-?AVhKSJhsF(G61%2O66*4>--c5Q>A4Xg45G-$YNHY-@|TX-Ij5#SdtQM)J` z9D?KXUEsR=CPxZ<<&3{lBE@B ztX;K8ELb zk>=&xC2y?#lsULSeUWkU_Ae(gb6GkkENi=z69)EmtQnA1;?elG7t>^b&Podl*Y?S_(G{)Hl zb@RFX7-mHI*R73#`v^mxARxKhBh~=x3VBN{TLzD#vKxh4 zpCZy#`C#DACe2vB*3uc3sVryDxaG!lwd_}?!yI$YgQyr~5RQA*yL`>GwH8}m-a6Lq z>MA1k&tbsMo$p*K*}^wI#AO0<~8k%OXa|rux`}C9SF+bz|=t$~0KU<5^S)6jwbhrI??&qSO(rj*WA>nf7!o#k}R`XCC|U`?#%@W<0Q1Oky-BO z4AJW^VUIsCj&z=nm+O1ZDOL;XH$p50f8ZRV)+Y|{RAL7jCoDcn7O4$~J~ta{+6pHz z**G=@pLGRB+fSCu_TYDJCvbWy2sZSV!`4+ru*;uL8Z(}K9Q60yAC}5Y%!bJv%f+^+ zm+du$+bn8#=fnkEGaoMyb_(PgDtnD^$$0|+NL`CMaKB;3=$}?fWOixXXF4MNsE)PD z9q0pCQV*XfAxSzw>~#Dto8!N)&EN7+e5ZZ$_bMCIe^W7IRrY{sLlo&2!K+-Q#+l~$ z`6VY3jhl}t?&0tmnw-fMURCMZhiCiHWf<<8V^*HrH?jYUc$_k_S1q@R@>(&3nmC6fJgIX`s&!^GDU8tKx`Sie6=1=E) zwF#5s2Kjq0Hj4`I@}z z8Aev7$+|Uvk#EWD&h8{o?Lb^s!CxQ`alwBYrrq!6INkH(V^>X?2D+P*`FD?&{1YF5 zm8;^HNhL$_P91W|75%9qnd`}RZ{aRDx2hEcJyDt;YQzii}>2)bIqs6QtJmpYxv z%{|Y43$GdSeDx_mc)^dOe3>F-B$Ty95S>DSz4OE_UYI3#vvx`(yVq6NV}&n;Py2# z)m&Las3bGB1eYq|%dd*_u%?MZpOs}~-8cEHz7PMSS#vxinGiB62&KljtuJ@6sc9aK z`GP>9p$NQPvn4Nmg&OgNSBuZ5ngkL4?j+&e!#t;>mUK{UbHyePcViS5jyL;x0=l5X z_I4z~+_m-gCYe85YkssN^IkhQ%O>6-xC3@eok+h3+xm!5KR$3)Ac&NQNAf`jSb`^6 zcZ|IJY)g$(EV#_fjvxbDdh;JuM36rYEx=A&piO?(HX&dIho~fTx{zT%wcen#bcXshjvSCf6h7|3^R%|LV-p!xN zPn!36%(YQZ)xvKPmb=8z{B%#a9T*381CpRdL?=Ry2N|gwTiY+#`>mK?D5oIctb^$@ zK=5M>;S-AW5%s#aRQTs`zJIIWLsMo|<21)&CbGjYY8p@090Fx4~Q3Qod_gkuU#|)w@$(HYPpD_X^uFnKqtkxkI-*m+5|$ z=P#7TstcY(06wTj{?;O6AAr`85cb<Ow~3^cyS)(t$fs)Y8mq3 z4D8=}fBW1G-k)WKH!z$7qa)goD>IQf6`Ubc0&)8{$TKf|D>Bd$`@%xRyEg%9N+(Tj~ zOLS?8-+bxfm?3p50}L7`#(6tOWl~pJKOg@d;^tt06ag>3&a-SrY^`EQtK#B97t;@s z3OU=x)$(GaZ`(l(*l@fLPM5IZL$mp9id$#h0nRT-^`{q|AbL=4hh~^qGJ*v>m=ZGL zGdurb#W5sW)q>;8$eXJ#c6-545R()sbK=lR2f}IoU@Kt3=&%dQ=iHeo5-wzBZ>xP=_am=!?MrX`mZSithnWz0e_L1^gnjOGNdZ7- z;}_3=*;ItD^OkO9aSg9hR;>!(o4Rf_Q+)MLeRDW>*u5Z_8{I?6n?#l%f7^64A<~Dh z$jg1Rq+;Ta`xrv+Lw}BNx7lOrT$^x#k{{%8j2c3@KUkCE|Hv3M9L*=1W>HGLvVTn_ z6aFqj>p4T`She^x6}Ftyp~JgODK-L{JoUcz-E}?JURIYubl=0o!#AQ!(vjki+}cBE|gkY|ezBK7nzYU9wg zMLqQH=Q)Lt2A=;hMNZhe+OC=FS$LjAwQ3?q3SvW5xBMMMlrR8$A@mzaRNtyZ-LOG;W%Y6C-yj# zuPxEd1YlbwWm-JvC-E_MVMz+dSMs&4N`j_AP@XpmI9aW<=21qbjy!@){@f4&tWT^9 z6q}?{Z`ptUo!GcMBCQPsb1(yfpWBqCACd;WHrqyWJnX27i6VzRi200;IjRo2WlloK zGG0U5rsk~G_Havozv1-H0<8({^gG3o_O?UZTIw$IhI@~LZew|S(~8XHQot#b0ANTD{Ms-=(cv@LzLYaauK{q)XKD9Lxp@%G z)~MQ1vkpCAfam5cQM>!%Z>dwYb;hhXtG$JvM~-1Zs9FwNn;^juuX2?wek#F%x!M*O znyqTMNk$Hu{9gChkVb=NhU2e+c?^8y(zqiX(CD)q3U?~m4J9HW;T#o#_&CBxWjgHC zyv5Dtmgf)>j5p;ym zjMj!?81CCzZff#qL987WeOi-Kd*7D|_ogfHt1Jd7VRO298JRe>e7@LPrzi{04?h)M zhXTN@R5+HqW4tiCg!kyn3qL%fE!0sg6piYDQ!GN8d!4uhAAR9@dqGQjO}Y;Q;@+-G zNkVEX!&DuYkhOtlWI4&14m*$@wzv}A0kk|0WM`mw5WI0I4HN9*uS0dY%Do!jc!xZe zv&QTNR(bqh+ zZm**Bw$k=1(Ju4xGEBZgv6i(ZZf-hd=WnazL4E8^(+EGlThQjL z#Er!-uhbT@$v~$<9idU7v*}hY$V>f91H{x-Yrt7*jfvP1CU7XdGgB!MlxQ}L2rwDljT8|DeYxW zJxj-ONmhdFFVgy-PjI>YI+z)j8QJxX**4}t(Bt57weC^)G3TcBJ$k3?d`3}Qrrci> zmegl@-TsI;k{pb?o$XK-70Rll(EeH!TZ%6O{^fe7CDvo26n5gOWW%#L87qpFTHV69 z0_dRnT$!onl79kU0phT!JTMeyDo!rF_X}J)92<%?F$~MTaV)lfFtzz?cmYuNMsZXm zx;M9)g(5L@64(o~<7O9^&{Z1xh=HnI^T{48TOk;>p|xh3ti&0p(+Z!yvLn&);tV!5 zFIiPBf&A&az;|gR+DaWJCa6e3li_Fu!RO7Nf+@^vvDRsJlK*~A2IdRvHC}S&g|@#Y z2H%R9$Q7?U83y>Sgom;zUS3=-Rmo)Z)5l3(=;jQoJjeC79*JY$ww_KtEGKR!1tjcg zH5ny0=gdZRq6iO=?Feq_&SfpS#Dz0NRf`xdtzA_(Oh5j_v()=MBJ|pYY*q0r^Sw+2 z4fNofBvc~^d{wF{9Hi77T_!`rD@bCQ5M2a`8rUWr)^o#7js7S|TQG z?rw3ROGEapk6t^_e_H#1CynA&y29--;=4ryRlAt0RD%7bI*fdEGT)EOXTetUg-7B` zk>J5RnUO0g998FG6pHl(rMf8a^=pWF+;nG%JV6I{3(fl<*?b6(^)lNHuZmZq4|EG8qHGJO2Q6|$g7~g} z&snvjb3jEi5-%{qcjlPJW$;5k)?B#jUufIA1xyeAM+ORg=T1+UG^i`GRe%dWw7wZ= z*2msyxPZB&H=ck_p5^W#3TPrNyD`*zwh&`fwAB-&`I zzU2i*wHN4MG$FtK(E`KX34#{q{)r{fU!Y@tVEBxms1wneTNS~J`+%-c1x7w}%(s%=zvugWr~ zuQ5Js$XQS{Vy5o$KQiDb(xhPz&HTk%B&259hRJZlNL|8NHyHaI#lLeTIf|Gv;U$+S zfMjtpoPfo;=BXD=x9WZA8a$?BF2VnEh=c|?)+TM34>7%V9!%R;!)4u6zpFDoGI=x; zQ4vR2eekd9qaxfgbe4av?QAr3B?qYfWQ6$LvgKc81EFbtM5UzeoD*jLv%Z42(8reM z|4ou8Duo?;-`JQ%wJ4}o`POeMNMDP6<8bd^`xAs67WFw~WS}L$tZ*AlxfL@h{`4T| zZOlBWw_&uXr=xAAEVy7Xu|;^%I-MzMLnj%^O$MzlO)Cl5viO3 z=*FDn9EGr`XY3DXeONQLopW6ck!?JLoM%m=Q)nJfg))r)=ENyE9|jlxHMn#|U&#^& z4R;XZzU2lJXoC}iaf)L{ENKsERDw0+D|%epifV^SUEH3Yc7I5sw2JS4q_bHsar)gY zMw;Phf`N8v&2fE_akppli;)BQba_BUbe>gh?8B%Vo?zPJU^=yyh#N!lyrQK^MyU%A z)x)3M4=Btn4|9Gkzc?<~>J{+TOCKD>)ieGN`7JUj5mNeVw%zS#8)JY`OjTg|g+UIF zpm&ysP4>s6fl#ietp=h|4NX7#v{Arj1BI)FU3!o>Y*3g>ycgj_Vry-nP0S?&LW-I6 z#@~(QtBn-M_I8`5&Y2@0D);sBHh_k9LuBBX>ly%vOv-iJtH5)GKepf=QlHlS`wNJV=*A)%vw-(I7oB&MGFqJN{2kBxkU9>n+G;mFWNkW|D0c= ztSQm{v{23XqhhXem9NqDnuY!Q!1Uu~rfuJH_j(il<{oO+!<$Q1^G^igJOe^N$h+P; zH#4+c{BTN^KV)+iiiu!^OA-_=GeCJI8jUT8pDKDFl99FpwF*)tcyaZJJG{(M*S}j2 zItXA2^KN}=qSsi>>UD~+K%ra$ZK^i3fy6qE(fR!WuJ~N70#80+%CVQoR(UZUhroQt z5H?(Pvfz}M$ztTxf+#K&j{%-)e?UK@@e3@l6j8WQO!p0NePrYj($5U_6IE42y?1eY z>ict*EImY?n0rx9O9Js$rBXx;%JFa(+M7e^{XMSAqldYOY2{uwPtA3Mo3L zc6t9L+Avm?tMRCuc_4TD7bneD}OFB@x6ZA>Ie1|)}4v%zomL*hJUpO zXykuwJ}y3q0qOEjoT22>H_iy5* z6Ig=!>AXf&*$=UYpEhvfWquoezsc)-A$RJ>TfDHOnKClH-%XjnyM3@usp*W%wAO!+ zrkl4>4_3%+LXuYW4R|&ChmA_2Y!~cVG#V3wsC)kqO%yR*R^Q zM>!^KJqBaf>Ru6Iae&)N5rKF3ETKAZubOpv23hAAc986PhxtR%hOGx=dexn>ZYj~hKiD7C^+YQ}f%1k6=`U|-e~>zMY@q9+ z1QrE(*U*Ig`~xCG?xp!b3jW4aS+2aIL`IK0qu&1J`;R(jX=BJ_y8%A^sm({8o6dpW zvP4K~!1JvRE}vviu~D1L$jOI!ADMn*Ok3D5wJjtm0JV$`0NmEY{&8lmlfT+Hvu zogqULI3B>W)vb=kxOd4y>AeF#S0NK5maQw#1zz#u&dvTU-z4$uD89?7SK!+b^3@j4 z0v?>$Syx!ONQ)+p?;baK^3a|U2DR&L& zx%Hv7I0y^~RM|crAFxkXQUo`}Io2J<9f4@S9E--utH+bYZXuakD~LA zr|SRXxFSTz-c(3+jqFV&gb?D|H%Z1dvM;WCi;x{cQB?N0u66CbOZL9zb?tetd2#vv z&hLMZ2Of{l=bZO>yE8zzmc}1LgP}rBCscIOoTERN8Z|nD zQ2$QEJ>TUi&f5Dd5K28<`ioWg4Ik$7n+<5{mc>|&F*MDby2ey7&qmLV9?rP4m`{{i zHupQz`^_}QzkgjhVXelVv9n(OhgmQlI%Em}NUK_ytQZV|<`Iu8ZzFF>vn#dxpcPCL)u@BZVzcbZ-3YSr zZ;QX}*j|=0sFLtbnQnhaN4I4}PAF%-)5K8`WcP!k&K(*kp!RYvcy_&!7Z(M$EH3?$ zglc@l5y-n+aJ9*(JE$cB4#H^z!S2wi@DEA2KrvDqS#ha0W2Qlpab0RXNYkUf_Qj;h zef+deSqZ=4{>a%qF~Io-W{HjL4hSGW2(-yX{D@wSeZ834G7-s2b?^_5yjtak(G@Fw z+{DA-uq5oTFr=!hD$t{;RXyb*luSW_04F_@qk8Dj!kn4qfqn25Y_rux z8q%prXt5<6<%^Q0UGjb0;EMkao;kDa`TQiFb_swkoQWrxGgs);Zk&|sgu4>5u$;${ zBh4%@*JtjlrbuacSm_WF{j($?8p&1V8Ggx?X4agjKJwz!Iiw-Wm3)Wvhd63OM1Q*R ztH1wZRIu_Xa^2X&v_dRvj|XB=k1a}wpQs*v}mU_tSU z4*ixbQP<78#=9p@73zc&mqJT?y{gE)0ZBu$k&XVZ#j;FWD+NElq(X-Kwy-pdEs&p2 zOg?_kRuNaWtyr2T$>c0RN8p@v9x?V=ae`VSFhG%HZ#`oE}XkzvB^8Ok`a<`COvXTttgj&b}uQ)tpXCwjh5=>TeDE*?Kt+P zMCv)?Ph_GE;0d)H$@X5ENXeaj^k*eYr0%+n?LGeK~*IJ?=KF za_kwntyvlvZz8eI8r4kXdq~>GfHMY6rBE+&0@dlToAtN3fHZ@eAwk>IOi9X18auT= zpWHt0GV7u7%thED29T9@QG^5hjVxw_*k5cy6h=`(;%oRtE7O!ylRqv6q-Un$93M2h z)1|q#j>smu_wl3SFEa7-)GHD`^Vtp%->Dvfp1=Z z>f99O66Sj`UiH=~XG6IybAdHphss!ZeV?PoJ%6Ne(ZGi)?@_CeS;(C^F*)wNPPC9= z;0VRw=jSzz57C)SSv zqud5%lX`n+KWx|;Ej2LcnAY-y{9JlMnK%ti!($MNpEx~M;xIO^-NQMyPF_%@HAO33TK~V^s#s+Z@)mA|v*Rt#201vakxrfn$wttLsl9 z{J5+BB;BU+CsH`z-#|ICg1&YoS~W6E@0c6z^NJMvtCtZ|Xr%m6&Abb?_X5^-gz zyMfk1x4b#kK&J`7HUESd2Lsip3*Gen3OZTJ5{b7%Y0o+M-nQg;#@0W-q$U{y2pUi0 zvCHuw0LN(^K(1^P%{RNt%1w zCMY#4C8F>0QlaKkL>!=ya9v}{9TD|x1zB{vnU}2{YiHEO-st__aKhhNBp2r z6Tc$mB>713d-niWoaXTm5rD~SkPVVsKR&R{b(23mO^iz!iX(7NzaEgR0;2J!c zTppBn&|SNt4cNBqU*q&Zy+oqSYKFhM6Df z$PM3Mlk6i?0Lvr)6Cw3kw{+xI}}cgSN}n?0by_j8>@~i6wtSb$JerV2_kI% zt@}q;C>6)aU(Y$%qdpc@NkBnrf}zgGjMp4p>_kQ-vHJJFf*7yC`vLWF=}?0XF6W2Z zWclO}uKUDLCVDC?@8Tyy4F+0F__zz#Us9>XGvB_g%Pq|K9=vaKZjV@K)*xLhT|Z2q z8x+Y*H%%CEnO_*t;sShfz; zaA&zlR)oL8TaC==sBV{%9)r~vo3Bz^z1%%5I%`Y2cI*Dqau ztWVW8vr9XxlCG$Z4AumQMsFSdcpGwO7?7=siG!{Oyr329l$3f(TlRU3lUIV6?=gx; zM7_Jjz_qK7LMFX6%`n=M{(hG$5GI7=XVH6YfE~+%-!b05%i*u+;*dUXQ0}#1hbQD8 zCy+i6vQ47#2KAm2((2ET+1L6U89K-pP>|Ws3E~L(*!-J(bkOs)8aCr0c6`u}~7#=-!b5-Kod6i1VA+P`dBF=+Tzx;7ht| zc)^JJluY33^!Ur6)nhv5}GF#zHrmS$mQ)`jmCasr^F6yvALl)4%OMiiSJ!U?O@X zxZtPnRgtw{XeOe-H>j7;%-m0Vod=)odhi1+I6-c*w8VtcSXE|2PzBS*3+MFi1oY#E z=V>Qldr!%VWY8g~OMv>rcz@J2qnECvo+Zv!n4|||F|Hd*f2h)I2LKuA{`&6Ja4H5lQkQK5?T7-nvty|;J)wOVpsdv9k|o65m$Zl?{Bu;u|t zr*wK{hL-0>tCDMrO=hnxPWb%0GN3%ld&fC6-7{4UsrRMUaVg`mYQcXKmn>CChS`PQ zVzxJ{%@rgE!hq4-oL-CKFE& z+gmaQVhqcDNfZp*#y}y;a|o}EJ-a9$$Ls!PT*c{*iQsN&jMTmYrb^u~Y5u%@k}7`= z)vwa^Z~X?Nsi@*27<%k0J6^i@EP2%=wGrb6YI$K6jFPF*K0hwbY^@sj3p-aIdbc*KsX={7|yp^AnW6)1RkpYE#EfCVIHaiEVIb%jl+y zi(8B`#dnY65_c=w-9HxZ9XE4~aWn3$o$|8L%wa0e?nL~N*yg-YFsR# zG3O{4`k?sFL6T3+hQW=NH@hd9KHkkKJDJ=KPp`a;CE6+ETd_3s+&Tjzf3-<|FBWw5 zP!Q4UnaSkJyzl$v%`JI#*Z7Y1z0%Dm*QsA-{jD>)x)Frxp3A7_b6`Kx2|HXr{86HP zvSG6|=VT=9!tWV&$z|xOD(+wI}`Do;|e~>;w*X=~@8(?<8_PV&e zRDwfjq{-{>z4(5|)!KxzC_m;KLceyb4(NKyNR}-6$h+_OpHYO_nGr{iAq1x`PX+_3 zRIdi*%;L=PL*z%*y9P;pLhk)RBBem9;!6HMP->3p)QCh6Ve&i>*(T;^#AIuPB0ZSG z3*&SsH===Daf=h==9J51d!W4S#(xxuyS&~sxyZu7cmx9}pgTyl%m#20vW1rX41)|f z?Vtds@j}_b5x;(ix&4j50ovJoex)vE>E_zQ+1GcG{Fm2xE>m9r{u8UWRWq7kb=K3C zE-Ov)ZJM2jWzW4i}L^$?n^@T2l0cR<*L2<$J5X{0;=(>G_-~@DvpdE z#rSg8N2GaX>*jA3uqd(pQlBbS8uMRKu{RL?7Y}W-;{UM_B|$bq`d2)W;*mhuT){Sp zcJX`)Rm7Ejrx9{Zcs|+bvvq=C)ko25Weipw&rIfzp#^qIF1-^{WYD``pdUIEa8HmehjV>ypZ~oc&QWO;P<)b-AFb99nySkMzpZL|4D`h_(&(KaKZ5J!#*=5n>lkXh zBen~t&cMBbf@@6^jkZXd@M(jONYxnbHZHy}JA^=%oOb-#k8Mvfk^L}v8=Yt0eug>BS_h1nr_Lv3wDu06Ot;i3R7~QSYiM?>iqXId> zoj7f*XbLl)Yu(800HOR%lmyQa0Pl+h*WcEe&UvOJsUBR*K%sOa4E#_g0UE_`y zUrQSt$Q)A{@He9@LJ3{QMfRobnDBI}TH zKNo#5_!rjqlF&>uz3J(w|eRtv! zXXg<0&gXl+vv_QI5p$@IlT^vH^I2SHDkrDLhE~+uZZ`KF8&pCE%Gz|?G2)^7&g&@Qqz~pU+w(ZwwRe>p#f@%zj93bou4*$4aOSVrjXB%{X=UEX z=ZULMK$1)eeXUG5T6oCPeZ2E_-hT9-c`dbaZ1Y=ULKG*Mc(mG-Q&a?Eru+GK$BqVI zQ(cy*1}qNVn02nP&n~O=6S47XMEW=8P3RZQ?A&y^^$I^y6es#`bp7nrGV8`J?{M-n1p|hoiTwj}I+Bt<^(1H_t6!*!C``pAH@~ zTzqi@T*oHBSAUpt*gY{DpJYgpypt0FO6~TOKxkNe2idjJMA$*8ggQ+))dMPIL)40ltlIFzaGNVG z*(8mR{r(**=~Fl^2USxZ-;?-C1#sQ@K?kh(lb#H~?XVfyaVhAt5`py>Pyj9L~cF!I#I|v-S?dFi~p+w68Jm$A>m>nN;Lp*);W7A zm+?j_@+%&q?{Dn^qUoC;^)P}uJ%8}|oq12?K;AWtIf(1oxz3Qg?8lh)c|n_U3cZee zU@;u@Hi2;NH9GqT(5wO4S@8L|M(BSO?7nkg+q$aJ$$ou#b>D4ullk`(J{hj-wcP_f zGwK{e$a3E=)iu0X+@d2*!5s>f597I#6&3x_>2N=yG}HC<@=f#JQBd5nsHnhK_N^=` z_~Qanjn3s{rr>R1rI$jVLH#b+H7i|9p$&e8P9e>DO9h7fg2I_D9>dBdQwMKzNws&i zXwk|1w-Z9oYGWM$L&*7HWPMib<*kj$GrOP1)zzrg-Y7DxR1~`DQ9ZP!8iEUs<8by< zvH7#|rz*>}qq3=-7#Ve|Dq;gQqZgI#tfMol?0{|JU7C!OWujRhdR#Im?CEzaj=A|j z0Rzi#w+2bF!k=Y8YFMn>Xs33yIi8=td_pI&Vs|DeO)|nEyHv8Ul2*Xf9*lRyH9YGt zvKqKz{gE=_n0+7Jxq#*9=w7H=$qJ>;_vDYOkZaU+3k(#@1L|iE500k@9_4tWaL!A1 zl25UC>F@H&E8M5?!y_$U_af(SjRN4NJx*)5@pyI(k-GGR`o8;$+gC0=*33Y(!xdd{ zLvZe!#6MAm+esCpg%^}C)645MG+p0((OhsSY*&t7VzK|tP(05a?_XzcTi#ys!&RyX z0%=!8Qy5Ba-C0{Pj;BE1zW983*F}&Q&YlDNJq$GCz0!7!UB2JC7y_CF{O(+hOh_Ch z4tk*t<%kV@|I|N?>oR4~#HU4U{*x~b8P4DHNp8lLrLB~#5<=ypoMPYfj>!XbT7J=Z zDT5g2so7Ec1(zT8wVne%t$VSyz3FwAsXsOdd<_sEaQ{(6wYX%lZtys5=PvP^*o*%e z#avnKE%b0ulz0~uKYk$$r?~i@(b`NgRMmt)?ihgeYo1-sy!(~QMZuih?{6CGVuk!5 z$BNkPlAH>_MaIfXQnvZvr_8Z7yAg64e%9w56U}F1+f@OH;>#~3YlgrI6qSU_jE?X@ zpB}mXITA3J=>qq|J(V*d#X0@APeYw*UPKdmBa5dMsa6#Tp-DJqvFs+nOn=15!CJ)i@pi^K_bPBq%+^q}VK zS(aBNPlCbhx}fy#49wt9AhD1g!7|q$C>gD)OA{%f-oy7+Cpk&yvAHUEX-?m$<(%;- znwpXknPQxT*CfhSSbzI$ppXc0Zz-_AIqqjk?}BxxwJL;c}|oD#R4>eKuWb{O`N zLFFFGq-We#`zYQvnQ6P?E*Xj-{4e(H>guR$tnNDs7a?vFXcm)QPE}&Ag_eeIg)G+! zeK!}+fkv$IO8&=2?vO$;`<0pm#U43KHr%k>-OODQq2u|tRwM4k6fA-@zs+k!HY~u9 z5i}Gh7%IyvX^3FdwgOoFzg9pb!+0p2OTA zTwD(?wjEqr^EQG6kNt9F=o6$+LvJoXLls6(N=pJ3nA}l~_o+VH_BsZrL?PzK6HsuM zA>b&ArL|vL2L60Bd!vB5u@VT&xF61Z$ujx&y4swsC(l3&r@`b>2}|7rNBRyH-V0gw zsyZeY*1Vxpp51rRaFV2lKPT?~BC78%>Ao3rI9Kk(FF)Otf>@A6FR~DPWUi4$fXZ&6 z>G`fhTDjm_oNv0oH};RO>3y%tuCtjnLAt6_Q%4@d^{%zj)Tn`|rp-h`n8myE^Ifq$ z$IneXQl(M_LYoyXP-ICW5WucZBW&Ri*%IG`D-tmJT+2_5gz|&VP22w1E9;jUcu-{l z2frNvfZNfRkI7wGqF--yBRBf}KVP@Msa$CNG|-}`dT0sh1bJ`e+2%=WwO-xP|LqHR zu0}`(-)eJY^P9?q71ij?#c1r`ss7tIovV#~ojhN>a%y~j6lDJ&;ws?&?>weXPUT}& zl%~P`cBe9ai*vdY2=%(lPm7S|W%f9>6fvc>;vmPCG@Q*|yJY03Yb}wwBKNwIqeeT; z+~dQ|c>tf#K%DG8KIZ7?NH=|NfjIEAx4~E^uSAuwMAzZ7SXF}qmzo~6qGFygvN0|r z!vpo(j{5?%*E`s)2P`9I4>!PW@gpyNQDY|NPYzT;Rv05XZcK0*bBBOMIgmL8PJd_y zVj4-vBIP#o!0hlB985?5A`j8yMx+Peo#1S;0D-Uh2H~)EGvNJC`Z1^IOHGVQzftE| zXWHZq$NE}tUG+RnUVSB9#reH;;n*$uV||f@#CYY&T9W7@V>!y=H0@DGBfQZ+yKSCT zogRZ3_xLIENB*HY=8PNHuW49D7;z0sy~Qh2MT|9P->gOZ-ZyPt`vzmhsS0omk4(3w zI(GZSgX(RS|2+4fq?SFf_85)--Kf(gSe5XRELWtMd*9pox?>ddMFcee2*_J3pA z#N%n%Pxv5JN8ewRra?Ks%H!2jOijY7g6Ps?e`h?DGnyJStc(0N>``5}u5mVguot7@ zG{^;e@Wj(S-5J@KJ<{AYj8Ky3`ind0X$s-TF-2wa6K(6KU`cFGwADS9vL-~?_b59a zlST1Q{{;T#v;IX1fa3CCm3pZ{(1N^)BlJWI|K-0-8e;R;69oTMMdJP`>@wxFoTzjB zju;RsA^f@B@gtHYL2o_W;`!Q-`;QLls-{FQj$*FnCr0;XDt)n!r zcDQ3}&=c=_-T@>fs6d`lTl4zZ^i=ID{@)pu8G`~}WO`A(dD37Ec)jY+k@i;e3} z^-moNh695r+>cOW7V~0g_lJo*jfVk(fapUKjDIk7%M)Op-TR;N&l|8U8AK$vMkANi z%&5n5Wr}Cj+P$q_p;)sf{TQ#QOFUeLn4~1;0DEJ-#mUF6bf+Ok@>8Pbtu!Mq@ybkx zJ;)#QHu2mS)__ATE9*pwxNJYBTvaPRp}~pvBv8SPMw7vW@v|Q9NB`accdctw=c%ha zq5z8}nf53LKMB`l4$Y51e<%A86if&?J7|)4tjVvr@I^BS4XpZJz6IJSQGrl#u554yZrAjWY-t=E%;2 z`U6vojpM-e^s_S8@*MryvEa)G zS?h8?{g=7J=An16W3XN7cyCs!bkroOdPN=Ly5#XQ;_W=_&i9+o4w}Fn@jfDz)IEx5 zl?k}awQY-U4?}d=w_}}Z{M+z{rMxDb>cOv$cC4^x+|}@G{9yQ*2g><42vtA(w4;z8 zglpjcn#(XyK{vDe5qt&z2sM>F+$Z;==fn>;)99Qvg zr;l?;E&9Z3lT#8O%3Aoi92Z$M>S5&K*Q?y9mHj-fLv$0|_WU7Sp44xDD~-pzBJBPH z!gawEI4d5&M1TQ|b9^$9uIosR4_RFQxVGrH1F5%c0$!O0YVCvqI6XW*B0Pqx6BNzO zlhGt-=WrBUErc(!(eTMLwK6F5TzTsv;e6fL7bN)Km;!!Wk1%!H5I4!6Qe?&qSiZ7?}xM@FP zeD}MtLl2t49jkEfMLb=m6r|MdCc!SQz@^#j9KJ8{a?$rrvm|Mauw-rPQo8*TUT5;s zugCCyQB1~{NWH&n@mL)fc!uIKSQNd^AR0B_{E@&O zzRz^W;p_R!+TXal(<6`lWWiL`+RwH&?9%d}w?B(@0BwD;XZMqCWhf}TzGjXos~uh5 ze|rre;pI9Cf2%N8L8)jPc$%3~PyvP}4m9Bib%?}h@l+X%IV9R-weLtTiM!eE{CmWx z0v?_hg`s6r$}8PbwO%6?U(#K>3DGNc`yOe(W^;BtiW9$If%uRaNZ=cr61lMs&PrbM zwLsSsjC$^fy>Wu`L+Jk(4d=)XW@}An@e=Us`1a^zcT2|a9NYhz2s=MMw$FF5zZ86= zkrL$eURB0X9NXs;3uwTz&ZA$ONALHWjAD*Q^XWa?a(O&jv2{n=L1snmcfzhb;<}2t zHTuyw@au*h631YUH+hEj$e}y3{?_T=@X{&XIo4`cc&~hU{+82RkLtwR7<%1%PCM2- zd2hbpqwBIQ`CFN-#bc+zoeJ*kSt)|1J?svx&HS0i8xpxTVYaH7dB{L}y>8w#KnRSp>;JKDS)xu9uSLyW5WQ{D-)x4& z9Xq(o{=mpEKSl&*r{T3&Q*vSs_2RO zpg9Pdk?mITN2a94GBDv>+|r66?#FR103OP)=c^9J*Wk_kg1@z3mM)ar+CNc8_Jr&YMUub#%$1eTKCzQ|BFo zxKTC9EdNio3&Lz6i`Qp($@}3Ifib}HMGEVh7paMWu*|C-1H0|QwY@8Yaw`{hdL>1^ zP8H=heY15v;%B-LobHr8H7uImo_AjeOS0g!wH15zOIvU?5x>eSp^K_LpAn}k8qzgx zl-MR{tJO28q%=e7Oqu1Zlb2fEj-3jA;gho8mjjpT2evyo*f#1b_7@Kz&8Nn?_q5l3 z0=*I(kvJ;(18er%Pb%YqVW1PJ%+3B;`VqUWaOJikZ-8A$@ftmvqzl*dPX9qV$aY&| zb4#plQf8t7)+ANuOiXFM&Eq!g@6VABd~yx%-+d~oq{j4`(0_rBo#}7Drsf88&ryOM z&~RfyPk7aMb){d_D&hzEdOgQq-#$qCcxv{P&Zs_5@XR6OZSd@r5-!Jvusm?+c^ zF4HfvFnX1wTkd2ZE*KYb0a*5R3H5`Q_cOqlq?a1fe6jII2Q^)Lg0KY}!q|&h?v~2A z6t)^rlNyujM5?^m1H_JmX=W+4Y~dyGyq75W3BrtF;uV) zu3QQC8^qZ{A`ybnfU}}mTdS7ovFm*2JZBwt7im=&I_;AJl0&=Cy?bcwukf*M9F_Li zrHFFzd3B5zeJ_D~;n37u2$?a?=R@NE_A4L6`p=ED{BD^79ZOb{{BnBml5CIA1pSX>Csc>=_$ocvyW z#(?VdNBO#}VQLQ8Z^GdpH4Q;0ono)JNl@qSZ4|0iusW!9l0ZLikl zDf=4o2AQ{nY)*UtOS1(BKl+bieOU0+^)esZ@=BeT#4^}+W->QdaPWy=1y9oX$<73R zBa&CLqsom#skB)=p|U+tT*fEOw>-vHuQmm^@9!JQC5X)Qi)KjDlx<(rcBDyP|H$?{ z7nE+`SAHXs>uSp#q&p;mr)i*6l@%BGjB5LJJ{h!~kkG=t1Lyv&pFgeF?8A^xlZ()q zT|h^XwJvfC+LLB|L*w_Pc#>2xxmS zbe>{_Oo_{#^wa8E?zo4Vlv{yZB@5!tm1c~`Un75BY>-tm0kx#B(j%_2CRILWwr4@M>MY?!xqp30}^_BXU9d zx9bUqPp^FAUwVIZz=&_Z`p6%K`kfy%TzinOSrqD03*-O-ds^&apnhNLXGS_8HOZ37 zuWifmckgLqS0`_PP=MW1^ASp8`-5QXC;f) zV1kO*&C}z43}j?K6JNgnnR7)o2&{o`x*B4>4%qQfpH?u(91L2l_ zQi1dQVGi)0AvA0TAVwB-WCbBJ9Jj~U$kQ}?Fc?5z&AqP~Kn4*skK->sA;kXgvFan_ z#rXv1K*7b4TC)XjRM+^amD4Xj=8&!tC6Lj@OVC@`h;J}?u0I_=MdO;s&+G{cIJQfU zI1fzm(~#Ccx7mutfYL?|5!_0@K(|E^b1^%&pctY9Qfbh#olG#FX&?Eg)mQjqJnP*P zAnh?krz&0wJ6YQ6Dn>~hcX^0?^751+4_7&Shy{#7`LCFB^$bS_u?)jnDSXQp@t zZXs(C^YV!fKvwaizUG1ZVeVHMBahi58G5 zq`y7oFlA@As6|T?xy~X#UR99|44R{3dLwii;TonXw+4DU4 zevLij+#W!*S#_u7k%C)m-cpUdec9lE6(F4yCf5(eoB1YWAnuR@#+wyj2bauDSGC`c z8ozISOP_itoe7VDt_z1fJWTCw^ckClg=H2z&d zf{|n?*lQhC*633SoG*m0G;1k~&${$HFrZ85JB7~G=4OIU=N!l7hOf1EvzwlbO-k9# z&KYKd>2&R?LY-+$`Qh6z+0^4G_V$oA8__t`;2bONs=}#k-zCgZ7JFB&OTS;`3Z2mk zBr6k0SGW|-Gdu}YtxkW!462CW*fah1Z%bry9^csk9{=3s$vqXX1}fCxAoG@PG398g zpSr!I@i{>%6O34$=kE}Ae>mlDn0}?JeS1%NsZr&)6i{s15~kBSd0HaHuz2&7>j#{B z`|2l#<6EYArBOdh+z^w3n3GTdzC~j^DLc)vx;n0|W%96RqzKWAuRnmCB0RfgQt%BhvfWnuT;n$-Z*WNxd!oU=|AW@%5G+L>YOh^$uojs^pJ< zF!`5FIM!5z-t?TBC<|H`v7%=tB3&xAfPj#ra+&58LszsB` zPXOdn*N@!o0e8wh-#;A+c`8?IE!+fUM-s!Wz89@6%rmCX*jZ>)c8g^jhf=HCO`)@W zd+)?CPtwI;8?)Ne<>FIvJMt|`dvPx`9_)&YsSnbAU^3D>d-nLTcPCNw?tI=Vx@cFE z%kv9ZaIs%{Ye_I_DW3T<(hcyrc?{q~7k1H}$lSo57*#E=V0bgzDDTPWE4{w9r-uS- zKabIvW_FVKLTfW8XTSrYaiL|@9fBP2Ob0$+r211l* zgFVg{`4^Y!zgmxu7i8OoYnAppx~}!c;<*=>8tLW_cMq6??sudxYz8ruZ2qJpXkb`!IiSCKB zD+M}fUl?^<7n`q>RU3%%Mf`1!3Ew9tl$TtTpBLuyei~VM_mhv(aw6^7x8d-)>FKQi zD@a7i?SZ@Ze){9t_aR-LSUN%&u8-m`Z&*UmnX>zBj}nmz_@HVzK)1<#mkO)}8m}K) z$|7$n-_4aU>i;l<@gCI=pKy&DL)X8YgEpDj}U)3tlVK8vlA$%}VKHy&;@Ud+0KiJ4EfEZv7+Jc>q1yZW0gb2YT#F0L zS1LMA&KnY!-tvhI#jBw74Wpl-mc1RA58KYboH=k-qQOyyt8&No`+Ws3+sQVKPxae8 z%M)iu`I#RbR&8FR$u^`9c$L$f;X_`e1T}tTo5A!;^08O>11SVKUP1rz&&c*be470W zK6&in0;6bzyZgygI3d3fooGJ{f1E!X6SJP7G>Eo3JHRo<*T^Gm+YfG0q1c5S>t5#e z=az)f!!>0dop-`;%HK*;_W|q@0;KmTs za~(wAzjRfgd!iG%PyQE{2ijjSt&}&ZVwIRMq&)*~PKF`fjOYO-3BaA0I&qJ(^{-H{ zI$c-_Xk{b5ktXH0iF&D4FhYyM!Giw3`s}{gm8rAWWI}ZV+P%!vH-qEf+^uifMw4{D z?Hu*$^)`Jfbv<6ix?KP84ukcw{M{9oo3bQrEC0_|uJZN7#J8GFHE3c-!jB-c0^4@u zS2~XJS$xADhFboDjF)!-Csub}+Pq-B@03rK0BE2@^ebVf%es7ATrBr(t?iM5Gm2?W zV|o%W2o8a zK?8Z^zrq2+2yXUMr*oCdsZ1JDu>(csvi02eD>LcmBMasV%whmrA2(6w_Mtw*|VS7>1D-CivNq(N79p4yHzff z9kK0?Yd?(CNJu%yx zv~)GKo)sqg4eBMlsS&KIEZ++_e&=eu$8Y4itWj~NMp%Js8#D@Mx4+v!4dxA9V;78; z)pVo0G8zBC-(U7iN`k!l{koy-{YTPIm6_YD+Bp+pDYJT%oZE9nsl_h72fS}gLdyI7 zu!5m-!WhniaGM)DC?(@G8C9n4tm`NbVQ8PzQskP z-rDXVmj^E|b+B2t?TrSwFpTnW7Ebz<BrgK z-J=4lnZHYQ{i0gRaVIs01+qUrT`@vsZ3ZjYi8aPkV;1Wm4}DBHltJQbUjr2EM+*8~%8%*hM|bBU z#`p^*Or<}>nt*xkU`7M3bot3+sT7RAzBOo#eKmF_SO49EecE~^{2fJMQk`l~?*dPY zBcOxBP9A5@@Ba~DDWn1IK>kYZi~}>`Jdq~!P4fht!F6>ksml!32K?aVYU-3ObvH$< zFZwp|q-}wTR61amP8G5hN=pS6(bKWLp=l*UE>RbOiCPdYX0V$aow78@5vXQ*dQNVzG#*q-yKJ=!oI9yQ?geq zN|$OEy==V0VulWdT(T%h7301kG6TM+xgsG~SDW9|SDdSzODrz?^k~`KPs!b9Mqx1c-J5e+(+@cgt(fe~k&$7(Z+zaM}BUfzv(1n-e4r$ot*(qV`=tFT z`?F;Um9tCsgUD#Itvd=@4%KxVab;2bqTEa>o1?+m^ce)Dr*uc>{Bfe#M>sTqE`|@) zKt^rpI#IVKleeYZ%4m4!060$p8a6IJd4CHia=GM&nQvKLjuYJ&q#nF^L2arIh@VFn zg6*Ta#W4nqgMQI{b=H)0*_kvLKO|!#_mY|##A-rl8v0O?YZ2|48$=rR@ckHX{iK`y z-#7l2Ui1S$LAe>VqLmbRjFF+;%_%LFMve^b9_=kzz;#dcySE9k|1uNi;hf{y2^%jfxZb8(Kf zKm~#2BT8&pqa2>(D;_jy(ipbG4}>&4@?)bMPAF%qdZd3v=qbnD zneVZ08y?tS#A(cm8`8m2%=7n-%hwi|5rQ!5HX`lMiB$ZH1l~;bqK*xSBcd;;`j*Up0j^}Z-NwZ5m53=k_;X~v966b z8=C+yhzHm5ycN#r&MD2ue1VVu>a(6$_vb~DSCf0L4RVt~_(4{mC?{>+7_9h>5ysG5d=tp7!}b&AmS*- z+Qt*-7_Srf-|>@6_)+1RrfFC0av{L}=&j7QInUZp_KMQ}8R!$}x^$r2$fP+3r?q}K z=z4y;@x#UqtZHn>PL52U`fA;P_N+~JUhx;l&m749Ykh?uGQVtk3jIO&74Uw~LDOyT z?x7JRY6MZ^?vGRNL+prJ>h?W1;RnN;e+t+~dwCkiCc%UNe(iDhMSDc|;$ja#qY67> zv0m(B%8P_fl?R#VYe+#P4^v?$+}Oo<-^Kp`iyBvebU8~%1;Jbao|U2F?;h#?3DKm! zxN*M(DEe39_r%YMI_JdC6f`iM(F4~!U{N_)5p8UUJW1lMbK;kYnPS~9&A{!=X=*+N zlfw}P)a68zuRK??{3G!0kE7WAs^UGfu*lpwIIjZu$#>#iTf~Ca+&rpI09y0iOUlP$ zr^^zg8g`+qT%>@X+)^SoPWZf0p6vm+TCH$H(vG9e$gM>I%k3Z0O1huJ7)I( z0Oe;Nc}Jym9}&D)FT-CCMP;Nzo4LMJAb0irE622bcf{kulYB<|JLkRVempqKeVr< zK4#v#4;ZhtzAAYC0K%G2h4kMT>ufA1nRl!XxN<=IwfS`40X`@Ed-$De<2?q@n<*z* zGUNCZ`hQC9!}7WF)HyL6$~J-fCI0{f`tV1BbpHT^HvS*KD-NM7ODvh-?mXAoUl8z!Dt#vAnN2LljrlNWG?CJ1dK=@PPLu+RW+eXRC^aR(i!up1ntRk#zvA#L& zSUxxKo~5r?gwrpIJWaYVoCE1vUJlZHdj1DR@l?un1bw3&F`g@erR66hvKCQD%(sig zR=zH}wvtmVtjWel3Tx6l1)yJ9H`#SrB9WPkDBaL@;5K7hLG^*$=g>amu3zcq5NW>9yTyv+keq3&z7+zqpw%G?-XG5r~dP{{U!b2iuD3e_@~4Wj+>o{{UF{?ePLcyw`O} zF#+5Dq5;uI^RB!-NF?IcwhFZDl#%*js`!r6;qQZl(T_Fl;S3_+GjU%gc(3+|*StWw zES?;*c&EO&3Q+Y()bo*FLHtS6uXU|P)M@sjQ5YfQCm0p-)}d$OT|31I;w!5hJWb_B zia6Vzyn54Da!a8RQBiNH_KugTnJq-Ku3G~&@lWkd@z+(h(WUTjhDeTEpWQni3WMBN z&^oVQrlYKegcZvWI8%!Ju=w$%#qm4D@#|k|ac8(m<;HMi$3e%nT}deDJT*J} z%=$y%6^wcx!#JhW!%uWvhAww*B>w;^_dBcjEo6#4LNm8+U(&v8@E?!-G2p#M%i-pQ z0}G3Y!<0Pj9CCliuYd8a-2lG^L@VUhvNgYMuwy?=J2T^Xl_9I6tBL4ux?FKz}MM-6c%2GJ@uB%`0wS|?dTiy^BR3n}% zrtp@X_D>Jnx;7LbRuf9Pa7RZ07zuus2uT%^8WzXE8|~|ZT=M6N%3JBwWw4s^Y@4yGsS%$;mu22u<-;M zFT`-I9;ORM{{X_k4xH8}#7~C;9~No4?|>pP!xR4ir6MoNQ;yZ`Me;-8EMMTZY&c|Kw>%)4Vhiqe-{!uKijO28!Pl*=VM~U=F z^u1*R!pnvSrZHN&cZz&3;+-EFJ;F;Gox6~Z2(OKQXph<>z&{8y;}*4i5oQDyByANE z>{KG#yFN(qZ|wQ-D^Bq(uZZ<%ZS0H^<`KfPW7DTv`#{|49yYSlAn<&`(@e8mOpY_S z4@&$L_@(eaJ@CWfJ)VK8vi-I= zRV4K-k6PYvjA^IMujCG+a<#NQCg0*mhddFdHj}Ab`JiOy9V_K;7JO^?dLHj@l3pG-Ksn$DdmLaPS|8mriK!uMX+(KMMGt_CWXnYw)kZ z@ZV}fb=}C`NgU+=06|}QSWaVcUrR=eV3J3ZTM*>+J;uXQSmD6nn&!M=<5+w*rW>go z8K32eJC?3)U1mKh%1uV~_C*rpljtkQd?RnFd}r~{xnM<Pe$Mwkg@1Y~-5HFw6|Khu5? zcniovBV>pmQN}$Mzb8CV@e2O{#hTQ*#-%P@$-wj;m5mtld+gSuOGJ8K_Tjse%#q76 z@0#3#`4v3L8yFSgx<0a6OcTJb6aA;ZXq_M72Y@cF-e8mYX8<2+>9(eA*&ilJiyD5=J{$itTmbo1zxo`PzP5iv1P+rM@~|LE)=^4)|bx^6cyz44$!h?u#TTE)q0V_sKfb3;*o3hP!F zzCKz8+DZ1Wk^Ut8=nx3q8 zCq*5r=eN|YZ#4y+7ao$F@>4mFZps zpnNRWekE%$X}7LpjxmAlUvhjO{hMsQ98WH%1iozSINjQ+YMf%%Qfa!5OR?)MZ>itv zvD+0MNAYK+bN>Jl{v3Eq#Bs%L)}nbgV~kg}+Cgu36gC%55>vrB#%rj!_c`GdoM0{k`LdDmBKARliX@IIB&_-gCKIyR*DT6z|=W6XHtA5PWk zI){rE{{UN$LUFy-Fn07HbgWy?5>Mc{(A-+)2?**vD*2yNK2@?jzv7R>FAjV|)8xOi zghO_!c<9F7!rC*aqV^S`(z3<{@?ANV$YdVLARqicV z=Wn2|n&OJ%OT9MpsYX+R4{TR_WTdW-K2UMdmTgSCw)0DHGt#!LXM=Xu#ZkS3%lX0J z)m)P?B-$}SV0u(EeJO$HJ_z`S;$H#kNvLSEw0jsF1Ldss* z1a%#&@PGCe{ki9kYkvy(rhGWUe9hLse)K(4NAa$e9sQ#Q5&%IJ%TgS!$8|aLBh36^ zpm=-5){OC^PDw{>3*&uGMfxjk#oHARxq z3Ka8D@wAGV&QaD3sX{W*mcA1H*_!8vBu@_fOo+(};Y8K&~zEm1M$ugp19)Kwo3LGcgajmP$vG zKUOZ}k_0LSE6w#(R+n{h{{RWTI(To#_UWd0jlZ+C6+2lZz!~<mhLB24JR3_{4cF--p3~Y0H2Xr{#trf2>;Rka(oG?#w_x^`d4GAwcX@^ zXN-PT;a&!J;AG?f0IgpAW64bWR#xhD#pq;5XCz7U{oz^~U9^+k0OO`HTAGj>O;4kJ z_4?45a-QkoJx*;dG&W=Au7_CEB)*6Xgeu*!(zw45@p~8^mFPOh{Cgj$tj%qqo8gZS z!Kq(<{Xk{sk=DJw>rc{Qfgy=NaywT6;h+Y_0|x@EU&$x-w0xeTrLvOHlU4X+ZlHVX zXhulR2YT~A_(1hz_mgfqap_-u=%|N6K&`B)3<>0ak=1RvX*b}l_JUbSIwZehVi`@Vv;_J(STK0vbYnp;R@ zm2xXe^TW1{8A#L|^!Be?@kB92b%>NG$6<=#PyBwVs!WFjv^?RwFD;q#k$4@ew9xhW zZJl9pfzzdQ_e!TAfzrJ0ZZ8}T0H?T;w#U-C1nf{q4>j~>fwd6MBYqryL|4h)4|g_k zl6zOtJ{KfO4n1mG=xBW}VFb?GqZr7qU+|uvsb0vjvX(tDTvvpYG%2`efnQJjK7tLw zwk~t&?M3X0vFlz6@KiSN3Fkq${{Tw$lVJ=oVE`OeO*{c^pjPkqeuBBytfsUu?ew`O ze5kI1KL=agF85FlD{n?sGA>1S>Z|4Sp~;TNjvom{rb)ge09Mp?Na!OGOo;Bis25G)1uzOpm(Sr_zSKdjy8`0#-y(laHl-{{X@U>f$_g6i;H?)Er}|tmRD|6st6Ulqc=O z`wMvU;3tc3FXpq7Sb-oCzbN#t%>6$_p4U=MM^(2oqz9Cf?&Nd_{44z${{Y~p{vOaS zJSTf4#f`hL&n*&@)9GKs561gDOKUxGZQ@A6-7W&C82qZKX`t&y{_(|F_+!HtHnMA) za9b;_o@9Ps2*(`&{*~w2UyWzg{3)y1>$fbDsSRn4ethyhxUTa>Rg>T~wbX+$CB%VT z{o%)B+P+rNiTiU8tqPv5*HA;`)+fS#&PXlZtV3w z80c~OrRJX-Pp|6pc`t5-!UKc!uZw(BKjKs-%sP8Ofv+)myj+P}5&2Dm_@w%~>PLn{@Adc&CiSSRo zWGM3Cfus3O!PAQLzZx?&-krGT{T(Z%$CdJ}jA{Go{Z)?)@eF91<-~FQ-Q0aEy6{Gm zYoin54-f=vSjWp5>d)Kqt|!EQ@)mRS{#7lrGp)Kg$lN$RE1~}Y4SF1RQK@r7#M-QS z&xqyJql;?Ejib;G1$~q7d*FYM;n2U~G_`*%_V0=%J3}78*T_1eAn`m=fHv-oW4CJk z+win;#o+rzi9(MdI0GNUUS?s4bm0icsp;kPBRZ0D=zdWAT=9>`6prgn@m-mU&QIan z@`3DYH}-b@owaX`-X?_XXzZh5<~BG!)%M56wHIFtW}4)AkG~t6lZyKD_7>CZE&LM; zK?G=Ue(HhztET}q?Au8z5sjs4n!BH4{2useqx>=O%o=wST{k<=*RLvU9;dvO`abF((%y#fd@wAAnUgbk%4oAHf<&DgnysUlg`#pRMveIlI zyPiF^@qEanV|o7be@gpmOO$~Hqyq}02C}q8SoEAmHh?k59jUT?!%i_A*>p^&fn3ju zd|hGTZ7J{WnN%n^2kBZ#_YHhw`%c_kC5(!Wzy#0VE&voe-rlU&ch(oe<+nDra$o=ciCZ=q}5l1TkP z)h@JcF5p{R%tgcT!LB!3vrR7cIBbZyytp_Y?B|N1bMtEM&B+6=;ayL{c;mS70egiG zMle`s7095Hxz&5SA3%IF_-m(WntHXu2fdf(W*xn&?H>ntP7NlvB7Guh370u|;9wy*4`)5U-XvjBOn~sZLg<$|V%%bK5*bgGtb4ljWA;70P}U>3<7+ zSi0YbVmI)tsL7lT$I`xO)@~>M*wfQ{VW%LB@OZCR_#yzmy^IVfm9RTkpE^p{J0muG zzlNS0T}Df5*ur_oF=X+NO7`1-hZncfL3J$ngL=AwUMJ!OU$oksbY4BH?Z1a=vUpnG z%p1N@*15TCX>*X9$GTRD;^@3E+H|Y}jNB?^1$TZXzK-G{J3y=uaMjdY+FF?-CN$-L zx+{p)IJCG?&=Y}8+Q}jLocD!1S!<_x)_U44xskCOXM$_nG*~Pgtg;a#GjK@cSIu59 zf*CYRb0RBYyNdO1hR**0@Qk|Wg>F5#sg91o^;bHt5cpG0x7B>@6c06%?sg}&ajEfh z#2y*c#+j?F*|7z~bv zzPDSerqkiwBOaaotGPlcBhQ?j&CBnJvT2rY9+MZ4!FU}7dFF+$>K-N5(p%YgvvXfP zc>4PFJZCfuGlcS+ZZm8Babm^@i1K4 z6G^^Px=lkt@b;Q+?Ff_3$x-S}a^4WR@fXG2KT*;3_@vY0S7_oNcXD<-R4~H{zqLYx z8^;*0rB*oLMKGvNpLAlmt*wrmD65_u2ZH=N@T0<#SwSb26hM+k9M|K=?P=hsem}~e z4fK1EvB=E+cdxh7zV-dCwM|YNE4ZP!l^4ox1CT4=PuaE@Z$2i>F41KQjg8|t=eBB= zlIL9wq3|?)Gy5m}2%F$X!<|OXD1O0dBIGMK1;FK(>t8H*>t68|z2eC{xr~sbkiLKl z{XF=SB!fwg+%fX}f%5+V59MEz+T4;u;yF?hPJjJU%fB4Pg z8|_zDhrpTv-?@UYAH?6_E9vipl4-vQ{4OqU*lr8BvFgv<{XHw(kegVW5szPb@o26{ z3Ny`fV3BpjXsJ8D3iV?Ym65@|rk1tiTMr57^J>?Q0>q@(^K<)Z{?Zzc#jhC8q*+W& zqz#M%#})e9`(fSPTl_ws?D8^vt@lnldsp*caLp#KGe~z6DeForshpn3^&f`b1HJJM zqZB}VnNn-%{{V@)q&^Dp>S=m(n-xbWJDQL7e~v%w=aXpNwDXGbf7%jRKekmj9C@RP z?C zJ69R;p+JqJ1LYXN?b@;UrDkceJ?r7e#;pU!PPR56uat0icdiTeiuk5)ZXxg&g{;G0 z9vQ;$1$a-uk+)ivSAm|@@7^c3gZ>hU=GeRTADaU;GK^s_Aw{`F@Qb}N9}j4YV{${^ zI%UZ0Yvs*eC%C#WS|2h}PEVzKw~3^aWz*#62b%F6JByHe*H#&IF~oF6ZJ&ph(m_6^ zcpYoiF8q0I7MX8hrZ8}$1OZUGBf#OnJa??!Cf3*)F&gka1!&}+<`G=G9^>F|i#{Fj zOfy{SC<76|9c#sYDEwc!@wTNM#*#dgUN9@nd}+gR>s-%<0BvFwVUS4U@~fqXkH#tD zCvHxM>i6uSVGz*9tKy6-l8naf>FHl*UfA40WhB-Ee;F9`uY~R$?K~f*i&A31Pr35? zdRN`P5SCW(4YZ8Ju>+j*(!A=A-q7^v#?i6K+u41scb5Yb6$IwKx$P~XIF~xlY;%Vns(~vs=Q>JPvD-I8IeE$G#;t9SE z#SFUihU(^GR6AGDR@$xSj(jw_rPNFP;h$k%8}XtDCek5kkywcr85zeSyT1usTrY-E z71$_@@t!EP^&Z+Bo!z9%to^RvB*t;HSE6aYDUQl0(r4T|jzkTS2hZjm*j_VO-wvZmynm*Nlr|T*&w6l|Ca8*$Qb)nR5WW!j!@)Cdxwn(Z z?7(8Yy5G-GV|xAb_{C#oKZfM3 z&dKCbB*sN$&vkDd$Jw*U^sQGUN5yfofOQolR^|szk+bxqngJ2R^{Y1j0C8#?-}R<| zE@)cS{(-5(s%ea4mKFpNU(oOD6Z>Tya>C!kJ|Um?$%Mf^mHg>j0D9Nazq4J{&9}t6 zFSE$e<;HfLclM&!Rw@#4dLP?1iQ+9cSF}lT8C^wV>pmKS@U-@yDD>+ zn;>+ro4jKjdr!(b`cUb{O3e*P$5eW5r{J#${6T?kEgVI1z!a7vf+|RsaWkQuB4Vls^1J}o<9+!XysSX918os;yCU!mPyn&Dba>|Rx!m?!#-QyJ&KhG zR*mguem#61_|DpAicO!0By?F{@4o~!ezSOY;*>hpj~<_@ubUev@}&M%`4{^~Y1(~< z#AN#n5yPHN#S7eeSKPk_K5qir7Uzof{iixpJlU|)H%%Y^**Ue`H~;_u literal 0 HcmV?d00001 diff --git a/quarto/integrals/figures/johns-catenary-3.jpg b/quarto/integrals/figures/johns-catenary-3.jpg new file mode 100644 index 0000000000000000000000000000000000000000..e165be64a6015877fe99c6d427c6ab1a9571e497 GIT binary patch literal 30906 zcmbTdby!=^wm%%)p-`Z>7mB+Sm*T~p;O-FI9okYza4k}bySueGEtcT!4#i!5`JQv{ zJ@=pYdG7PBWbgT8_L|wVW@hiql6jeXSqJRN`r6w704gc~761T%4nRd903g8;0{jC& zAOoQM3j+YU2;~2RO%UGw7Y`x;fcW}9{GaRr$p6I;uk&9N|6jlVoub1NGZyB0wNM1G5TMa0DkRH-hcCOb8+P%{HOLj#Q&}xu`LhjzcJES0qTG8zzhFFQd%;K zittp+(%stH#lz0k699PGKzyetBV($rsV1lRQ5JvzmkyD{+|AADp9VX-c)DxKOVj8Z z7}B8a{9E>aYBIOCA`v(WYF`+xWQE>cE{}I~$8yEb8E&h#L{>7uMDFf&E1IIL$cIH-a zd<4gA7XK^X_J3h#C$E3*`w#uodTeVK9W8iD2tOzQ9{~0M4}crM31ANJ1JD4}0lfbo z{Cxl7mjSrKx%~j{@N%{QJAfy=j1)ZQpBk*;SP|d?umo@b*x;B4zy&{e{^@6USvXx! z|5yL9{U@Uje_-vlo`0DN?WLV ziiVDXiS-JAh=7EIh>V1Sf()PA2tofmUdZ?;1hm{zsIN85(da;gJYfmn(dngYe-dd< zo-*)SxQAn462BoKC40-r^p2T@k6%Dg=)JJa2U$6J1w|z-Z5>@bc(+?xS=-p!**kc6 zdU^Z!`uRshMn!-6{3RwaDLExIEj=SM|3^V#QE^FWSzUcYV^ecWYum5hzW#y1q2UoI zY-)ODc5Z%QePeTLduMlV|KRNW;_~YH=JxI${*3=y9QgU)68$fE@Zs_xA|oRqqx~Zf z0;119!r~*N&~l>^NNJ#%gI?3|grO5kCw#B{i9ye+c}isAK8Z=pz_>rW-CDH#) zpz!}+68(=r|0B=K0stEc0X|?z_y7sOBR$MBD;M`sX5#XJn?SZ=zVL+JxNKyy1)&y_ zp)}jPWYj=h{q|3mAu5aXE_GU^Z$o?DOfbpe&?4+8Gdgp#%4EUTC|8v{=Bm}RzEq75 zQEo#DwlB+9wa5WvB~<%db@Kg!R$g5KS3)@M3S&$10rF5Bs}ntGQHw*gR6&vxo~J@o zWSh9W9D0OT=+r7gdZ6gY8?~gy)a0PZ)H=r9Q)RvZUV6o3F~5hW6uYc+j&8w25D-1^ z?na8fQV~c3y2tEqOSDOfypwOVQuJWZBw5)j+3$;jtm_rS@|##(IGL6GWFLt(~-_R|)?uJKvkB&9ll>7xPJy-JHbJMElHRAvS-nYdPrG zkk7g17%V2h6GQC1%h*Q$%L!(}petcj}KTvdG@!jT*0WEfPY$?!SF@_E##u`zm^Sb~89)E@6x z7$H;l+RrX39DL+I*(}LyuwXoZkX(;6*ywl%dN)k%)5QGM(Gn|gL}84(BW6;CZd_d_ zW?^{>Txi8ae^LBhz={Paa0zda@UEawNF@(qm@0w$t1{xXIBKvHI3?8yN=ml0xa)eF zXm2`XPiPe|bGPAoTEOl@cburJWvr8v!cZbwPWbBu;FpeB^Gsr(VEc9CT35ur9JN@` z)yE47eVqOfIBCqAn|`*hix=&j?zGBgOLB*!Uw_JQuGFQSxaP$+Ncf(kmw-7-hy=?p z;c2X~Adtuv6$%q4*`%$JV5tdYwZyjmbCHN91PYVqA&Rx9NPldU$l=!Kst1zN%2EgB zrctt%rO8B}to{!7OL#(>;V+JMasIm#9?*)bqjd2Cm?{1ff5j^H=L^}u+AB50ilCxr z&X$z_UlKDKA~Lvcd9$`-ti(E-8r8 zuHH^_f~)1r#3S9L3p+m^x*b6;iOr;_W$B#KYTv4J9r!6Y=Ic4c|6X8-ZguIEyD?J! zW~mYAPA}^5U464S#@Xl>or?|JWS;SdRV$Dk; z{fCtn%W`pUkSI#C2t`$p#Qk(a;H++vyd(ZRxfQn0xMAyt4VY5WWPV#CKhW}&uAY%9 zPy?8og`lFXNZZi_w9>|^7n3z#HmrinDGMv))$VBPn%W4LR+V>&K>;>31=JDCZAc^| zXrq!vw`Tp0bnlF^n?I9C^Y>HxtKCc-SG;>C_FFYw(98nK6(74~BX7pZ z2hC)iXQJ}7*Y$IEcFRIrW)^kog94ya?A979+U@11j%VrsFn+CH+dMUtDmvI4Et+T_ zC{?fCMr+NA<l`o1!^B@Ol>6YaVu*WNYTM8} zf~8r9$OEh?<@78~#S@J6yeo?;P&ZuU%ZwmmFxT+UeQsn$U>Tx|mnXr-^Pdn@0MDi|J`oxT2803AIB17?~nhF zFLHa?wndgxbP0AE-pgg11JVyPkBzY567ugmmIz`Yh1P}nS#wayH!Qpm|M>u`(BqIR z+pt5WHKXP(0^k>au&{38A$xYMi;3joq0to2K$@wIsO`ySp*R=Waj1y&Ik#-+p>tK> zz70il8YKBN{&awkcvS zhX?tUk4?NwiI>Wp>BZM(#Oem$o~O5BZ(+}VOa;jY5o$fHW+?2>z5u2b!@loSa0_O% zCk@Wzqtj@C-eIpQWs2D4r~8e4mzNOyi0NH&ru-(oJlT8zk1%u^>TP12FB`E#-Qn9q z>4>EsB0g|%Jo|;oxxgRU;lMMDs45dy48W%d^aqDgguVd6VUN5}^*hJ)7XbO5ZbCGPHN!Y7)0!G+iID)hE zbsabF-mpVnCLcfEa=TZCY*JEvcmY6;ake#Lv#%3>lu(E9FBv+wpB{bWl2n%<+N6$h zr$$^G?-z0AeXpW4sL||(_OS=_Sl)EHWa1sMZCO!ALkSSYzt^}{s(Hu)aW&D$en%4< z+Pu^8bJjKtsy|tm69E~X9)lTUR1HOiE|I>M27YF22tcQ(QJ){T@W(6E_GUOgJlxBN z6;6rUaLkPfd-tnPG!iB%a$%OmOHegVE$!WK*>{_t*9p3eJM3I`3Zu025BC8xR;dfs zDN*{M%Rec6ke!U$Hp2`E8aOspC6N^bDMI|SJ*o<3x#eikKMNs!zpVHUqF)+j8+jAp zhE|9#DeFN7ESZn@e)dYe&is@{vsOdnOyqyPUucu?ytju|_;z$@!XU(s9-G!82owVt zN(mmy$nM!hTclQ+u1DOUb*pWanoLgkpr^VJ___`e8iIa=qs;pG(}*EkT%RYAa2Wg3 zO0E}$`Z8a8K8gBd6Wac$XPfnQq`+OXd^_`R$}H)7T#!M{;sCqsQDS`L=5}duwneP0 zz5BV#VI~{0&(6jHUSkK_*Ov7F%i1c70KeFB&z{63gv%;0j*NQKaEz+3**P~bQ?3y^ z>$q6|abN%o8}GT14-KEa@V6w0lxnAr_?Hm13v_;sKm_x$5~wM_LEi;o@%JIzc68SL zX&X(5DL1`I`A#vcN6M9;L?da;-`}%x5&cB_j=T3fr}$8p!})pO7ly8=ph|J(Zl%j# zHdktdEsY$(HH)His!wLkZ@9ioBU*OXlwH>+wa}%S34F-j6UQ8jUL~$XAxj$ftR= z6zp4u@32NHTG-kek>V9!f&^ST~%yaq&)n;rF!T z_d4ihloVU6G5~{SmjRq&5rVe|4V=udWa9yis#CBjBrJbO6o8Thu@+AiCj0TUEC>SB zntaRc*PY+34R_?OFKb^>zRg773gokmI^b&ICr0`azQFwbm64eXWl*_@7Ff1W zIW0;e8DHRECI!xm?kQ9EAem|fkJqr7p6Z&Sn-Fe36E6j*mAwG$lC|zEIy}}2+*BSu z`1)vU1%ase}T-O%DY@lpRFdD!-I%H|3|NA&N$vjgZmJS*cH zs<1R{w9Tfjl(XlY)LcTZWj@4k$LK8jybFOL33-xYqVFHqBIOqKn!zpvR_Lpz;u12t zPN@MR#VZH~+Ji3uM46a!-!#pofwyM8V>FgUBvv}@(Q_Yf7xKQmIglg2Di5z^)U7Pt z)o{d6{TYg4wOS_c;JkqpP_w&Ga?ot6+4@GHV=cFr=&NX2y=uJNdk)@9LN2`pt<22SBC_8yezk4{R@W7%EiU}RGPri{g}N5F>cp}-L_;W?rpgM%R<1IyRwdnx-KUi)>r^n_sOIipR*a_FOtGLMOD1w zLX#;C(cHR^PpG^GP1r7c>#S5UU+>7iEjJvEEV$#s5Dc*O*BHdl9UPkJ4GFtTG!T{3x?~1)clDibBu8=YJ=yPrF{^(hW$1$r~ z2VPQ|GbwKR>#)UU5wCV#Ehw7Z^{b7X+|eiWuf?L@ZfzcRg8@^!>-)0-)RO{OK?++f z(cJsNBIa3cAlrkVW)O~1V{PhX6#soANeMVm(2|1+&(L zKBPU?q{;7@Cw-$u!{mf6l`v;d;v3XeZ9)7J5ewQcaHS2PKh4gN>H9p$_^U)~4m*Tp z&xEyIh5A*8J6jBqs)Xx6_ks_91YPRA1a?aJBT8er~NL1?I8htRB| zXZX5|d5Y9^#nT6%@oi{Fc|BWC5RWAPn0;I1jL!rEN;Ju9-4hopXP5Qa@T^dz3GAsF zdDrzr-(#ok7BpXEPO(~zIF%LYF56@gYKK~9PjWe4o}(q+oFzR9uxnp)MfyAGN^Keu@QJoJ0XU%D3abfa?{}J63rz^n6KuFJatEV3q&Mn%=dC%9mw2l-5k3~XB6>tl2TZ{@B*-DO0Z`u zql^9P;!7?cVxIX0S0p1BOp(#_K@RVG_Iy@@zS)HU2a?tzBnoH zmC=V5D`&BG$J|p~!}@-1rjqh2TT~RdX+P+4nY;jorXQJJV2?tlO-bUl{G;H#J{}bc zXsYoTN(K*#YA?u%`~9829;}<_it%}M@~Ut=qjWD-Pm|a3CXuAi`gRZFbEt4A>uv~! zlKHf?EHzDCs|VySPIzbUfQivF?kQuz7E4Kcef;KMl-%v{bta?mWYPJom*N!7V#^1WDuQnD&eZw)aWiT3+S$I$wbwdeI3FvI~ISj^9>1> zq|{@xbbu%D0ucS?{1#9F;?q1@Zdfx5o5OQh=!_#<`x$oLOmf+8ZYclSxNe|>KpTdn ztkev7V9)0AZUQ~A60Dr57R6w(xFF48&T|rhGd%oA-oMq6H=k|vNYkzBCm2~}eMKrC zY%l!>D|2)>_3)2O`X&R?0dwko;Z=Wu_fc-gay*)wP{r4h%==lVPmbWPF)CQ98AY;R zJ!4IOZ#ELwkZB!8jByQ~TUJvbGSAH3WHwBOW6F}=frW%he>B7LGB-pz;&a_fHhokb z6dKWwl@@?iS`?W@$W_EM$M}Q_b+(kXFMxxpA94{4y%KVNjXIMP9Ns8VdLNIE?uQw! zF0iA_CgmV*1!y9sJPw;n%nvQozF&PF^)PHIcY8Qy55?!&U1|3E8kv+)+fG8EM}U-p zWcF^bNT|fj^WOM)IT_zzjs_W$C7#kQ2J244w&Pl*C%idwOn|vMZCbyPjSkohTWCnr zC0g_2mCp#lTkQA4V}FM*r)rx0PN=m%myoYDnTDjl)}kGl6)h5@y}}Dy2-HickncC) zNEJzyiaf8}0D8@J5)MGkZ@8biX~mEIswnEGYvQRj4t^}0P}e6_I^iRzOr(P6Wjsly8ZnKl)Oivf%WSYIcWqk$$CAMb#;5q14%g$+w zzfw-9an4I$rz5|VNn#L;e!bD!Iu_<%_lxm--iT;qi-Xkn?X!9TC~aF~ z`*QhxfYtk(oEwo|$&gZFxxq z8+s<912X5Mw0-393v+?QAcSAjsjwXw--VA$uLm(hvo%v8muRo-L}RB8cnGJ3_f1zWhNCa<1zMBm#uVT1iu}F#q7&lhHl8UxN)lyPJ z@!EJlo@uUdnJxB>-GDHaeSNtDLFyu%Av0VNRJv}%MMns%jx}OS@6=js1L>)R>?EY1ywl$(G_r2)~wXVNGlerHQ zC`7W6{Sebv6hu-qbQ6{X8W(RV|4kM3?rAulvUNaM*L~5+fT0ayn-IJ1%io+|n1Fdv zia9svq){y?;5;*X6k@Be6wcjFQl@ol!08?Q1Qw1($aVeIcC2)cq1|tWI=O`Dl8c;c z`=Hzfd*su{HSc&61%CI}`>|z7i^ig;{WQq{g?DBAxy?QSDh1KZ1mpQb&6?PYrx#Sj%J3iTByZNJ3H77_{4R5-gy6E>il09|c2jg`OMq@x7{JGZ7bo zZ5gpy>1VB>6c$a7*b_#J6v`O{ha@zXu1KOj8!hdu$~IpE6Y^M;UO#Q%dfb7GyL&^h zW#|^ipQ}V#oh@kPAFs3SE);1v9Q^6*(0=BSS`H++=FQ(z6b!*N3F+z|9i;Tsx2(81 zbsT`(-$j-$03@fwUyoKa(~r)@%RR#{0NO*l=+(PjoRNIUuxvo?r@wEWsoHLE-x?SL7mk6B2lrsn5&PqGC~F z<<@?LmeZ;1hqn;UFca$2`DEW(^D*qUpa~L6?>ny(ar~)ms5r--Ab#->C*7C1>~4n2 z?PiISCh-*S2|VJ{KjKdDKtTdlMdDSR@$-J8v*6VD9kK6YYdDH!_P_Ie2Q^^yXkKx> zP9y^Yt&polpm1l|I3Lf)c1x?DKQH%n2+@$gvMluTVWeMdA5&1g0Pqe?!q56k_uNkw z1nA0Qy(>hP>Gb)cea9F+fBG2U@mK=QE=-f~eh;74LCpLmRC*;0iADoGK`=;ACM-xgOuMckSrpKg-Zza(hWvRd2NE67Lh?O~^IxS!Ym41)J<{GA& z{yd@LoTblIrmE_{(Me|YK+sXzYfG(R9VQS<|^a zW^*ATCG1d&2pb&TH}HQKjcmsee8Yx*iB#yca4>#t&u*a;!z}5PFhuuNACqg4vcsIt zGGT7!`_jGo>T@|$)L~M;Rek?!+xO$tO3k*x<}G0IoR)}}e}_$QnzsO-752>2ALr1nQ)EkGSb z1najxsbzKzrHWqSzN-xv(2U2DX)b2dIkx%W1OqGa%dKv7mXp&713~s!j|=61uy&0| zHh17q-+6OJ&*6l3rSj*@6K-S~=6LiM0A`<{53w$}86@D1>y2d5g5wR#SIUyLE2cs| z)Q;4FSv!}Ays8C0;(Mm79$T>|)^Cd0g*;pUGpn3$TCUYQU6>em002uY2MQNkzwk%@ z3)p$FeOgU#nLP8hr5IkIE(3Ui19pCvBgR6Fw`ej-Z!&!2<3zoCnTLDdzp`^JN3uM7 zCaNv`Y1H>@EZE9KZ^|k-#bk7bbVXo`Y^DLcaK51So_j~R!T%Vtdat#Ibn=%@n3c~h ztk%kkel_O0JIveTfQkdkMxB6s&9j-;$EN=r-l1jEmmt>Fl=7cimiSg$e!gbv0g8=I>9%@0z z=}iCz1te_~HI;i&!li!SlqT>sG)Ik+Bcp^a_vj%7EU=dS)eUVQ%z8ME*&+XoxmhS( zoO0V=aP|UlP1T9iDSM17Jf(h_d91y)sly{#Y>`x`Ym>T7zYiwb4`ylo(4w3U!$Cqw zSf1S-Uk@{S6DC+4OJ}|5TsTM_>9y#av$4R+D0CP57_l0wQyG@}%nB@iWIt@5(X2In z0TklH==5jQ4xuesQ2A(=SG9P~GzlTgaQD>TM{6IP>FQ`vDX*Ior;R3t7eEkFU&!Sv zxX&=ExifX5a5C+JLg#}lj_}^FO}T)TF0JniK#Y~=mNCgc%nvMZ`~-U@O2}{AdKG+m zDv8~7!2QI`Up{5c=@`D32dU&pqXrUPEmAh8X<0fsevzUaVEuY}0nFB399gAD)blZQ zSTt5W)QI*y!QCC)9;AO{P5Y#hdn|3*rce@bD(}ru*la^#DNx#)!Bv0 zL7usc%6xVD#b}SkSt2$}6-NflBx6LPwqPRAV{Uw1Y{_zW7ptK583lTR@lp|R2#qpK zj5$G(@w+8OlAD1aQcVGxtR;U@)OV#CB{DxCM{rk4F4@~ZruNt$A}xL3>Lw%fvUu{u z)%xpAJP!+=#e4n?Gy2qh;*hrY0Dp_mVR}coHZe39HgQ1qrKFW+L#t`glJr-oyfNFJ z52YMdrb3r@A>-?YTp6MpTH)N801o=_UQ=%SHAO^AQ4hG!NF1o_$N%1G1L+TUxKYkd zvw59kKzGAPUSfwOxU)1(SaW`LNzKJ)m_cby>tF0p4B?4^CN?=(P6s|d4i-6yllbNqobD^Qaahm#X z+v}QaUI3|OA;^YTm8c_|DyQJ~QhRsw59fB*L$}N6E2oM)Aw5tov6kxGDoAz-trmot zbnzLbKVT-|deJmW9q9GOr`k;w)O^%sNK5^diW&Wk8sMNcf7^Sc%T{u3BG2~Cx@MLh zQp9?Qhxn_x`&3_qB>e?YdfQqcAl5$}R!swSx^xy%pIFW&zFq^Ct&5-Y55B**jJe8< z#&gL%HBhO2H73r!x87MVkZUk_n+w;3&|YRA`%cxUxW0gf>swuilr(G~4!!IL>WaZxz?I&A5i?jUazV#DNT6UKRDn znLjfcK<7DPhDRU;E~fOmC)%&rxK`dHi@rYY7dk@KfvH?i=S%>00k%=&->EiZ7CTxW*h^N4VM|m$ zh;lt#5#{Iui?)n%dmruDZ2T%s)w=I;IMXCn2%F4o`Wez7+q>AGa(Yqg`R%Npdt;Id=i7*>!u0YVB&24{sT-&1-9sT}!i z^=-S0m1jN}uVj}?`H>yXheoxo-4K3Gd*lC|iG5e#sM$o7nGvmEDD{Rx)Q;SLcZX6W zI3)$w$!UT1!tCO*lJj-1SYt}rB2RJRJcAEqzwU)Dv#H&3LXN+OCMr zp>RN$C1xCENfy_TuX`7^9Rwko7aJ|Cu(=(4=*6O;ucjyJ_k-Aaj!9~>c&H%AYfVND z$hETt&OQxk4A4>H%{0+A!PnRyNR~tA%M%fva?;A?kzGv(V`^=&eY{IaMe5w-3l)fu zt#oTU?*jW}bpWRr^qIv($Db37y@x0T@=VR2m^s;9;E{EIABx6Dga#zWiN^IA$`L!u zPSAM(g9zPE+B8gwBO=xY$@l|Do(uk(g}(+%WOGyWV<8T!Q<2X-)stF{+u|=e;%DP)8=0##hP4`%Rwaa22wz zhns?osuT4UgWOI+$=ytLorFm^^3)ic>Cg6qxxC?#pcwS2wndiQ!c;S0nnY8Xj*T7k zaQSrT&s1n%V52WSO(hVv{R_?Azj+-RPl=jsi_-XwWzHr8>l^4;cfP=9#O)g*XkTaM zT_k!!lhTJWyvxdmQW|AcZu7tDjd?ek&*0oomw7lcWQyfR@>E>J@1C?IU*Z`*Tm@u*IM(Plbd02>APK*#%{*igaiK;2At zgz?9Kg5c~r6(l(hi1&SSG#{nK2t%rf$mjKxnCCNTl?W5aX%*quXDlus?6lk3A6zeh z?u8(~7eMdtc`sLle%wrAW|U3R8U$o{A&SjHe@NKTMm^lZ+p&KWvnskud}+LT^W0;3 zS&?=oE*lg9j}L*=rLWwE6G0*<|XSuq#BzC z{%LM?%-9+NEL6yf)-R}~1b$Hn?4f4Rf%Q1NbW zeINK+=;8la&&Ie&5E3sBT0O7YRhGQgT*%4oy($U(oU^=<65^XqI(T=l!-w{1d6d88 zTlF8PrTMdtvTO_R)`VFiGH_(2^KiNmg59`<|C59q=>ALCvQ_Pwu|*AMVUW1;ZC8Py zlDKpL7xlU(YHD?H@7t)3MZGN&mL~C5}08&#t+4 zm=->IutvVnz3t|fM7g?*Vd^0HLigFsqH1%VpPuQucRcb&K2;;i)6oyd9|aO06vU#r zjmBd+7+YgWp7m3>K#%f5qZkfoI)pQS5f@IqOK9#Rj9i**$#{9(7JEz`S_?lWMq3Dw zYyhZ@^b{$-2`DX3wqBM=S!qpGH#}5wAlRR?u%IG6^tek1tGpIymgq9UcUPbs zzt!U}WEmnL;HvxR4KasyngG8lOL4WZV33eJ>j2yqsw&zd(+Ggo!}x%jdu>yki)#a> z^l(+QkEgA0wPZ7h?2MwoIjrX}>4;TA3RcC}DRERAf@sT!Adq6qM#eunJ;?JW;S(i- zsHTG>$6|m}NSW05O!MV{4~)e4RNgQzQQk)sQCgto=RNK_q;HPS?F#!-c!h8KJ9vAF zHosAXT~=0_5Y|m?#7LdTqoPoy&cC@H841Si=hTKS= zj0Kfv#msNk(@@9nLJEP=u66sMVGxbF+?q)C5(;7#IMh?Xdq+7s&VQ&d{hsLsU{YnT z{-(oT;22$?>=Lo|@@uQJCvr<0#B!JxLI2`7x46(Nb<9$Y9sRx8pMb5HtT3P7`1o`o z82iWC1uY-y1JY2!!*`b*)S7!ZKD>V@Mq6p!KM{w*_G-GQ&8ef*Ddp09fBhnVXt=XP zUJt_i_)LUZ#=cV=yo4U9P~0T2^=v>SkXdK2&&?XIlV*Qix-wmG+BowagAF!zmLs&D z4EMKvbd+hoHU0hqaJ>E9E3ElSW%Q{5?2l8np?6J9m+P?5G7B!RbR=7{q`N-YI94PX z%wh%&ZUS>L&B234Sq7lbm)YV6py!*E?dpT+ofXE(zb9;kFwbF}nzo1`8)|0kIfVK} zYG{$NFXAMo3yRJUdx8ZBzUKwwscymD{GQbHoS|2LBfH_w>E7bU(r2tQl5LNgna=H6sX~6?REmwgRDSAam@v zMON$Zk0r~!j%`!U1>7?nNY;g~0} zBKz%t_cQhKbt$NZTu(!-?4$+B?u)y}@stj?%d0r)_golrXb+8s(ECk|xJk@{iU3dR zLF51(kU7ZF<66-V8%=du0jE7HAZuBCWef3`+u{$JBrZ#U7>j}DVV3d~vRqt4vxOMT zX!huIGCnYBAtCP5S+VhwsYz~qLX*(h#Ch*Z5>YI<7z@zD5qS`c5kY(!<^ZZ6aAtSY zFijK&srj)dLKn6dapTUbYejVwPZ+|ya}-^CzjLQ>hz}+G7|3(l2d51Brs?I-U#Gua z=&v1h4QhMuOq1!(E1-??0whH^jnN5;U2a;;;+k3V7p^Q;|Gi*~lRB)td6 z)C{{pX0ybHt^kP+tl?urT}^rM7)z-($5(>rce&dK$*6>k^aegULmCu;cCq{tunW$zc?)ipwd^GB2?QNgwzu zS;}KM)qYjrk2?^%nJi)`mr&VEUn5oWj--8z>zXL`URv|nXnj)ojRMUJVZQhl|t(w|CvY+7|WcqGyDL$y*q3`@j-Ps4;pPN}lBL#Z-pkXzRv5WU>8;lkyQ# z#I>44xbKFZP(Xrz#>iH1eY_ssSgSj76#3I7Nj_*wUJeijYY$wDMaW|n+=}$Q`G}YW zsWYewV(_33fk&(6v{|M-h-+f}rlq;Zbu61r73!j%lgz*?HC{Hpm}0MEZ-W&1kX;;> zmqv9);>SU8eo&?Jzb25Ws)Vlg3P5EzQ{Eo+zp^Fo7{1%wH#O6jCky$h{>bGfDA;d| zRNT0ocB}V0mD_OTHb55aiS0oc1FR=~(3`rqUQ&U_-m09AmNUNq5=*~(U$)nmJ~&P^ zbi&}#H6=9-j@d3EeyW0VCIaG+S42zt)bt;kVrX9gY^yH-nJUL#-V9NtsG*;=a3fKN zepS`a%7O{@ttdBjA&HDi%hMu)BaapMvM(&oJs?8(GJA4IF-yj_+3-u~sEO2X$uyB4o{*=!W-9 zF;s@K*bPvmq|`#>4(xpu+4FYzo!~>q(9bb7b7)Zzi!l|@_Xl!%7851h zT7Rh1jXz(w{pW1){Ftl-X~L_RmeIyu)R?{OvGp!VtdskhXtCg{_crI7oi9G|s7L`Z z+iX7^^6slAYP101)$O@o5 z;RO7Yp9dTc7L%=Or>(8tDe}I%YP!Ri6sf?)TLf6JUEH1}g zgZsYUdC{c*ZM!n#Qt=AW&>pxpgS~n@la{4yCEd`yV)pCX!c1HD)S>-B#7s$ndq-Ah zRjjw$>Lrg>yD^KiA@y6)0m&}rn7FW*pEII@9v)@DgM&GRN%4;%WM{^zY1Y^mOP>JI z>bLj@KU=#dSSplqN>0KRT@hc?q*uUnB&>GV*Xd`q?V1V4N=V4Xp&~p>z z&d+Ll`i65|J)ZQYOEo8P7}Qc1Jy;~7dK?;UbjX_)sV2C*6j1O#1y|a0h^Q(Q-wqwXvQe{mAo<_LW(6sdHAU3R&*R96o@$NTGo#K*b3bR?5D! zT>qo|uj462q<6D^*OX{`EEUz1Bk4rJwlu`lY1ERe`70A2oGp<&zk)>&v_pR~{q721 z!o@rh`n!D*V*`Q5%#Zq&IR6@O`!WBX?nm|lj=2>R3%%7Nka}GwP|#d#z8+d3hg<2H z5dC~G`cCU!qh-}{?*$OU{*>*Lw}(0Co%F*{hUv*Wjd%FIrJ7v0zafV{e4d&>p3mX; zeye}5&W}w_(x%$YV_WUTHd0Oo?aTQ$C&_qDH2v<~HsF5%CDn>?F^Us zCL^g?^20kvd0^V6*pE^${($tbn3;nG`M@=XfF=dCNs`@#w!??_>{lQF6AnB;5vVtN zd_~?K$2@6B&mHyKpJ$^{TJQx>hxRE@c>@nkm{7ncv9D|iL=jsiU(Yy0HCGipZ_|`{ zK)BaS)idjSrn9m+VCWOaJ|@w8d8pi)?&%ozCV@i?bx#jGU!|tf+|pj(61bNB8{0n| zMS9Mf(u>{V+~o#MU;1QmqKYeIpt0g8zoTxweoMMv;crC0NB%OIf#;(+vV-Jc@Rh~b z&PLGNiac`B?&L5S(hHzc%^5HuP7?jY56%^n(O@@ zH?*P3vY{@K>Od%>#lC zrrs*jUx!rz=bj4X(h+5RIN zie)Y8vsvgce_$B$rP5?$`IRC?NC#yYVoi1xd*wQD34YO5?d1#LN)xHhk&^}4BfI(= z*(lF=@d^-UZ4EStMl?A#v|uEKc3+5L)cD8yZ|OkYd)Zic5R^1hHPq|ECx|92*mkSh z33K;~bzJyTg{98SrPoaoz8~jTl^vC2YvRf4$ZbhVF1XDWTus?O+6&*qL8~7fGDp3I z3HW_FbzFkvo6F_%7C0{C91hf&FA7#v2Z zhvd3oggiLQ*d`I38T8wisej>50s4r6UCmnHiq}$+{g>m0Ub0y4JUODK)ESU0${Le9 z?O?ZVIb!cKgX48L;kP=Exf}_l$=ND(1}D^C5lB?otftBn7X?8zJCZUqR1wY}ugTu4 z2zaW9j|piIwjc0u86wEKA}fJw}tfr z_fXLA3Hj9PF)`$)#^r~R=f4{{k>5gQr&BnaImM=|+97 z&w;}m4_^(y_^}+SZm2453q~qWWf~#d*#%D$E8xn5cIQVO8&|y4W({VvT7;HSr?l1! zwAOtSrm3RLr*G>DBCn|;iuXD2`5bny=RBu{LYhy;c3np1qCl%yp(+y z(WrCM@AI8ahAzVav)sGZ)0q2QB8tA=Pu^fq0`V1L`yNs2vDQ(BY( z&pc8hIR&25H8rw*okJUEOaoWe<8Pv_D6})hqL$<+s!CGOtzPDWop$rp{t+BI__qK7# zF7EjK*$U|hVl8W&Vw$S#=`eq21b}onoC;o%&LOrA!U!{KNc98~$syt%omLdPQG zk8X)w>hbbgaAsW`8@7$*22l33r_pxp6Up5V*fo_PuS-2K=TTybH?uWepFue2&_@|7 zDLVw96?^)gEG#o|{6%S$Tc4tnBOrN1zZ`sK3C7)=rIL9<1z<=vczE`u0mvX)$q|Z( zamw<-&m}3OpO!aTZf(1uoT8mp>De`M{?7Z5UyX57%|3O?&f)mskUT=TPREHekawL} zv@N@Y2CWm-fP0SY&S^mW5updg|8mlb*Kq9Xs^>1SBSLsSBhvPdNXb3 zJ##NT&tv;@KJ3pnL)%1L>l6aBz@K>lH)WZGt9DIo?d=&d=`q>M^i*onPhXBBUo>VHl7ZRUO<|=JKMs zt}EbH;fPh5E99XijCZ;VPZd1?iQVv)d-8_=?%zL$x7jy-XlhD535iGETxfO0UNT%F zex499**30Ot`V2I)1Jc}n0X)eDZmUr>9T@d?ktPFBPye374F~`RG;n>s&gSzkd8jW z;7`H17{_VACW`9(sYiKTEu-4waJ!+L^>wI%76BzcHlQA4a_C&#pBKn5!PlPlboF@S zUk*-3BR+e5n4-(NZz5q6@B2wN=Xmi?Tum4#Jg~86aKmw_BfuUv4!(uq7xx?H9|nv1 zrDA^+uwDS0f9za({Ls&{PE*^2hYun!eow>TxMiyXve|~Fg1n(Ekm1@zBp}$v=)|t=SRKiRXo?hN3n8+8mZyi`uwClfo1)PF8)1J z8F*k&`NUSZinqH?-HZ>ti`Zt-?bw)s$;!@Kp5tbP=WJnJpHGea&MqI8NJYOU7FQ|Y zZxZcmNpj@JnTC27PXJi+GjGH#0)G7A8`aW)R!eyO%1w@#3()g%i~sGC=sfUVe7=gQ zUpKB`WfA@PDDg-_W~II0RdVW>z-n0ReXKgK$okFy3&Ax$%83`l-a69n+-|m8m5yxv z({DKPWe3}1n&xKHPlx0I5BidhO{ z0%w@+a%T+a7m3w>gzo3j^NO~+ur&Zr6e$fe5Uqx`;IuymB`p#wu^Za zXKE`GyBX>$qVXl>@<;&3pcJLB8h#jL z3MHSPF`HTG+U*exM;9NQn_5u3uol3X>^imF~=Ki+;)MApgRp_XlAty9~f5W z*whv&x;SHAn*ig~3e32+33~SOH}2t!2j3N;HSp7|{J;U)qX2s1rGngxTh(K^STt1S zreaP%=bH8Xc&A4cF<$R(6;u=L{zY@zYC7tZYYsEH0wVtabw_W;w!Fo;o<=_?Bo1i- zAB*f0?L^viOaKSJOjTV9&{`$Lvu@{j1|`2G08p3tS7EpJ?F*cR%;a@G6$It+u{zHOritbt#_Z zw{UYaZXb{!W7@l)0p81|cyGko++0f)m-cK7IKyACMkCxpjnAR4G}PQj7R{iY*c*MS zSR`N3(A?@N0-P2iy<=L1?X4IgsgC zpVE>VN6uPasPPnmvff?C80nL6E9~Eex^!*-00nlixF}eki#jpL=rM6){uS|Mn#bl^ z!2E8wT%Mc@fyI6C@Rj3-{tB~Xl?dr&XDqnBpQ}|E+*8Gpsp^Dz>()CPH zuvfP7x)u_Sqv^1Ib=>QB*CH%i$Ow$GfI1J|gZ%4@@YI%vR<&n9S#559WA#i{+|b=L z36%%VnbRY(s)1b;{s4QNhxXH7-r9YojDi+f1_u~k4r{pn*R@Fu=j0PO1Y;XVrE+4% z&eK#&Xa^Bxv)J*|^sPI%Q&Y8;LUs~C{U~c-F#K1p*`)sfX0qHTk1~Qs7#)8~?lq{b zb>D=#ez^!yHjjMPZg%ATmsdbz$DkK656--ATh$?lRDq|95eN(?)w9KWFT!i>JH&QZ zUk=?smsPNm$fIx!@c#M(_(+2vS{{I8={mpKqhB-kX9(L#>Im&ny}Vahbimwmp4hBQ zVH7i$op$`ZXZThAU_am|BDx)}EaNP|@zRB;Jq;8L+8paMa#gwfE5$r@X(oj%9wf9@ zF}zFWB7c1MuV~XqyV9f>&cl!M_*NdVr^R-L)UM@p10J8@NuWH{7CXH-U#orP`M$qO z+toDtxpau{RF9Bw2lq#M$%@eFer1jG0hure^!^?ze@E2TLH(U53mf%+QR(a{C4GS7 ze$Ll+;_rS+{J8Y@s}|P?_J+bTcW>xL32e~H#dr*QRY>D?o>kkp2LsU4upQo^6uLF6 zmiHJ8=$OZ)aXP+X(z9+(^CFRtLtVbJJ6~Cuu13(i6byPUE1hVg@(sj;d^$3%OGEXZchBKBKKA#i7+TDdhW+#UUf`6>8H~k4y`0BhCoC)P1ma+Lj2UR~d zIp&xaHIogjl1fHPMn~X9aWjj(UiQ|={IN0HBdF+r5B~sFX!w&+l6VZ0AKfaQ+j{K- zrCQKjK9>{cYnclidKKV)6*mIUl8UlMP84n!{5YsAo?;mX&N)3kl_bdW$jBH0^s62i z*(6s}k+i|RK=n8RnggG{m|R~AcSE<2Y$zPpS)dvG_#nGpq|AM=Hmyr>1V-P;DmP5& z9&yjjOtEW={X%cCMqX$A#1X0G-F{UA>gNKR1LzrHO$*`fooWxytzAie%Z`x31Tp^r zQ(jefD3bVF@$xy&J*&}vC~B6Fc-C(ZX;Lh@qLc5d#g3FvHr^gJ(Y;T@CNZ z+e!QAIO)KyTStJrnx*>;Npp6fp2rL^k8UYi!$%9^TXeCymf8ut$nE9xRnX*otg~Pd z{{Vc~)&BspZ;!72Dt_I+41OzWmPbdt@mItz4{Dbd2YD9K&3kOE5c3i@`F0PyigVa> zuazw{Q#Og`FAiTHLGiLC~ok z575`5>v34!T(6SHgaDf4?mR!JETdGiMUhYyZ1I)HPs6oMz}i?5;Z%WgU*!aP)|P?c z`)KXH-*&s1_+Up*rFpiI;_0AKW*7H(JD3h~I}fF6+UuHxHf$}MI;SbU=$2(LKOZ}kIhcDG-=WOEqL zOm?YA4GZ@M=(+7s#-3zyNWDAOOy*77jz>MZVy$`PVF1V@wN}7Vw}x{#c)sd9WaIL$ zQ_$FXb1Z-Y&5`(5Bc?_YH_GtEPCk{nrf=TGxW^+kAU9Z+8Kzq(-BpKF2cKS*jjP>S z#E`14<{X@M?OsdbYl;5=;V2rGo4D;ic|BLCKGof67V~QQgI?SmqJfUs&!D838WO^S z)$T4%QyWx!nuAamnpMme);ysBkI?@B^{cbFvA6pxTNMm8AFgRNr?zOKRZt{Y_UFFZ zRs!@E);HQpUu_`gEslp3n{5ns_H#5`d6MTIf}_)~jkTkH z#|!?z{{Zz@tZ8;99pQfyG<#E~Nxo zfU$0hyT`8uMG|%zJpRUOSiCnQZutb?&fr(vJ_l&x{{VudXik9be~WrA>w9D?^Y>4h z-06`;CJbL{k9K_xz^}JF9jQqF0Ps}p0@ZPW@ry%Ik3oNBQ}&ThWB<_nRrqfz4NVyE z-c-x~0DW!$01Eee%ZY77akFg(S8M^;s1@^v!%vr7g5CcBia8G-Ma6v?c^lpEpYp`x z9gjo$3c7m(=x~~R_RDNU+)NpR0rjgo&HRyDTtyO7!|BNb*X z6yHg?-|k6}Gt>^%vj?#jnQ?R@TV^~e?XHsge?Gz+RVp#^umhUSlG->X@|X}@gU4Vf z^KIT^96eN2wg~Hf9z=CHZoD$5FRCNM7yW!g``CXytfT8(cAb44znK_to?Li9-`#JR zeqAd^Lu=hC=w9igZPmP~a{>PVrjU^8G4%sJ)l0_~+Faf+PZDV1hMTM)zPDEBk>6#! zv+3nazC3mqqQFtpE-mHUUHKsfLMnyjuh}JQD}?gc z0NVl1NB5W2ew`~y{{T}u#9}hI@`i9dY4>X{_ARxZ)O3twamt_eeLj^|w!l`4Sx@Zg zEaLs(3G%n5E2h#l7$(yU(~aoy0mt(dz>9m;luc}?#C)!}!6WHgIzQPhB4k2DW4Pr< zx2UPnfu%L#G}~iukYHeX*BSOf9o6B_%fE2?3h9IC#$$8lfUS+g+;prfo7-`5BzDSJ zKU~s|fMPp?2=054Pto08V3LkHRf{`wG$w0-g~=Rx3d1KgkKxC?K-!G+{IE9Zx_Nl{g*~)6ZzC6a(R|1EW5@&BH3$x8 z!*&;%j7Hv7Xw?NUQ*F;T^t}tS@ffHb_Zb&C~sw zb^`a>L}pA3=O9-#c5d!9*r0Eeg>TBIv%i;cf<+9(^!nDT+rtj1E*xW>pFxUXQG?9$ zOTPeB#*m9yEWeIYsK4+p{*86HV}B2PvRgtPhT#Yq=tOhR3er2jNt{7+6{BI{Xr9IZHh) zUsAhn-bJVqH?RckPsbI<_@`32)MmQXF3B@rO47LO8bh3Y2SMy9J%IB`uA?(Oz$0oL z;Bo9Lyb_)mbHa-A*3)J-=3p|8+>ZGAdsm}dTHu9mL5dOr?VhT!>Lvx~G5SO894c+i>I~9iu-TC z(r^C&g0kp{tOn1FnhziL{b64l_-ju~?K06%F}%=>eQs91(C{Uvm;V3-U(oKOkAIo* zlS9e>0CxVwarDJX_}8IQUg`hQ{80E`3O|djcYWgozcDZ8m#k z6G?yz1ODwLi8~_?Add%=p4Bzpp)|0_selRjJ@HHojZSGk-q4&345NBhfw7)FL9Vd7 zB#biS+*W$TZYB*nfg$4+(nMrY9i}omWbuJgfWbE@V%F_1IBsw=@}3XnQ%R%AsCX|? z)8b~5Zx7x1tDe>xiTQEu9l-np3c6y7W-k$3MsPFGgTbr57}WILC&iH7>)19ou*;=t z3NwbbxWbm>p;p+#=}m$>x6T(Ze)9^%WRFA8kIt<}aOkC#OPpi!{{RY2Ye%}!ykT*n z-u>v@g4n<>9|Yskh8mj7kXbNEKhm7MfPKZJT7-xtiC7jGApI$Ii=(G$=1HSpFFRL` zf|5w&j7IP{kIP(R9cxcmb914zGa!(R3P}OxdJd^;JX1v?gmTAkZYyTo$Pn9@%VHux z=t~ZtN};6b3ioXyF4Gjf{SOAZ->_Th(Tk;<4&4#Enh+jd+LJxNX`=Fo;ELNZT`}JX z-V_elHP}gIrrpAK?cPp!6+%do-c$Bdk+-)_DS^gbSjnmUj}ea@_T%{)<|ffEboqbM zrBz4!rLs7$U5{&^GLgF+)-Cil+MSfKec*=}^!KG?7C2wB+mH~xeb<$%Q0e+>9h1o` zFZWdKs%vIoDvy!1jt_HzU5=YSlNgas<&j75`c!~f!=hV4zhY#M*8`u%p?IyDR*>cP z`*BrgvAvd6OK6!8oSrJp!pUMhwFf60ngE>J+ax}0Y1`8ntVg|gtr}M3EOUp|uuv`zTkUoN-4_MPR z2)sM+y7k6Z=fv_Ym|Xt=%aw`WFa7r^R6fHUtIKY+3yYhNv+gAn41FuP_@8%SW#R|$ zzMA4oKMx?&V|D1w2m0s#0D($aA6!?PczWvHbzlyDVYl=D0P9g;CcU^e@y1}rBp)t0 ztsNrf@@ThA#Ejq(=qhyf%EeswAI;DJ^5z2}K$wgU|fyBGAru78ywq8|go z-`P4lw?lOp{H_+h()e4ch5rD8ujw{{Gv)kW(gFVffnQnw0F8V*;VZdLgQP-(t^K!_ zeT~9#UuS$WUoZX&)upoyyZ$if-1a!Wvi|_}RLA(ABS-L`|I++J_-SzG#TNG9fDBTC ze*x0H<6G1PjI*?yD3p*vrXw9Y3f_{v zHWGTF<5Wo%WoLgbMF(*sA-!>oV0u;+kBZu3B3;cSa-m_uj1W2zj+Kzwl$JLU!I8BH zP`$BDe=bY8CU1~#J-DQmfjte_JV5s-l18lOBoI#r--@>{iM4A-FKcZQqK=GPipaG2 z7k$6VgWH2v86>`T1Bk%qK9yQnHRxc!inaNh&9_KX1Z2jnPCpY-*=pMUp?DWREZs+G zIO&gEbgqL*(4&s~AZ?AumiXyZ64Ysz(_6p}z*W=^0Nc|QEh|_R&szA0YbL$nT`R}> zW!t>)GU?ablz+#>NxBDb`t1)H{_ zO|!pcXX&wI}Nh9neJQd zgAtB6Jl007;%l~jx@vEEt$lmLJp-|YWcV&?|aCZ~hx*4pb zTcWdZmi;PA?JZ68fj;b;RY;%==pOrMiVQqX4gtvR*j4zZ)h;c{+(#T+c;l$WbULP~ zZ(xwwgaM*ZJ@~6P(KNBKi2yFUkF`>04;6SdKko1>_^P0TtnziyMtfd+q-KBub${{^;}{)7KU6elybl z0O1$Wbf1S3m3xgfKGmub!2Z#?P)dK>yjvsBsWl*ZW8Ye`tPL5LZ!MG55Ai){d2NJ| zO=k+iQAPf^0OFnW0oE7O37w3hH0(B$S4`3{^Il`g z!hk6ww~Ry2IOjcuHf3oxTt)YHWO3TFXldBM)Gw|rgU9AEBLt2+8rJZ&!~Lf6{hM?B zj&t?-af;`)eYaN=%1+g8Gg{gn5vB;-G!BE`G?LIRN#L2hy(3CpybIe={{WKa!R4MqjW)}wnBrV0z~uBMv##ZW>P5GFWg7$OMZlg*c%ik5CENScvj>0) z993z>tlDhTTROBu0ZN_;2c=QIQ9Y0DWy`P2k5Vbx8bJCv)7$rLxSSe5ErGg&40M=_luBxMkKNAg_PN4{~)Yx+35&o`Z9X!Y%YmZgjF`QCkL{|3^ z>I|{Lj7yRT?d@1M6FW*R*J}A{K5tS#l~C1mU4Oy;Cb`fx5(KbIu(BVK&N3Z8*pWzO z_BicTtS-rEWt1G1$3E1O)I5#5NnJrZ$}6_=Uk{JH* z_4M|rOmf`Zm2tFZKHVy`6|7y-+G~4zX*&78A??8b0P9wa_qMDUgxj8j>BVJQK=58i zZ##9)2Wq`#rli5`vOK^rD>t~&1G_j$*N@@)Qr}!kb~>M%a9_%nOLw+qi(uNlz3ZF2 zmEe#+@RrU!DI_zl)b3!@uDtd@j#mP+CAho4(^B$JzTk`w_^vkbq;K9#<$>+}+S=3a z?x&@5n!NXu zEVsAv8D^E9Y2`fbX%1ZeC#8DFkE7GHJto89y`Io?`OV&=Dr5J$lP1RGbr47Ur2hbc zSC`4G#dBpfj?wb*{uG!P{vR=2>AraYO}L!@0J3^ky^XrDFE`EGnukxf7H27dhCGq( zDbPx1hwU8!G}gmp(_9GcS~$r03CfXK&`WgIix^+JN$ZX~RY|8aO(wz8Mg1#Q$wEN0 zFH9WJ_5_YD{ws)8;%=;em2`SS0`}5IxeTNlrKlB}0!g7v`t zDsS$GE~I`Fw=dzH56C#p?XG|DAuH|=g(DyERZSar8@@2;#2(yV+E>JSB(0=)0t<_{ znm5v}p_Wir2N6mnZ^vlB9<}xl?8V~^58{XX6eGYt68t{0HlN}ziJBg#Xk}fJ2qx4m zwg+F~VUC@vKeoLM8oUqx)ci5fppH)&Xp>xxphYua^vF;}dk(CF?5hf#Aeoqdx^Z4D zr(T(SPo+gF@?oE+Y@o$^uC5HaGg!vEu*^C9D@uO=<+>|RC7MqhE=Kr<1>7}M^YKF`BS3PcgX}S zcFE(iY32lAmC(N-RM*je0flAh3{EE6ew4g;MH-XMa zYUeKW@{b+GiwNX$RZqT6Yj|AAcc`@TjBoiz;ZxX)XP?PnTgqnJ+XpqGWE`P_>{`ot za(BoM7=K!#mSwc{)oj=~5>Ia@*_59uPnE}Qv@*{} z(X{O|$DR_COZkSYp+;UM##-l39F$-7?T^f(yhj72c|0~xWqkHicRfc!52bWEMx$}y z?P}it08g9Ao*7nXW&ranW00I3TeO&NVE_`gpEwxK=bmdpk)GwJPGA$0vh(XZwqV=u5bPJOC^ zUAjjcv9LcnqaUHg76Wef+TJZn0K*_N6YYR1yztAX-1*jPoSdIvM-^Y~fcS=WxIZE( z{#AeND|tI^2ao{AJP)N!fU7e{e-nV%kaLf5RP|eHWU@2|j*tGKxPv8LrdmC;^3 zytcn$VM4JieRg#fR;W4@sCKp-P@TZ!kf9V>DfEbTPMMR0tq9_X@m{d@QCs+bPWkTdAIe4R$FH_4wD^5*Z}9I+J~6yOHPmw( zT_QI7k9&{aT#?>8RX-@`WhywuH{sufV(}KJ`u4QzVGow`&2a0Vve_rzbNJop#-C+*I^3$C-k+yReY?C`mC$Exk*9i@YzX3~zAGc3KQ(ne1%+pO*{(iq zu>{9Yynp)D=eHRbTUV`iz71AMt|u8A{IU6(nx6?vb9caYvEFz(&dT(8(cIqKq@H2j zygbOHAE-D!)%UmT<)GN;{{Zk(pA7g{!q7!+r)eJ(^ti3yi~x|q9sRZ=QP?hc74RR! zshKqG3ey}*s9F5pgie+B-@#J0U+`322WiKd@n1{n#&>EK*{om0lI&=b<+=aW{0*^? zj~iHAM0i_g9P|V&eLD@SNu)<}bm~0wyNS4H#Q4B zlRSs!Wg@)#biC7JTUc1)avWp1=qRx6X6h2neEw9K3>X#19P}TbO6fd5txe&(!*VU&-Py`^iQp&ZjN_pP=8gXVgpNCkp{nTj+H}Gx-zH3+n8EHU+;+2SP@tKD+>e!8 zqOuR+Z*N1|qQTJVd}ZLhC&Tu(w|*9p^&JlQw3hO}4x|j2qckVZU^3h$j_Bnw$fVm;JTazE1$x&JXvuVyMce_k;uoT zZi{a#PcmAO=40lsEz~Y9?ysVG4-4|+)KtV}=-eGa(Z9QaSr&KX+&-Ie(RY58&??0% z$i;a<*mSIEw2OwYYqknxf4JSi^rv7Or=raln-~kp0CwZIwNGiLyUo5O^4K1Srfa0I zf;F{hkp>?P)6%vgwpo-;!)YFrm>AmCthdpE=jTNOk5OFjg#0C~_{&_59W4#QtKlR2 zs_wkI6aKNt9_#9KFmyc=}4I(4a6xVlld z?e}A&r}u5+=>qrV&p59lnhT4G!MNww(0hN4BooT_H#oy@+lCz!W4HO@ty>oRKTnkj z3VUbTpFl#~H{{TLfac&7ax`yKwj|J2&@rlk`cWS#j>^ou^P~ytT)R0SV;!rO*6%K&mMgW!)foU%1ERG|p(7t7 z993J$R7Z0z-gXmNx=-3BLn9tK;{2I*V0B#Z{e9y#^vpMrS!pS0CN{?)Iq!|5iq@CFRE+DTjbcG=Kx7#FY1S7J#VyU% z$KG)o0qy)mRc%V<+U8c8&4b6El=SITWR;_mJN6kvwBxs4l-Vamd3R%Nt!4h=K;JjF z82VQ__R!o$&vo-5Ps*T%{cm*{t|g3lJ4(g=_Iet2t>xQZdGG+qB8;gSP#cEmu!zpcBvMK2ohpT$?yIP30bTsWSi?9U zcLZXimrB2OlJXdU+55gbk%|#EJ&#{W^&7i(f&fvMJDBv(r{P?*Iv%aCvNmI0MtXt% z6*i%-YHuaImD>4JO29@2K*Fygx?4M$p2!)>?g~d-=QQq#I7zQ8=ehZAL^oe`X9Ft0 zbXh&U>fV{+uM&7$T#ok0{hmo8@@9tt;kbnLS3~m1hx?>)+XkVt@Wz=8%9k9!m=ER2 z{3}ZG$|+-t)^a6?k%T-PcG`PT2J~JZ@b8K5*6+ZVG3k1tY_^kkAV}mVwXDyDKG4cT z0nxEs=BsOWV|@Bft#0;LYm#A=N;lvSU#Q};rPFRyOqS4#xPc2jyrgd{+i5(0b<=5I z5$`lRGo)(~eXh-a<p@Hals!#TLZw&Ym zyK}VmuCn7+_(|d?j(Gn7wR}A?fd!twG?uCJS|2uNY`<+54+3 z%zq&sYT(j-9K#>_UZ<<;k_Q06q*<+;e18kv-XHaTts3VYDnP}a{yPg6x}C&p_d)mc z72J3$!rI=orT+kK_|;_a{)z|g?k*D-SB?GKTVwmiz#Pizv8Xv4yDg`Rbnge-nS3#% z+i0#ZR#_!&O85`@?I_#y*n^7aB=Jk@w+(-O^3OEieo2WeanmI8k3rB;a61nZc*O>T z{{RTjgzbivqlOIesmt7v(XJ%_0G62k6a!-Vbgv%NUTgPHG0wyCvG2G2pGxZ^RMtGm zava0+40?MC<+V#uAYF*cdSr?$23jh;jPM0j+l`-0Qcr5umT3$)W_I7V(Bhq7jcaxy zT)Q{Rj>5VLbj#L~G=~lW>^fAiIb9=0xQ^{4h-HX5_RVxgDQzwS+#qIR0ItHrNb+Uh zpW&{5T;FAMlXb@;qz*I1dZovsEdUIE_ zyK8d}V_rbUJ5YdX+s_rOO%Tj-W0Cx-_P1#E+BWG2-@sGsE4I?CQ|z&A!7-eQ<^Cb$ z*vBl&6KfCvG}^GyI6uc1G%W|K=@XUhzpcHHEAzbLN<@KvqMz9+Sf;$VSUx*nccuX(i8w3V`{ymmP!wtkh!&7*0S`slZ{+vKc)Rg)^DVz-(JB3<^F!QkNLnvRzxzJspx zq`J6Co^}VH)q<&nxQ;Y$hO5oPsc`n_G0kMcYVB@uRo*c5%uPm!yA{hi*Yb#HHXbv&{G=4?=Xa4JJBrP_IJcPnwU z43EmVKMvSyT8z+(xEe)!PdEKuK5f_?v(Q(#=za^*F8nyUmHoTLcm9$B43G6n3gB^% z@TQ{J+>$sOj|X^m8-KK3+JzlK*jMt#D-`%c!Fq~pZ!rKl{{ULc{cEd$&vL64puS{# zP6KWr5$Fv;{hJ--!b7Lp!IvO@@NEM={?u{-hi&2i00(KYFt%9tdXu>RwXU&P$#0u6 zyKi7X9D52pMg5(p-$7|yPnPt7)cb z>hW4c@{%!_fgl1mew4XbT!z}hT@q#0fXq^U?Bk{zrD`spzjT9ZbCHgf)9Eni+MV64 zl+mkM&gqf?0HGtNrYavC{{UuadR3*}iUu>>T%!Tb5;npNeLK_kZ+1^Hz)fvyWiT?y z5=u^RSPIgDXj%N&3IIEX2v5JSoj``n}8QY$lAyk^cZJ%W^8C&lHnwkrZU-?&GCYSmC)QSQPFdbH_L| zC^1~GWpRI{d4gzI!!J#^J4)9}rLUi68St(-JmZ60&bqM0DElk0432O;xY|c!Tk>f- zLfALhB`V7lB^a?K*@w@^wO1(5LA9C{vJV>$1~3h9z9utXAlvh(2aMGVOKDOA6E{5n z04_08XwbnG#E|P;#`sls<7*!E8|ViLblJEpfClOF59wVtmkL_g-FcfvA|Xb3C(^Y1 zIi^`$_@>=2!~)giVE$|Ej4lo`N2Oo!U5=M+;tQB`gA2hNU;_hg1%magr7wGsxFc8K z`SaVtcG&*_m)jrx9F_Gi?7JbK{tA2HC6ne0@jl1_>Hh!|G5!_v55edzqVW%ett5?? zZlkBE813iBUr&C>43~fKQ=bYhI($*L{{Y8zC;tEnNou_clRy8^_-o*I!taJ-;q*G_ z@a~u|UhGEKA7qQoDddy1w%p^|zO9$x$HUm{?TmU3oTZ{%;?^Ob{C2!E;D7v*n{WF5 zU;Pqm>T8eq^0fZ|t|?rVc7G$HJDikW0`M=AQ~MUf80q$C@&5pz)*PM$@H)E{(X2)| z{{WA8{{Z-!^oW1epQT{U_nxti=8t2Co521GGVeM)mhL(GJbs_<)oDBj;9{swjbp+O z`F4;00Ew=~U%&Z(Duw>yf4D12Gc(U7@JE8i;53UYWBz^P{{Z4^nZ58&gFLbS01}HX zzw_@O{{RzSn>SYaS2KJ5tW-Aa2NIqR@Q!i)m1oyK>J$7cOHS~og`>J2UYlyaNKk)D z(T}ZOu>Sy$nsIK#kn27b@Vv3c^69p$a(OVJ{*{Me;q3z1o;zzBc-66xu_y=f72RsT z>LdD^=PmyL);G{rmn7qLEeAvM3HuZu_w9ees@Z88Nic(J4{zWf;avu$_svkT`_TRr z(BDJQB+@jP?Bi(dVNx5iDE^h_de)bwOQ`%Z%h@vED55;;3R3wU8z1=FoZAM?dO z!n&(33iw^v@6&Bh2dpXn6;9LdkEL~5f8IYz<&$afe?QWi*?$(qabbTvU-|i2m^rdzP=U2d<2WSJfjbctf&)MVu0OD&~OYk>> zy8i%U*#7|FJb(O6YNz~q{{X;+YH4&`l+3w27vS$GxAsk*Lkxc5Kf;}R;C}^6WWrq@ z#lE)jpZ%7)H~#=qf8G@}&-{6{V|E=5B0q<|0T$_v4^JHb0ME34{7q;0qrg4|v(R2k zO%}o?44@RYjZQP{wcn@wd#Bd3JY)X=BJ`i{FY~1bt1GePQg~Cs)?X0(9X+0#YYVmh zoJ%Aqr2hb|QV=e11p_Y-m>Pul2L^%i!1w-z%fOHSrU4{?Isfq91nBC( zTz_CP0Pe)O`yV-BwxDZ&%fW^-{{!P$0637ilark#$X3n9(ZQ0Q*Yye|2UI}&UsqXq zWi@&(HUTys5QvL|lShbyUx-7H{tDn0;^YN^?!Qd?$1wKV`ccGzhOY< z8xV*9kbaGWi;MFO?mrge2)w!WFRT;`{)+~TgA0TQbQKxEA{>)AoPXup#NqyncZwtU zTRsr4xEp`N08<>{zw};lWPk8LBye8D`CZA!SpaHyTu0FVD4JV16; zu2_K<&{fak{S|pWAue8ePClWlH_zX_@&CqwKpIa#pgVg2>jVfy6a@ka5rII|K7b5s zpi$UZCpK4*8(`H{U)caiH^9?>Xuv>J z|AAivWB4C93ZVW6zLNM4eD_M23fKx92u=l8gS)`};34n;_#GGxE&#`agTXFfGq5(; z24MG4fYmf=Z0;t*`HIO_=78C-K z1G9q7K;|Gb;AH`_1t2S+>VPyrDj+4$OOPU%6Ko0sMmm7E1}JPm#vntG5eN=a2PuLA zKx$wfuq7bb<_`rDAvq(0*m39?+%&QU>{fWWeHJACNZy`2iG;K)VIV6lm83sRH(Sfu4b%fdc_M z13|tZcR-3gpv4ko0NAezyqrM-0Dl-L9uy4lU$r6tKa4>7f7;ameLP@Ea1N*t6bVRh z{$rmNpuqs};7YzV=m}td1}F&>4)O=O)dj#<0usys{puhEz!zp9ekmXn(CQ1==lUlK z27lx$1MNm2I`Cbi<=;D=yZum;%7$QA1skRO&b8+0bM7dpn1f~E+BF>mVlx9*H{Jp8DoE;{`wRHvI6+e zEXGl}!u*vnaDhp*3XJBfoQtDWpDKMI^@-IMxPYqb` zhgb88m*ig>E)dNhJTOb~g|3=_!UEJQSwP0Tx{`eWu2;GCO6?V%`LFBsRNxBWoj(KD zbf7~1!ULNAq9gw6+6>Tv(f`Q+02XxliICdH&CN-Oo!!Be&D6rt%#zLA(VpGQ)QO#w zje{K|Eav58YL2vYqc^j(wsjC;_}SLQKyPay!l2Eo%%SWgX=!6C=i_3j;iIBy?t?TJ zv|tbur5E-R^0Ifbw{$b5_p-Nha24_rVfZ6l2!OB9>>OO|TwJUG1*@yKgPW-r ztAi`!KN6&Xao}R>SiegR1t=&fU|P&v2yWh{*!XqfFI&I2~Icz;=VwVAyt|J)PYAiN1|w86NzIM;CT0DFNJ|Eq@p*GTY4A8?A}-&Hpy zU~nem3X09W&L~m)4)T2L7ZbOc%gY;tQ&Q9N@(T)!ic3n%>gpTN zjZMuht?xf{V|se~`UkM%6O&WZGqZD_R@XkSZ+zL@`nt2bw|{VWbbRs~cslTxJHY4v z#~%{FADnB~aIXY4H|Hz|%9s1Xw%Q?_3T;L{(1eXLP2D%6nuAi&K(2+HNht3s4 zj(HdY3JOhQdz>-y>(q6bQ0Z^_r>xf#pHr?rLN&xDlxkG!-iXzOTC^O}YxVM4QyFOP zU}E9dbV}TYQ1T?hx^mG7Y!4?KE|!c!i$CCtl^hf)^D<|wG>{AP$y8%ksFFf6&O!X@ zQCiW};isB|gq2U1#apUEZ?hHX+<_WWFZN5Nni}Eb|DO$qK1mBLh4|8h@P2aB*RR&mHf2)Up36$DA>iA zQ=3?Dxo)Y|(I!^ym`ps$EH@I*XUfIN*BSXb!MUO-F5?0fd}2HD-0OHXha(csv0kU}tJltgQe23@a}DqbFA&dohoz`3J6>x|8}9JYp9mZp*dJtp=D7r$ZFRc#xH zU0%e7j6Nm^Ro;{VZ;uELUX`^z@CHZTW#xa!D?@e2N z8x-&|tX9b4+wKIT(4^B!y0~~vXgP*2g}QV8P@C09X`{WVZ|vtd;f&8eKuk@CRbtI^ zs&rZH?hl3DMeE2pd6&is(;~j+^g5#TPRs$6=UsYqqeIT z!!*?_juJ-qu(;?9ru6QX-ZFqUz2U^ju8-R!4Fxidt;5@-dljnJQIkvzVTrPd+J}S= zJ>Oyc#28m@2AvKl9do(Gdta0)3}sM}p0cD30Y{`w?T$FBQ_As*4*X7}(iybCNUy{5 zF~iTsVzsILiPY1X{7XBH(Y7bPVZ0;}*o4GP)P^N~3~r%ToG@zgG0?}mt{6*^u%V$K zlAr4B-uL0PF?4LuhmmHvgNj5-7^w6!%`d1q&Qi)EZ}rvO+`UvI<4SGawRN5}+hOPE0)<1wT@ zm*Lbp>nKKz&xa06H>lZI)g3`dBz(2DCVakO>t2*HQS}JPbWiE^|sZR>zE_bT5J6p z*2CS)t!5lqRiF`4o zSJunSNw3FEhWD4YCq-A*B=|2ST1XlAO&Sdm;&5tEPE5YDk51^a=JmiydoT8?SSP`) zoMBG92kIU?7p=am2!TA+T!KH|z_KAUhI2Vh`V#c~x|7iJ&(AEM zC<$N~#iL<5m>_J<>_>(BPr(_Zcc5LfF?x$6>j*K}T(lHEqrNB4dW@sEUY3!*y zYJPGKWps-Olsv|!5*`w)#PJ0!DD&1V;bNOJUj&2nzUQlE`VkQUj#_ud=GZv*z?^gT zAtYgQ^$GU1&ap2{G4v=jQ$(6|k_e2eWh|jTI8sxY91qc;yQMB1$gaQ?oJnP5XGUP21PssQLvnsV^CiS9_b6$k;c?Iy3m)?m&3=Z%%2#2y0%bobMOb?HaAAXM zL0WUf-jz`=oKdz>%2KQNFxg-w3nr_EOnd}a@O@34Ta~GT7>Td)Juv-&!xCD0p$P?C zAjM(eEWMDjx7HA?SXcGFUKWFq-CiFXBOk59ND>T#41pdQezCbgkr#%ihX65Z^OhG^ zM%8G-m-AFoxX1?;HCnm?Pr{TGh|d+h+rCw-k!FSU!WeVLE3*Wl2Gprjllr5fcWH&- z@vIW&Kcorq7KWiuCvDA!ZQIQSOER?~>Cen~9`WA%5REj+-O|!^epjTCI0O&oeJm;E@g=yILu0R?Ld!jxnk=D4+MAEwrs! z?yr$^W2*{usK}02M3hzViHU6DxGBp|CYy8FoH}fAOdKE%SeEE!%#1UA!#N;gIUY-1Fv6{PL8!RWa3enAP&(=rIdFrz+j~K6HUvFM9L00ZVC! z*B8i&hcPw;HiGc|*Li#7Iy(mt zIEk~#+bz{yuMrmdiT!Ue2QIxQs8N)@e4qz&%pL?D6||`#=U=7@n`<<$M;Rwt#$5a+ z`!YW1wU>W}xzT+~E8MeSx1+p(nh`>@(R&H* zZ{5F)#O#p;D-Z-d2}~(rky?)w*p@$ZHk8~Pz66N?UTNENNO>oUaZ2D0)$uf(fY51J@f zG-Zf6_3x?C=A|5djvA0A&g|_o^@iMgL|F-saMTNmzdymu8hOB##Re^xLK;-9p`vv> zs+85`MSC<~hfeRP_x>-A} z>M6~u)$R7pw=|!_9$nM)g)u1Hqb?tK)u3cY;fW{ds~ma!7rtN z1P3ap5UH&p229>r5$N~kNLQ`dc$B@qi&3a6m54MjJt1bQiy-}u2k*!h{Ff|iL3EFHSfn|`*Ypui zzE^nZI`uBM9%rOkd4mBwHKqk}*M95ysy(@R0%M^xlP;vF4?L`G*^3`xYbr?Tf|)Bt zcU3wJ(pWbV;O9ybhTu`{Vl5t;2Ff^1G;-GBUG@Je-!Q zn#XxG4L_e8ly6afQ{C-xl?N-EKXul&TOK6+a^ao5{`}e+L%EdfPwn$p7g`WL{tp;I zor|YpD@99p$In!EPSr@eX_EN98E0e@3CmY_$!a7kj&nrNQTA6)h#f!XrUd4fp?rOB z(&UNlr*h-tG6!Y{ao&R&J6^#af@H0dQx%QPkHlU_-P27`8%=>4Mqy=MZ9Q+ERTsr^ z9#k`6e{Aok_eIDa&0{+|uvI*&-<#)Fm!BqMI}(EmvU*40)#`1us<~%!$3;8b`pQj? zQ;j?yBfIXIr6WFu5G-{tswk7P)G-nkqkZ^Ni;rD)Z4M>!swjQv`lO(m7p~D`7yN#@ zav#p~ZRzxyO8oqfcOIPu{{)?VtJ#4E7fB2rN^4S95n9!cn?0T8g>dX^bUo;5EXB$+ zMxDw8jFXT!%)Jc#owjLlUKVH@K_xnphO+;iwkz*=_H$W3s=4CE=dL*aQ_2{<7PWlG z7u87pmY=m3Gjt8#pL?q#SWeOo4DTM6o)^w7C_gNEajxu_SY$)t`P};)2H!-I&ol*v z_ZC_Uv(fafu+L4R7JiR)A+qOETKHP?Zg)>obs@#(T9-HCH6>$Qjg&15ko|D&V+8XL-L5G-m8BK@MVbL4~KnX(+#}jY>pdlj=6RJtYSMmR(w% zUNw35$HRM5E!!#tO)-A?^i$chGoA%%S$t)p1+|x;moF}`^0A&J>>G%o>|biCb6`RJ zgR@~jDGPan7ihh&tM}CRiV#!E;)&)V^KhIK@QsgWND9t`*QSBboIqKM4J1?s@zXaiN?_>;L#CzRd zGfuKnWMP=ufjhl=|FG|c^S1hVgq&!H+Bwqkm}NTLG}+=P*l{Y`8{m>1S>HKTbn3@$-44L75$HO!Cj0cO51VH?x-=e}PWl8cQ@9d%vqzvlVmJTXu&AM>Za7W$Am0|Ce|4UyTYq>g_A^({9{3t~p2WOj zEPl1LU5HryY6>H9N>ac5zK-InF;;CeHA z_7+{E_4~H)Uva>4Kh=0%E=pxk#E>L>E*(fsw8Oq?L14#kqBS2FX-vWMb2(xtXvQkf zkE>O#Yn!P-te)(=K$K>{I}RX!dO^_@%yy4ck&;f`xYxd^S2a>w(ofqrP-rZ8Ev(n| zQ8IaM%t%Yuu>4N-yQ(0m+Am+B_qrmDNX9EWr=Qd&;tG=(_g)n4=pV+& zbxx93kOlB=iQ!-HpIJ|{u{{yiD1B0v^8z0MwIJOv=$WWRsn?$KJ=zZbHL$w#T`M+? zwpj$ae8zk9eo^L-28xxN(j^|{f}NZmYnOf{|NW}iv%L9Dr^ob{FFi4|XC~NAZd;dQ zi5-&$3bA$qG0(a7AwP3j$Sro++-(z%i+KX*@<5<7Up=OLPCC%;Yu}m^Mi=j1C=G07 zc}A?~f^oI^bzMfbcOz3@5s(qDl74rERcMe>u1ttystL4qp|id3{To*OD9;& zMnhfbsNBLrL%3co#~$CRFT+{utZdSP2HN(Ba!XeMkBX~DT514iwPBGP1CQg#E9|~i z*ZfU4u6Nj>HByV>*<`SX%-GE3#6{`a)%uuzOMo~3z6flWFu;0 NEUIYVoV7+w&~ z0juML-y};Cb$-rc;H>R&Ya?TD*Qz`boLv+CEh}}F+Ai=T_gQ)Q*EVN^p`Ek6z-exS z$+aI1_L9EYef(!+W=DO;-%4oLsk?anSX`q_`ab-mK8IXoMJ3iwVPor7{=Mpt%eVDd z+h4oJbfA8pb;!nD-{Iu4ov7WnEbK`9IBPiJ^uDvY>0%^X;Uy&*DVcUJk;~NBd{ECS zr?HwBC+(mL>CpTUJ7ikDU$aYc2<7OW)0!MwWu56@|9P%_Mq9V=oeLF}3ZKS4(qqEX zye(+117}((QlGaqlDs@}y@f4-_^HA8Pc$=y(_puM8LQa4U4#$Pc;FJ`iPlN!&od%o z(`Z&e7B(Q)9f*5y!12JO#^uU8KDvX}J8y%`UASaNz6{}!oRo{>2Kl&kL6^0G zB?mUAT_rZFQ_MI;-yJoAx{>??Xa}UCZ^FH|ka1Z{Bq_NIEP0Z!Z{Ey42VN=~f*u_< z6k5U`!H4z-D<^t{P!&l!tdeqP`&dRNjVpEytu~>%Sd8EJ|vbGcMW_pt-e< z{rQqzQqp-xRhcot4k+cEiE9gAJ+FU!AW-FlPS;~Bl^V>JX%3Z0+GvURdE0owd1FPp z!f?iaEMuCpM4X8gEm&H#h^B)Ir#TWD>StoM?n-w$qb)H$x00}Z=^18qObv-DpLy1Y z9o+=gyN3%9kU7GLT|#K~DRI;2? zKQhZ5jhm5q_FRj(Ofpa!QGKLFO-?ai)B_b}_byroBcbJ*$qJ&kD6bx9|0(mwp&Y`` zKXY$;ewN)$3-6~=yj>NRD;D9VD6`8D5`a@VbKE7r=SlgZn!jdT4BOb}s?SgD6p;Xl z@AzT+U5K%L`pn$Y)%5e*l+{{MV&mp%2g7umqbj8Zw(+~fFH5{!)#7?J9qT9M?|x$?c?as#4% zlxz3w5V0St=Zh|X_CLCf3A8j5xxFQOabtS*X5g_z9P^hOsB?alqp$NM|t+yrq z;?80n%>wm#$auzyE<-tKwouDn!&25!A69E#g^`gAJ2jxIV{V%iI0n>C$b5S~f8X1KOK5mUXzA1u
Z13gdMXmi>C0to&$tg~wJ5Eaw7T*JjR{vn zrNO{b7sEQ`DeWcF+5~u$*>VWio>-s^pwye&1Mi^ao>f)S<}s-mA0v6?NiPIn?pK`i zV~E%avaB5@r@bN+6tPS=?Y*b8Zc}#$=~!(c7L2n-0=!g79_o7Jzh+ZITTPWsDB?!O zX(!W>4@mv1J$VytGBwAhki>3sozrdf+Ohqstdg1?Vpe7ZIh8SYzl|@=rAuNP0`44p zEI*%8^2s6QQ(OtQ8g%p&g$nmW5%JYz(~xV*1$E5z? z(WTwQY?!BYvA>}AYM)Mrbd>1No!1uK9`Vt-IX{t1;)t;+Vhn2y-xAy8jQZ;Jv5<+p zQ%VpH+RS%M~c-@(AKC+G7FKdcty!gSXudUu~LQjiK5Ah~- z^x6EPOdk8>?i1vCKStCl!r=fCTq5txa~x@kbeVeBIY3o@T4M`t5@_Iu(*d{ zHY?qXjDmmlZFx0@F>18keqn+}^A?ccF%*!}g7h%`G%WFRi-WFUYXh^&S>5wozkWQu zCTUK~SJ2f33{k394>EDkkWgKaRo(n`djn+H+uGxLRm31NBC5CWHjeixK6If1cHeEz zIkpbZS{YemS1pkQ5q|-T>j*s?TCIz3Uf>Z-tbNlctt2V?!J{f|*+xa$Rj(mk|E83> zsIV<}k%N})&{1s_gB2kME*NaZW{;-!qWRPI`xqbB^q;T7 z5|5WeKhkD>h>SI6>ArnIR`Q|c@a*X&h#^ut_p}s9Ox?7`!QgN2s%I8CQqAvcuVtUz z#p~|8PPtz_8^s{DhyycFsc+fKC{M!sM8{Bk1C^-h4;^xLHmH{5tI!0p)Y~Yh6prfc!$^`L(Ei%kE~X;?b47h|;HMk;_0?i=j2zh) z$W&|)X_1VwG{cIC)^4)_SUqL80jzt6tcvY7AL($@pKNu``2H6xE+yc??-KNpufk#^ zQuNDCFmNyQBzFApMHl5~Ms1AoH8d4(hYV6zM#@inHv8A}{)d5c`s%@Sj}n$gIw9xT zRn+6%$tWLd)TtfmkttjF4y=5@mQF>STGC@|jgohZnx@@~%dA>#-2Bx3tdbpz(fFF_e9yD`SsNPdA^W;jJ7cTODhhQ#11lU$T3-|O`mNO? zw^pe#gf2y7DZQNZepu=qHy>~)g)v2T+E>mk(w1Y(_+J-H5F|eCMSL|TiX{G_Slzf) z_+q2Ek2v?_9#T(6?uetMhT(ZO~Tz#0o!K2rr~#mC&Q^&pORrQ&)HNo57tz|8ygIs)j?y9kQi+ zNTT15f3~t3|6#p~SCIbAS$?6#kC*s|+4<65qC?9bLW9#!coIT$Nr_<-oFzSd)D*DK#?biaJ9zKeia@6vHt0cidQ={SS1>Q#WH7G%I2>g z+_GfPLo>D^#?Mh>_DZq*3VCs&wRCfA0uWN!n_{!_=gXA>e0DjC)aACH)~I*|TD`U< znlaY$ZcnoN(CbLSA3q(I2VfY7H1urb%Fo>V7R!&kgG=2~sSwBWhmW9n^$+-uC~j4? zYeZ=hM%3^vfH^QC+mpY{Ao}oE{yFA_vW~n5&4%BC^nB%bNOHDwc`=VYg-ESPSNE>rJ&}Up!{atf0_PI26Fn``nr` zj9*w_Pv{S+3at!|r9-^ofe(I`MfE3J910A!$>nF{mXRvom$X9aYlWBYg}U%_a^L1` zeNJl77C*~%?-f0Ei?95=e1rd}bn1|CaM^xPv(@NlRYZQo1Um41bV z9bbZ;y#J_b=Lu;+GQ}%9(&ZE8pGu266Q>qa7~Ddg2wy;TtH>1{j|t{~?I$0?*=S-< zV}Yz!{P5db#PF7JW16P;gL}vw*u@P|=M|EsQ}n5GLHYF^m5!h0UsgN%PN)&7-{6x@ z9FIv(wF=Y-uq}yGpC_V2`Hc=Bh89HXWW9Y*!&WNhT;=5EvhN)jBg=X91pK?#G zci`a>_+88eG41J_B8(tcjS7j3@@QqH{Iu`uqeHLrP7pNBl$AII)ezdDNh)d&3v|=pK4_l zwAycf6S}cFCvB|vwVHSKd-a1*AztU5!7!Q1 zn+?8$DMiMg3a`&>l(^A8j$IFt-!@Q*Dq31zup)^yJUet*)*+P@H#m$5_{}qa8$|sz zD2#!hDC!{cg}`bX&BM)8hqhkU37$*JqkY$0`4zHN z;;=V%h#b7Ko_g(nUH3T{Q5>)Ce?$cW?NON>Ze#%@_X%s zUw=~T+XNSqLn=H59yCX5e-t<7Lv%8?P-irHWPGYMYO*aEzOPJm!HL?nN4y`DnIheA z5Z-{07Nu5FebsrXczj3X{Vp&Mbn99sH2l^GVd$OISxC9A8@*Lg^o+rlb)bPdV>iQ_&@j(m}yL3FlL(WYT@I99cu)uMa6t zrxb1N!W;xD4K3GEssip1BTjk6$E&rUCj3gT^Nt#R-#w|Y#D>Idm{&R~pln<`p3L%4)NOpp*@E+m zTRoT`)tu&2(|vZ|8RMkaQ)D5k6sXWYUSCJ(UQyxfbF8y~E)}deXVjU@vrqS`M0N;m zjSfb`G8KDXr1auf3Aj%@(tC`ZEbeSg^&p0+sx0Y**MBb7+ixS4@){okdV&dn4((le#QWV%*WldMq!6)tnHVKO&cc z69*aHQ1Q6oki}S!`9~%pox&@2qOHxG4vR%2Lw#v7n8+SaI6~>)?q~nj1f17gOdp-#lUlLt+7DT5ZIK3D%z`ow{UBFl<#L zQ_l3$a=EHmqeTphDUdHUK(I%56$pUCLhtEU(Tt$9a#Aq1s^?V~wAlQ|jYIZA9XYQ8 zO^iy}+i92R7u;53<0!`@&J3-^9Cn_!sg&Ci;gCj}cI`((F+zdAHxLe ziw1s4hx6u*O-jndtP)%hI(*GWljmQ84D^BYXZ_ySmH)Nn5#sa`bU~P4VLX?2R; zmFahTg~cz*=&tn#3aaYL!S7qrw9JkkuGaH6^Q#i=@2{Ak-AKe;ov9l{%6>Fv@2Kd> zpGgtgXO1>C(kX04PBd=1D7(yr4QJhmeOR@@vi4fj2el{eV$m|x*{<#G$l#xSlxwN-KL$TkB+-^NsSGg z!fgB`iu%aM%fN&7GOOs-?MBWMS%oBxpmvLgvzDp~ zdoiR&7~en~L)Y3+ccH@WM))f4PmH%mleu3Liwp3OQn13B%LFZg-Dwyc7JWp+sBSRX zlk;%AklOPXY9nuQ@_T#u=`HFo<=D6ZlMJg&W|yz|v17dH%iG+xfoIA_iSFh$@%i8$ z*lVo)wt2?FVwDap%vEiY+~pcUkhZDxzAWo%AHBwg>=B-KnwjTZmrDnM7W)%Rrq6J^ z!VtwH89c}9c?K4d1>=kw?Zi<1{#}%G1H>%35k-2N5t&3;L@UIW5bY^9`IlmdwU|OyNcPDEkY`BDs zBKD?Cm97s~67y&0TIQp;FfK!)h@X!3g>L7)khmwD z1ADRvo>QK-pl->HZFSM)QgMfYp4z0}X;^fE%D9SjygV1Lvu9&%-)XtEYA0Ws?n@(d zj8j}Yos9~4JW3`;Qt#%?z*-kr*c4H_u(qsuMR)mUMIF6?`4P{O)uJ@UMIQ=3(Fdlj z_3e`WcO2PrYWLJVK47fA!Q+&2s+S=B5q!nhUc;GNcL#~_+ydw`O=_YA{9#GhGYmA& zqtcOd0RyQEmZ7!ZRO%RyYDjx(tGGC=>gaCm*J;`HYnq1i=7jx+Vg+RZWedtj3i~1x zXu*4%?i#Vsb67%`f5=9q$hVsFMKw*|;z746w;ygA#f4G#9t}4QY7TLyy7<7`1C&U- z6*v%WUH1s|XnXk9&8xms>))?*cA{h3oNq$>%#V`_M($Th$&#hTljw}FVdSe`#e}?n zSEc|wg_u?({dSvWQx#GPO!s8gM~HQ_Dz9UTb%*z4yt9x~y+J>1TX8SVP*S(+OK9L6 z*RJ@dQ>pJcfzZBR;N%fwbjAy>vg`)d52DF;o?OR$A)2^rphfT`;BIp6(b-cyj=@^_ z1G)1O;8y_A`UNjg*Ho83kAr=0P4$oIvZ;m)->4<5Z{3;W0( zGAT_^sS@6d5d{9gT1c{1=%T>KTuEp}{tsJUdv>g^aY)g;d^|)~K^u;sioQ!jt+~_T>vij~5!4x@_jXq08K*`K zM8PkyZ{ZH?=Uj-FKIuOh39`)@r!j6oQ*X{#kk*eY9|1KiS1N zBlkP|g-_nO<9f?_I=u5rD)vtV=U}vwN z8zo$SBm%3$##@*s|M1byES&1^zJ+YZq6HW+`EcN5aP277jjbaDS(~@zuK8-Y0pf6{ zDIqVrXh~Y62g0cN2Yi@vV=z25$DtL3NaZIWrYZ1Fv5K8rw-Ds|Je$xDzAqUnrDM&Oopr3-&+<@n3l z%0-xiG+`ReBxZ<+#>Mq(?X%No7|X4>me{jWe8j^t7Gy}y1Xa|Juqkc(UmfOimacPMW zRNMvqXaa>n9hvW0ld!LMk~klK)*`ni5d5aVwoB&0Ze%QbP0ELG&-dK@vnwHOyS8{B zAKA|;ev9Xe_&Ak?zEn)^t>o4Ag%a>h7i57DrZJaz6XT+i^b<|^h1Z3fra)AOva%R& zFlQ<;!9*ZP66?RY^m+oJEQuC!|gIm$X=lx88bf?OXBu6xN^wze(D> z8gbiZ)Q=x?RO3S`_^?4?Y3Fy5v@9k2GPn90Cd7QSe6_4PVyg(76e3i^@fX#X7sYQz zYO^A&Mm>e}A0+>FjaKRW;Q3EPiQBBCH@cFhDU%?{hP#xswnLt^Q}6SH0=wTkL%fMVOT(0kQ#*Nn z1xMgCygCz~#4}&**Ix4mj<;JE7F=jvK(^!MTR#pMO|D-s|5^xu&e|>S4h5fRld-zC z-BewYE%Xyk`qbdfd*k+!Qt`SmZ0R^3v&66i&o!^SduT%y;=PaYH~$3Tn~Eq}1LM!F zUOcp!bI7Kfvs=CXomdXCC-61)aKGiK<$%hN#b5Exv_-%VVDeIh8;2=%hL-=nIf`$J z=MOvC2eKQtee$P0^U3nxVb}pEInX&^6|~ zo6QYV5v>D*V&|@o>xQFr%+ATkR0n*q^>pB$(7ZyM zvYLU{jelt#^ll>qABwF5=}PGM_DU(PQ45oE+Bpv3YXSHYMC4u~6Zqv%BjXB|;oA|+ z*gQ0wRe+X_c{$@$@~t-C_ankGE^b5o;OoBqL&c1bAic-rla*gGPx%!#8@8Al+x-`C zY}F0wY)}!Ij27F`n=<7SQ;WalaiHBo(Ua-C`{XkTFPyc%pIMu7k_ZJy!*UBOsri`3 zG1&JSv1z?64H?TUi-tMVc2hVK#EIF&DXoL&mG>N;Y7{?!7yj6Ey&I9xPEsX*Pb~rJ=WBgkXPTkS&b8?#lNr5a`C#M-`1R4>vCt^M=$D=R~)8Y-3#0Aa4`GH?|GiTkSh0=vqw#Dt34q<*Bhj?~%Vlrgj`^2B}hZ5(li zv#FoewP4)BAF!@N*Otl*x4zN8&qAl$evd>vwed~~DeSjGba?u_kpXR>9sOv;e$Omd zDw?lsya_mI)s*y2lgUcYfb8{qaJ;PFZkUp?mH(_=_wEy=dC)XNOX#ok9(`7fpkir( z?aWHyS|c8}h9i*H9bSo~M?u*d}cFBmr*5xVvIglYm+0 zSi$opF>K(Vm|we1(^+3qQ|Xh1 z-Eft(Kl@G0Qxzu%9}~C%dBlAKVkBJ9LpIb10Tb2PbdfgL{Qh!rXvbHZ!r75yQpu^2 z1F0dMrSp>}8&ybH|4skhh+Z?JNA#I+$2yrWLD)7a(nP`@Dm0yTCX-;)bBJ+nAo;D` z_0EKqr!Rx3E!680)ICq)csnhhir_@MZ|DAY#0LeD*~50emKv`z|2Q#g;EY)0+gUH7 zm4E=N3-@zU8|S-rH+{7CJ^&B=+!<8y8pYa|N4oMdZ-|X0_-e~=EsdV+pX6`aLRX6k z@s!smO7CDy^rY>;*j&?LV<-VxkJdak?`TwcX2^}kS!>pLzfuAv18qQodSDM+^Es2s zAq?mm5olY9uo{~yRnLz^EexWc##$~b5w(C7$_CDbeq@B?Q~`#t{|gqAn?5Hlfy*LIZ|?|x!V4}32xD7n;ma+MmPA5IOw;;Ek11b#8T zb%w9&a3<}(`vt#q9KTwDHY`{dsrLJuY%8_1CT@~IRkkjg^L_FyKHOoKJJL+88S3Wu z47BUdr0UTT5{+{bnR)0mgTwnw72ZUG3ju4cQ7Uxr(&B zFb1ZjkyX|wwLeJ6tD0nz2QtlNPLLyaRk##r<|(W4-n-YOs@uF~_5jEHd)4K$z z?N&AB?P$A2QSkylTw$XzrRueK(n&h`RCJFw5e0LJ8%2ZGr1yzeTZhe-EzDKs!>9AL zwLWc=PCXv`MWmEFgT9OZL9TsQjUi*d64=o3q}&|pw$SeI3Y5h@60c2YDs;|b7P|lYfsv=vo<^36Y3lfs)Dzd5s{_59<_P#wJEJ3 zF)T49rEK5ZkDZSY0`_r&-LEpsml}s_uy$3Fgfw?;7r($?UxH)}+xB$HV^swQGvKk~ zy=wat7587swVJ!{ozE@Q?>$Vmkc|=7nLfn!WHU@{paxATTO2wQj%I9mMpow~jq6Y1 zXwba~Ro0Bpl6yh=VMfgpVY4q+ZYvaZwZwm+8rB=5Qe3674}{4MSXJnAWQISocjT=^ zXEtT`4jUAw-&6^((-I-tE^U7EMDo?J#qx*naJ`0g&63ykng=Wv5riKqEe>WHoILsN zHrR<*sVF{tM^!D^-j1Rivc5n4aHu*?NaHhyd&PIt*%{d{FE`q3%xg)u1*#;}UMg8b z5OWhl$BVKLDq#5<&(j`%+Fb)p^ILnZ+UE&R_OE3=+r6K<4aIz5cwEoSXz07 z_(n-2ablQbUa521Vh=`)u{NSx*_Vni-BrhdjoO$~{iL)0T*iL(C$h#!Q zNx@rv-_aalf^nzTd0ipq$0-pN+m<%7tJ>Jnz9h-6;O8^ak0L8|556-c7jX@)(fc6V zrP5U0=wTzAH)_qI@AXvKW)t!6Zqu+G!cBX#$oaD~>S|jgHNsh*&I&RtB>&v5o!e-P zPQ$BIIwq{5P(5{V(+lG59a}N za6a3-#cyquN5>T>10)SZKiLYTPHex1Ol;9XURyGx@+_uMl~>Z<et^y=K{z4 z#>_SJI9bYSr$Pb}PeS?p-$8-@mx8DALBxq%)q{p(aPR_&GCdwffQ^6*9RYnTWl-tt zIp!Gm$tPgDE74Q({~_rt!~hYySR3n#02?rjzVv1}=g0p< zq})FOny>k+ei<+Sdu#Q*5`L}u*d*c$kAe4MEj*uNQo^&d1bGrrB8arS`?<@j)?N2y{cV*!xlU^=jj@jS4w8B%g z*A%&5<%SqO2<>){Cbt{nLGtTcJ31ZGUnI2#+Dz1WE2H&j|ME*`FOiN6}D!zIkTjm3KQxV~mgK)Mb*I2Lgg8bz-G@{Y! z8i39pSi{glKuunUw}2G`o-2=;MqqAD!-Ls)A28g<#kYe>M3k| z7}xBx_hQdrSoKLJiP6DmhRsKLN?#XEE#2|{l@<%j3wg)(?46yS35=p+;r&WG)h`ch zgf^TN%hOWUd&O;+dHIxwiZ0pvPd zglgn$1Y%US-BJb*%DeestJg~VIQu-otRibRFVM-&Sl@35H5%65u5Vxcx=%k$GicYo z4_g<$@S~gbtpKdwer6#4aJvQpjQ-(d2ku|O&fl1Sgk_f?r@qgLFWWm&QI+?4#(gzV zNE>V87H3T6$&cXk^GnC{X?nb>ezK$2 zaM1*~_!;JPaz-~#;ffMgMFo3QHykWD_*&ozQ3c8QlbymNGh$o~AdXLmt02I1@mOI{ zHg%CzC{+!`rn+5beF8j2vzp1g=MwVL-Kk8){0v>_yYZrGk^*{!Dqu-;`8~E1jHR*) z>oaaCeSWo5-E&VXuyDZgxvm>p7+8A({NvV2XZw!r>4eT8+6r#?odUppeXAZi`Hw(l z6`@SK;scR&ue1$G8OFFBv!zL-yG7ontoIF9-Q#!_?+m)_Iqj^H2jBLOS1qzq1Nx)@ z(f9Qk;8e)ZwjbJAk0>0V)ZiHfYDSDp{{y<2n_(cL{AOBV(kI|B_E!_T-Soz1P(|4n z^xhKU>0-R&rS%Q4RZ@(YiE=k;Wv?XgJmh}0G~Go93C%6&(9e<)5*B-AM7yU7V3Iqb zx_wwn%}5fT-eiv7hi>Bq?=;N{Y?}75UsEQpN>e94^UH!T6$h&MJPE->GA2Y@UbW-R z5T=@;Va~^I2&Hu3Py;Uw;ByNQ5Ek7*g9?#WDJf0olNFf!wTwW$mO{zL(bJ-PCPynR ztQRgpU{Z17q0Kh1Yc=vcrIv>P>w|oyT+9q?AYd#(g(Jx>1E01VHJ3Hudr`Ll%V5o#g7VF584M$YW#KfjLiS%;qJ$L zM=YH1&@SKB;)8e4?O7=<;f-6;z=b@BBNZ+=pZ!Ln`KSc`kHGqSk_mew@3yI4;GYkc zrswvCcsG0#->gXUi@cI5i}_ofB4xL`FGDIt@uccVy3CNo*=?bH ziHqq=uVV55S=AgmJ5szFu^x~A5dhqWl@OS!kr@cl@{2_5H=Y|p`weWo+OI!nZwn)Z zR?0b2Sn@POHVt|6>#&y4wX6c}%#_BD`sxmwt35`u>d&Nxcp?%9`W!HA*lm%=^WGfB zvrG4=k|<_NM1Q>g)jT`bs8_i;#prh&JGo`HSoCcNNT#-RbA8I{;WLCtaZ61i&v^7@4s7{?(vEiZAW=rs=n> zd!T9CZk`L#!Y0kBPe-|nc9~`HmfIQoJT|m3r`qqj(d@tk&bG$o3%^Um-A-{UtM!2 zr?GX@ZbBisM(A`U1`g%Bs+k4LbV0J1utx8;rk@5BoLYT8W99l#2(ztS_`#hkp0d?x zf9#lyTR##m0~*c>>Uk4bRLtdE`VJ7h<;vLFG*hMKZ;w*_!jNtnM0AdyPU9|W2qxY#C$r4L zI?5`PSRhiiLJTH)Bi%6=mv$nfs+YC+d}Z{z>6S7o2TY8kv#VgT&&Kn+2fGdl8q6VZ zo=clurk%aHypiLv>c1b|i=s{V5xoi&breFg?JORO3?aH)PYkH5tJ-;!p|vSjnB zRqpwLKj*=RbNeYq;7GD}n#XTG?QcsNQD4+0=&fpL%kICDHi0q+(}P-mb$r0J94STY ze29G@R_MZU4>pvbya}SF|MR3%rppzVD_WMo9jsZX!=XYpI z;kZIX20p(x_JMM|=x>-1nK7)HJ1_H$Fri-zMf`?Sh(<`rQ+D!h((3e{0=a+&G4V)Z z-4}Dd1xUu$Rx6JBA3?hPHEM8-yn8{Y&5WJoAA#ib1>`xif!nUi`Q>2-_}JRi%$L zs!%!`cqW^xH~nscdj179+Ti7>75EwjY@@1}Q-qGt@7(<*9)=Yr#OeJx7DsB98U9_v zTg$zLmpYA+;q~CeY&uNvEP0_QT2sw8vRGWT{g$@CgH!X&y4b(!Ic9Pe7%}?lQJk$& zAG|~`*dh!zQIjBF@)M}6tp$0`^dvnLSCwVbEB}vx8<>okQ|3`+#1KX=p~8gb^$q_> zN_#WdWP@@oBccMjx@_^vwtGa=sz=h<7i>%gqaE|@>j?=72#^QKs=gWr5tB$R(AOLE zy|{NZ7S3K><35kA-L$;$yc``D!wwkpNN+|aV@(ttnX~^~3(m{*$DH{;ucda^IeMRq zIi8-(dROj=1Y|lVhM2PH?|uvZRXjB=M{bUT>>x`MXS1XHI*?YTo0rjzDg20aW&NnJ z(uBm{*%K1JT;P8MO?&D?0aHRvtU^|6y&^sAD&O4XCzTNEpejm=ddP84$O!UE<{c-A zI=EGXQBzJ|+f@G5L7ERZE)UrSLP$hLL&uWrC=sg|V_)U3Z`R}5oYG&`);=_u2sVnY zMclL)-HSDRN9jD;uU)%^a8YFuod4|uPLFhC9VvB4NXh==QhJ{~3fvfuz&c1wJPk#* z8u$354%)IfzOzFhwrtJ0(>Wb!|G2)0?tpfU?CkE^Q{Ni|AAA*1xug*=l}Q|0%YTbv zwKUwL?XN3u4n6B(53&Qj@bx6z7RwGWCIde3qL^A26x@{H{b*PuGlygYXjD_j)H!tpAJn0-!7^VQT+b>$wL$uno&Ma&P$_dc}cTUCmW z^pq!up1Ie{_kVCJ5g&3(iI7x#CF9YkvI|Q5lOtv8Cu2@u7BT8KmNpxqrDZ_h9Qj~i zmN;0AXa`z$QqZ7GT5j^QPczq0Qi}j;QEaB`Kb?^MV>9%HT zXL{@sw%5dt(m#JUXR(Y|s|Qyw@Ln#*f{x0&ed**Ty=As5CPSn?^u?f@eLZ!x#2O8q> zKR=E>xdVB)1I`ne7b9 zx&Ygr$>3e0J;JV!BTD*xTtZXMCztQ7L%LMo$a9lfxA_fiV5d`Vt2fx_8EW2)bg3S? zh$1H5#aP1Z9EE(xg+eM_giSFXDnx?W=G4vX`o;z&CuqOEh7sda+4x-~dKp zAbItyc&Sp13$*`7dAGX{j=|bxSX5X%`ieMUklxyAxq5Mcm>b7Pq}1mpNrH9z(+ye1 z1(P$|!FBpVuhmSgQW~A5={0AA){3?$s&(In!B%|{I+OLU`TP>EL5^(OjOOV;b7(z$ z%#ZW+YbNR``8|`uZl(Ub)5J9@D*K1eY!>b{lh=l(1ssQa=#)c|e=CYbA9d0|q zzRaj)^I`Po2ryWsR`R`#Zh?9t=TmjJF8qOAYvaF~ zs;^$s!}fBhs5tE{#SD(>C%KQW+NseEGIbWG%tK~yiChBRwLKAgdN%|qh18Yw8#1?Y z8#W>VxU0bgy6R2q(1aXf^!Q>u%t@$r;q~~GB~^C%nAzN%GTr(H=S+x5ekJwv8LHpq z^N#M0Nkn#RjsOMH65pIwBAW!^wB70+a-9Rm(+I17)*csdrQ3dAc)j^SsG>Ion?WyD zZ*+{#I(%)8;b!`zC18W?pme4*eAigG9vbOLS)$TC${V+%7CD|XXw7GM5f=R3qUil% zZYU#$$4_&`s!k!?aN1F5y_(VD7ARm1JGRT!v3iC89t%cEwF*n6LzUr6)wcp$u`)r$ zS>IgStfwzjP^nr6Bs~p-S&{UlV7&zIKXh8EQ&k_mWVH>I)Y86`_*P@|{X{&TR;pj4 z*6lg^8u)Vj%$lP1-2_a$p!;~jKq7wAT9Q=o_-%B>OSbISr&xDGQ69JeKT47Fb_iVG z(!6;%apJyMlHNa4awZ?hKEJO#T!mcbL6nJrY0 zpN3Nswdl=AAZK@BofIkGtxv|jPMt((iCvcd8-(K)=e)qwG9e#3i6M+M<+zYqqAE=zL@8CRuo93 zmph+Sfb#L;Bj^k`kzW!_kgU4rRGgJ0@TQoXa@LQb3f>`CuYAr$E?^{i8Y-pn?q(WR zmIodA!|E~qEBc)1nO7BLxa1>O%`7Z|TchjM&Te!HGxzRb-3qAr$sY&Em7&dFGt~6F zqWhB?21468+w~7atVV~5Ds#5(EF>a!^lPyN^7*o-=b;2R5Su=)JB5G@FVcKNCZ zetILYK(BkCQ4vs_H3s+jQNh`@Xa%_)8UUbA^T%b)siiR^jit52K?Q3SPlRDiFJNn{ zo`QKjwLP=*n1;JNH8|a3$L^JNfYe+DTZ({*@=++xXszVg(2Ke#|JzoKM+;6rS%Z7W zR^DAetu^pL$v*-sf#|>MLS27p=PkbMa>Ic`T<;?vdK22}%o@Z`BV#XxtiLHXgZ5WpHXWLdI%LmVs!PkvRliSm4vbH{8Q<{#X}+OlTith(4&Z6Kj4 zx(qFV!c1}>7R`UZhRGAs@dPw;;Bgoe6?JCFaitH zt3AxMnQ<$!m-#C*1(DWQ^FU4L{3Ebq=oAwg63s(B6BcxPlw14rZomh-y73wL&#*9z zOC3+*_@g!2YIm_wx^5z*%ecy!{2+8a^*p!_wn|-Xuv=;9?2N4JmEJ&8v>iw#?2xeE ziS}`b8e&VqXyVhd^o4AoJ%0+`L+m#Ao}-*`d*`nv_`SJ2SpTf6_}B6LnP8S1WMF7% zD)zlX+H7y&6FGXwFXA-=nt**tAMi;F1~URYWR9{|7OWVRX@M0|1J33PNy5b`j&ln)Kypv_1RT5mQnG+ zcav`JUWzUu^AB1iWNDn`l>y4h^*RNV!`9EY4xx8vCiI>HwY1n3CTGos_Uk&kh(C+0bCPYRJ8x zxU2~mx#v0D^zA&*EAHInsE6sW_a6c0#j@Jy($AeZvU=EYF-sdFuyywNs-uO@X@Z38 z8Q`$NHN>62fTtQPJ@AE3cVn$A{&?tCaYklemFD-st2xc#wvEleoTJ|{e zS$rNvopb^-MG~OXLblTcTJ}yu-UfHa`NQ{zZPtRWn1b{kEUNqzbx2AIQZRDW|N1sh z=zc0GsCB%sZe{R+MS?PX4ujrI=Zp#ma{R5CF2Ls!Cs-I0vLCOZ6!;(n$0fZVV`{kI zSw-YosQk4eUM=I;$w#KU2Wq#(zA=f(*8s)$_Mh@@jJ-u7T~B?Zn~(aM6MTl4>X#a_ z+ce5rC73Q&@CG*9N!j#&ajnsnTs9>nULAn zsTYYmJ!ou9{6NifNh(6^GzZZG%Pg(uc8h;eRLKi6vw{T4N$ z2PF@;m^PjOQ|*`PjDa7VMSlpSLgdvyEbi8`L=s>^+781`36esZYd*XetVYo_7oY&D}qtm?}9a zgWK&Um!>>FN@fmvl6P{)wRcI$c7Gp(v6IjC=sVaI1-LI`Dn;Eq~t?K*uU)6nbxVeG|EHOx{ zlHA1d0^AEVjbDw2lkyIoxU_^T$)56g^r0PY&U>#9UdIM%W$o4bCx^ul=NZzT z{#QjQciO%3OtO>^&**?&p$gH_Db&z5H zvfjiUe;|N?$5BT?)i#i}eM+peaHwBPzpL$&P(#`4X>5wCSCcYLJE04AUwT_bKoT+N zavU&rsdyWdmxy-WU3q7z^Sda;X7_~KLE`WI_U44A$!|j7bL~Al^Ai4B9yy+EjnO6v zSkR##zKhDc-*n}L^Y!LBtj|5;=9s4E>dVI5Y*n?ESE92a(g_gNDr1`K$p@a7kg9X8>QnWu zPxUStt0qL$=Fj-gbTBcfg$8=gwNj8oK-jR%zFK{--I9c{v}`abQo2UyCfrNC)fS8w z8q>|1V;m>RU*iH;TWY84TsxLcCpClP11<1_YeMK26e_ZF$>{fq$NSZO?f@9natEiO z+cGk7qj#xYG9%QYmYCyC;=hH+#(U=MW#7h;9P(htdPth@nA~{zi>w|vlxh=fS=dj` zA@Y%f{{ZUrz&7&0SrE6-Fqxxh;M(9``nIpaz?6B*2Zi&b90o!&k5;n!! zVxIZR@{PXNLwo-j0r)h3)zN^@8cZj4rRP>%|(q9N1P2Uvxt7&|Bjg>d ziTue{)2@pUC{J&Wa6+0KxJ{1#x20lL;v#R8v4MWHPR@8cCbGe!drPeN%5bGhyVo_= z$TBhe$eSHwNQvo5bpuSi%Is3#E{Is7s}-zmns7XFl;UKA# zGEmTid3wsHkxzCNaP|Fu(CD(QSy|_9OCf7w`my3A1@)$3`|W`eKdD*#I-p41B!35Y z94R3dvLGd)OO;qP6^C(+9!O!Yv7NdXPS zz;;1tC?>7OQ1y?099%fDufw?KQWC59*X5_o)!THB(Pm5f&lR+=XuVk)UK(Ez2i?u9 z#Oj11T?_nQ#!2Klqp}8}wU@1RL+rQZyY>DGl!uv z`jZNNm+xcQhG)Qi!lpqjHFVCBoeQkHJwCKPb6!XX67T-8uDKcLQybY`sUL}|a5fZv z_u5|a`~uppRhlW|+u4m#&Q9rc{%Rn+4JiJTASO6PyX#!{s;khG`&6%?OEgfWrSG%D zvMbYHgI=we*=6TtKtlGpi@jxA`>*<^`Guvn5rO#^6N)5=6UWNwNB}01=`~Kdepk(G zyh8*vLn5j@M%0}O5i8n$Bm0DxetZ{c_X^7>YH&zq_(a9r{LIcB6CQ7>Q>FZQZVXnX z4Cwxo)#VLzyvqGGxa6Heu;PZ!))}ReFSpqE>ugjGo`i@N2dYMDYL;WKr?)yn?W2)j z!;398t39!mZEGe1#<)7J$2dtaz*FKi>v*ns5^O;V?C}IDdvysex{~dl5P*&-7ZY|o z+f~Uc-ti%<+7&)IE}s3Q=A+%2y@n28PsFbCyaAe>?n{+&4;MG`xQ+#!wqs6uGAQ0R z9+(6!>4A;KNb{%x(_8$kKK#nqhWa*0?oyW-tYMw%fFXYSl{rwY_Q;jW60jXe?oQ-O zl0WHm2e3rzv8_8Z+vZEu%C29;6MigoIOD2$OPSj!Bd{welp0vgqM5CEDE;du5^X>c zS!I*aCgWHfXdXbi{2s6EpzW?v=lwVETs}Xmk!6{q`OAsZPYYE{|; zOV&|3LL;X~ZJ@w7q=kjG`kyxW)F@EFQ+M5V(X8LPNyXMO(4p_vNG;19JC1Cnzxa6O z-v%rG=U6j9c_OmB9EzbH7zSbU{K1oGup4gJid&D?Aq}8z1)g$TABzUh~ z7*jr8Y|hjH%eG1E`b^^wG?D_9A(dvo>J$F9X*1t~h{X1--Dz`w6JjHY2MmCP>?M2V zamR^bl?=Fi1d<)!qC;SaZG~tNbOM2udhV=eCNSN)smv)8a@oEh~6Ji0Y%=~K3DOU z4%RL%MvNUOvfC8XaqZsU*giC{s6SNhAzs*mO^?epQCZ?+f^L=Y7!QFOf!%^6)5tEI zNr=UsSGr%byUkzU{Eb%Kb%oR?igq^r0sDBUMcrRd7jFnXt+G%w!~9OK_a=z8ea84v zCfY|Zb3QsKlGq0 ziI=h)KC_h`^Y%xBM}}Q5(&ZRFZdN^2x8P;je5dSe&%ss^<u9s;t9!>WD!)EwG%T zLisnkWp?q3ZPI5Vb@Dylj$TH3lzuj}Kb@6#9?x|jF~Rk>{Oz%KVEbg$U-q@3BjNlboms0T87}|s_?dArh1yOk(S2)-|`hJ5oVA`_5L(oy&sQ?E~R$a3#3+|Wk$yIGx3GMf_z&gRp* zXl%)*@Eft2F`o2Xm{ERvqt^?3I>~Uh)!zR!chgui!hS&koA7(?E#qWJR*^7gv@xAT z7siV>?s2-EM|;1q_N+b23hc1&1zQ?Zng{`;_?mZ|y99H=(*G8apiUVDmETe>DOb!T zlWIEjD0$hGeqY+e2jaMNv1@rC|M&6DkO_xdb*olf-&Ho!GLMDt?}YM(`(eD7GlLJL zCG@a}=j&O?(~-Su0Tz!EvvX6<#(luj z^0y4kaoZH@`E~Rk!N<*x+hpFuk|pa?yKwC7pbw)&-;Lds^*)m{!$|Ws3_ncQQM|Jw zs%1+{Ae1`K-jP%&Wdgt|Gx$cYpioT__1d7|We0OE@bExwQus(;<-Mj^3ZnohgB27Z zZNf<#%-pEu*a)!@-2~Bl=6=bG0@3tS0FW<#7j<Rxg<0u5BQuh zoxgr>?|r3jlDYmE53V3Q*K{XqLUJ)CoK8OS6PA30s-3x1Y~f(V#_!BEfOk>!^;OM_ z`stJjtaY*3#nB+kgmwv#f@y;_*U|-Pup={hAhBgt0UJDENCsb@ahjer65o;J{&?PZ z*B0#?g3j09=mm!+tVAq335iKNvLnTx9pWBI5t{jNjD9pFHSe$-hBcoF39Or(% z#9i+CUb4$&hc^^6ME;{p`HWNFQlM)srpE3GM-dlJGR!1?+FnH>=}J=aj=DmRoU}rp z{;_V2`ht6rmzy9aHof!H)r6`e>-Hi39*Uq!WTO|iK4-LmqZKTGnxD0|_oseuLnYtx!8Sq!Bb?qay;VqZKku`ZCyXxlPrUO5+BTj^SH;R)G({(7aet}M zb;AI*^-(#uLbaUla6FZd`#biU+h!KcLl9ba^1zKI32FJrlQV0$^Qf89)GNq$o?TU@ zzR!mz@_F`6e5B{uc1o4zafYBjc58<6rk4$t3%nn@krs^}$p-))wqY2kaK#kGD93 z{xtY&P*aF0K;=1W)bZU~obb`kI6p}MXGeJMmFzu05555RMpAurlp|YS<-PEP%YbW@ zLo&KtsiEGKe*_JT99kBkaW=kwA#0_rsUj90?=HIiXKhlz$p3N)_s21z4x(8wV;N;T zTz-6>grg2fATOwZtq!hUt0ZaZJ*ApO?vta~!bGMGgXG=jUe}dfF-_I{YjH6}%z(YPPn2lfw1=_&glNulx(%d^vWi9@n}00WVujN^iKzwArw8Sp=BN3Nl;4dD z9?t%Kk6~*X9}b`)GwY0Z8~pl(9iSg{h7wXfLOlKzZs_RKp8yf^u1JSQo)+asmf>@x z@TpVz!%rWbflMr~c%Xw1UE=}gW1m8ogb zOiV@fZHNiY{;(*#(d}&s^rjXM5gvt9;|oQJ|4m({FhqEyPJI5=*qOeYLzdrh69<#R z`2!Dsuvb{IPdYNUJC;V?n<6;p`B{mK8sc>OQ9ttYGI~g5zR^+#^0jLtiO%?k$(X50 zb?5=<{g@P(MR*rih-jOaX0-c__K5cTP~&W&5k&=OGeBZkzWguMtW6h8g}HQm6Wzf* z!_An}rmRvw0LpmVCrA8M>+X>gu<|@OOMX$J^LQq(7(i~?AQX8s)Lm4%m9(@#&QaaV z+(5zva8>-u{E3Ddg+Rw99>xCY_+&8I`t$END;|!bcE!ql1hHO-mftA`cP|`b)`a_< zpCilVLXL}#N&D-z1w=Z1Kq{#LL%IgFyC1gp6(yc6j0c94I8uRXlLEV2VXcj&>0YhV z^i~)_DPjMYYDZO`S`UYb$DVh{+(*y7NK}?-H2GS%Ro3Q6Z&jLA2H&!^4*z)4=NkKH z&z38tqe~CmedKO&ViXEp&aF&a*9} z2S1>d(&<1!*MH0uzAIs^1NUIH=>KE}wI=U2Z{;)V;2`xE~Wu$t4Ml84w5Zv!i3 z)Fw<_V!Jn-ha*1xpe^)PXt*pi;fGN*lM`C?*_3qxz~#{* zzI0M7t_7v_-s{>sK$krYkzgq0IRp-ko6~0!e%=n$`7_5=>o0FsD+Z`<{pstP-7viS zxqr~c*kfM(n|Fp!HRe&<$)@fVW{oy6!PT|ETW&Yl=qMo}dLed4c$jWls3x-UQ+>Do zVCqM3Q-nKNanQ7qm=CM?L= zRL7=?YHH!9`=bs}Ntz(pi!GgpclTbv6HdU{0bC8#h>wf$=JpDlDxFbu>$M93%wau_ zal7L=KIVZ|j|_9My`vFz57YhtHq>czzoFofdS+IYir^=CKxb+DI7%r0S5rAZO5Nuo zq>9B}mdK20A-mWc?CE?k%p6}m@nD4v9H<2@J_%&m5hX?*U8;87Di-?D)DfNj*k1`e z#!Io{w{uO@ft?k3ffImR(6!+8YghX#buT0GBc^rJ8P675$pMnh=99L-1g~?EtMDuV zdHj&RNWXDBBh>t0c+*`;FeL|;#D){l322GT79{QJk}L2jwexD@Pf6t`%Mm9hHNNee z1oO)SSl>M$)ct&?&(ORNm6(Fy*=SOX=EA-JSif>3M>G zE)f?3r$79g9;TB2uk6VwP){DS&~v)YKYKd@v8VHE(s4eYi11_cYDaRppqGGn@WkgU ze42wyz?(+qu&p`GJE`TN`R#aG27}#kYB^#x+kB0Wb1QR~7QBH@ZrWgL%x)1GlC@{M zbpf^srL4&4aLk)8FMg8AcMf?2kER%W6cX*a= z3;!eVcnl046}$osuT5qu(Su6vZ1$%!{#a93gaQp+e(lyOY#Rhk>kIvrvo$!}qwreW zHGTYWmw)IIl6ejnsu{9CQ2d0q)G`r!39Vvi>kG&>4M!3~=@DtNV#P@t};fdn>a810nwP)qXC8Kg?ddDPv%BGAY-N7`9(F-2YIq&}-h zjln9f;2u60Dy)?Nn6NG93V&c*SECv|lbBcdV_zF3*Rtf1xi2@dmE7;Ps!%Sd=&|*5o@$ZjTCJ2__uWb$ z!3Val`)E?#-(Syl~AaOwe?wK_w4DK1KSAJ9l9 z9hG>JaIP%dE6F=X=rRF8%O%mfx#L1_nX!39!!^J(3N!#PSWkh z(&u|7c1vDs_3XZWV`Jyg4K7AQ;jI$XMoIq6s%X(7JCzlWP8Gh*eV#qmG~^p*NkEXmPja5(?NnQ_F@W%;ORXibZG+qBT$`KsbxpN5Ea=!hBzuo+}&tI?@u7N4ZI>- zx6H*Pd?87O?d@-$O`xXbxDXCg@V5s2PGwOn=&*i+&)oL;MtplF$o+9h!ym* z(h1F7*xvQyF2c$h2D|PhD!=sdW9f>Jc^-dx^uQs{HL%Y7>cct2Q&aH2(YtX=(bL?5 z^_ag{YVL0S786!|cj7d2AFtg8=$Otz{)ow)>uAaxyORx84GfOR;1;YnZJ;NR-}Sp zIk-63!}PyTK(=@Btuf9>Vc?@((f?sHN6Z$++O~k=B2rsRd?rEol-R(~@_o~J9hL)K zPed~P*FMjVj|r@iuOg^(bgHuEaqr65pDBJq{z{7Z`REvMu#S^qp}3>@KWK1C$sTd| z!xFd*)G(xKpY-s50L|9$uhXIBiu(*#78OibmyOzvUBiC_@6f7?T=%Y&yPJJBn?F}m zHCKj=HWJ~n!L>5qVv9s}v?}@nA`&`gv;2^np_w>oZX)#vo`;b^xA4FkUG<%j>y6kXISQuB~jHT$Y}oZtiBK*DvqY6tlFwlE(e; z1tW0gc#{R*;mVX8;wpNG;Bcky&wkB^Kd0e<&X&g*Y7+H1Qua``xfq00W9j?YyX6h` z6SG+n$38kobUFn^pA{Iv$(pSPDv7zk#eN^!6Ec3F7p@dWg!$+-H$Ja6kqy5W!t$%& z7!`IDDh7BxeEBn-VkQZW(pV_}WCz$?$>E-sF_7_Dz=_DVjYa*R;yW5^Sd|T3JgUt;RT%eR zO>%;WtBXHWuCs6+duUiS?1hTbP?pkwZ<$sMtd+8E>fw#ghQY0J+*~$Xe{3=} zchaEaZYjsEs8Y`C4EnK;sQS$Wz!-^*;Xzp{_ThR<@ES2f(!0o#YSnNJRNOPGe%s0Y zO7o0S+?g&ylUD53~z{+OmqLFL>w{E+K$+Y1lU(I z%Ndt^jpE2{sCZ1r02Hqn)qC&etTLp2M0`xfxPM)SD-p)j+M#~weY|Iqocoh`nnUz% z=J4niNT_T*K!pA3Jtr)z^{mA91vT=(e$>_OMe7lJvE0)_zwO~VV;Q%_<|do%0CsIP z3A>*|JgQp~Eu*yGOx&8xj!~EFNu;fw0KQff1K*ju6zA@;W#Z~_35y{rGFo}?7_g~r zEYIaDFbH9?v!iUbGHA&W)9@O=Dh+;JllTD9Y~D%a>Gy&4SKSr(Aqwa^ul}oW_q*IY z9GIRnGQ=R@LqEgcCv^Rz<|+@i6`5sohS z$$M=UD8_x=)Hg`as(wPE*YN;ztz|ZmbXR^$9L#`6s@ohX_HCL(ix=`uB2aUJmH5;O z8`)VH(&Gcv_Vl-206ED7A=Xi@TEK*I)1>uNu)~lDHDH~QvO0RN zGX`<2eDJws8iuv|Rk~-a2^f>CaaUHB+Ai3S&mWQ|^R|xSrKPL(f_vkPWm>ziYs2b! z?0`RMTBegH6q$<`fh~P(Zj9u_{|GQt!O0ZL>VfV|_ZT!o1Q#f-OHEfzTNGPWedDf_ zz03^1R&wP5ZU%1$*n9WWLN%4}!bRI^K@mV;Zs~qgv%XJ&;&^Ew3Sb|ws*Ar82Il-N zN?jb2mbg5zOAF62t<@Ump8*esG@a-oimMF@c%?W{dh)^!WB0~FO_qKIBt+=}GN{|E zo5Vp?-LNd5ahH4eZ0?Zm8amqppdenx5CD0nxJfuPs(S*`P=LCBK0)`~5uxK#%DLC4 zpm#?r_{bH!iu@9(EmM09T#CWYp85&b$Q)QI8 z{dn1^r`tTSuY)9HflP?nlv(n%c?Ex~*+92!u|h`bvFy>zPAGYY#>VkP2JYY{Aq=pd zG%lyyDDVP(*eR%)KGtK0TT>1;c6(*cuBb|z{wxVsO|}Ln`*fbyI&W1Y@_!_q_amF_ z+s35BW`8)3`FU@~AGn!T>tCe@SKK+{SujBDjZb|`>$jekbZI$Ak~ zbn$LCGlkhPcEosgybpwKJtyt&ofFm-EftN@;?vO#pCAZjfzb?vs4cJ2lc{h%xj;5MdqrA@v zyi2yXF&t%K+a-=@?TIdU;Fo5*$w?oq1g7teSoOed2&L)q(6o|BzC~z)=#wS56E(ja{W(w+Tr;E1dJm#2p;=YdAW) zvLyG6(*;SgPg^b4yA1p10Wym{1YqyFt5QeHZ}eZ(FS4ci&wnEfhVj2@7}F>^L=nGL ziPKEfqVLtDjik2g{6F;4U3YIQ(Jw^f-ME!h=s`f~URN`h^2ykb>o z+5Q$CH+3aGMUI0XrXrJ7USE&xR z_bx8GurqJJr2G82X$<87`b8f{xi-V3g2w?lf9kB%%Wsb-59Ua7*H46XOy&Fx(-^p$ z4?moF_k?}1ewY4T7}O?^%#GTfGWN&5k&cpk{vg|MGqe4pfTny)Ld_+F5xTpE%KGzV zj{GLjvNO?QaV*APw@`N~^8{K@xx!)>?)=^J&UnC|f)^u74$T3xi16`+=qaUx>%9`{ z(5WKh-scTsfILe@sS4Nd`Pec2Lrya*KiSVT&z8DE`9%H!L$AkWZ3@9b_a>Q-!h;fm z9+}dP7cZagE_|zSZ0}zKT$Nw<{N)n;R9wv*!x5Aa==e<{V{&xn@8|^|k3K1%&xUxS zJYC&R6KT-FU;_56v=BhmN4vtfZTtJaMPc!Zx@%nhV1A-OUt~w@O@<=p1B0aaNCTVJ zO`|tlQnf_wmgkJ=Dy>9q837dDj?9~Av^!PB9G#H%f+1(>CP{WXXc{CJ9c#R{ngE@Vum4g zbmhayev{B^mX)&>?V?PQ4eFI0D7@sNqpeSCHES@-TiF=OZH$R2o);dpVr=SzXhwAc zZ<~x`s&A>g8*K6C)Q{PyM|YNA8rTYlM@iWds*76uyK}GFjAd!6%RHHC4EvEuCym~3 zc;Yf@fj_H&433oI+k^(JZ~D%yuU^HjPo<*!D!l4#tB8!DX|5l}pNP(uaqWZ4ZP|pY z9TQCx2CpjqY6X{A{Qx2If1bjlQVkc4P~)ODCGHZeNb6C3Nv|Xnt+#g^{5F{c;;2# z;QIhkd$q9@-Nl}7F{!J- zlI^(EK~W(+^qXC7l(go#7LJ~2hY&DX9Xj2o*U$U+T(LU=2@&!+kUW#d8baWVFQog- zL=X+tI>T7O=fa zH7KK+ZWOFl7hkZ}@OS~M1 z{K5M29#_|zw_?X7tg8f_NsDhsmI`QC<;Cb`ug1f>cPBPLf>vPn5y0{fk=`?GZOfS2 zly6YSUC=78-=hZc4aCQK4W@VR14Ar9ofu$`jLAN1WI~R|R<_-sIO~=GM_FLi*=nqQ zaZr@ArfQevGpn;2afXs%BkBuj?mht%RR(S`Kym=E-2?k<-?Mb1l}%kMrMA?`4EgJ5 z?#sc|JEkO(gACA+2X#oc-U7mR2trKnB@KXx0&ssNp|XPnyw^eD2IiQ?XtuUtL804T zbCi{Tf!#bSU#F4>X8}8e7zEuv{YPg0ye^B+m$DjJ2Kn}9XS51qX-WeJZNEv|OVx}r zZF{qD8$Q1}86_I)%|{IUC1l+Dr&p^o zHHI^Y`x;oq`*LbJ zsRWBXXG2^U7%UTwyxzaJsJh7(tlHLFMfqp%a z=B}H*6ux=WGM&3{6P)4<&oi`%e+N`?HLCFJx9c_9{oo(?FTf$w|FNA&?iuu z{r*4$jf`BnoA~$l*ihI;y`cuw_AUR&#VnileU+WtuG)0H`dTiW8#6xe8Udp4=Ezhn6*-vGW-s-ZQg=^BkNh_80 z%SW4z?9nY9z+M@_xhh$JsobpAVd>x2!`+$(lBg>n#hG+Q)}f#o&BSTo{NPyL*T0}y zQzIqbiPt7z+mK>W4fb$#Y}2xvgDIIZA;{0|&5`4i=rFB@z7!;QH}sFOf%P|cSbB)_ zZD>uZINrdK>w9IJ1bD0*A|E1IiQXqq;pls1=m+`Ie^Pt=!jWGdA3)==OZ6$olLd&6 zj`uBq2QS?%!x(Vye(S*jg(`>qx)qUcYRjExL%FShl^|+&WINkiHfENBVN;{GJ^xCEVc$Y8yeJ^;vcXGp^|qp zb&@5}OCYhGKclcbDe^qk0-nlT57V~T`N>(6*o&4rsEh}@yO9WY+nb0PaP5fIbR80+ z0DN7S%2`w1XVPZX%xd60rSx^{c5R`s2CKN!&94y0=cpbtE?2|R6$fNa@F?pd!{rw& zYo|D2uW5d^OvMqz_xdo3FDD1@h*C?s(fC>mL~1A zs2!x)%)YL_@leP#9>b}AM`uD$#HE78(P?i+DlN(i5%Eg?;nkpv!3DRw$)yW>li*dF>|Som*zCfepZy z<>h473mU&`<)Z2#PpjivmoeOU_O-jP_I60=!o6VM;(h_%-1={^-No>Zkf#ynHcd}u z*Z&-dl~xm8cv7n17QDU2Mb49BWp)r#{%pv+(cJWJ#L3)x!bQ*g` z8D~9(yR;>}d+(iRW~%s#6cZz+Wt?PWr|9{{PqYPt2z3S69WHCtaIkKW;7pS2gm^!Vo8Kv6gQ(Mwew0S29hAO2q;@sAgffG@ zhi8NTViE192LNfHt?oU?34RRCkRc&uxC>?GAQ64H1R3&GA(ZC=5QKhe=K1wisI96$m2=G5s%%0Kt5;kQ-Zxh^XZ(2N z;u zx|6c*?>>|IV^=3hx-fTnRHN2?s`91TBb9dMF<6(B-aJN84j{tVb*o5hS5MNi1Z=Zo zFC^ALBXmowm`HC^D$a(nRVCC}3u(HOM72G4S-6iB2)eY$#$@PacATt=upMzY+hj>5 z;ra7uV7=rGCkDLGDUq5At?`p(4Co?s9LO6EIKz>c1&?YEns1wmUI{p_nB|=J24(L6 z&PZ`hWG`Z~uhMrhELlM-CS!rZv4o`hpxZ$V98dg}1gYgOH%BhX8@Bb4kS~`ijTc%m z`&&V2e);Xm+S7l|A?rGcRofG1$yIeR$Wz~n#Z|*Wc-<1ddRSle`KHNk`&nRjL0{k7 zJHwYcEp#;8{Nfl2`_Y%F+f*w4;)T4cULIw}Lyj-w`g@l^e*zyxZRc<_j2GLTN3`Bo zb^@wZCH|U!Sc2v4&4>2$$F-NpyJr#WU1lDU7Y)P9BXHX7!f5=oglGS|&)KLXVC)D9 z;NqG8^$HkXIV97+_`=opfDrxNsXvz61t>MIns0KIZR%>I1^c31UiP5heNoR*;B22z z&v|sEM5R(M+ih6W%v3Moe94hG_|Xr%6XVS>kY1q@2%kz*2eRFNx6JyYIyRH)MZb^+ zTFm@5K&C`==Uu#!NDa*%F3o13heF`p#zFEMx+a93m-m|~6<9sW`Z)BL!Wzb%x2*AP z+d~1o;sl1pL!1|)o`~pT-$yTb`hYtl~rl&hlOx7xG zAXg}l@^id5ZTjmPgh~7WFlj$OWXy(+7E#4-ZX(RqelcA7>lbZZunvFUZUofJe%ABLsJcMu5iMH+$U7)kBVEP zGtU-SxK*qlx5312RfgzhSXye<>bn4gPd^ol1!oQR5@C+G<3{zwPDIzGdH`!yX}~v9 z71m2jFIX$to9NzKE%CaC68cySPP$fa*kJ7rQ*78W zi<6yM5ua$0;5htBCTLuXV#ed8XWYzqbE?VBPeun>#!Bv^54q6|o%0}7;V_?fKA^s= zo~{1T(ZtV4#m2wYy6;IDVPM`L)!WTY)Cx5?4VJci_~LS!zA&;?yuBW|V)!m5Kw{Z4__T_`_&A**3P>xZ?GB27@Q++*>sKZUa zqv+jk(`=lSj_b)bk397qiaPajk#YInQs<##swF}L_4b3a=YEg`F%Obkti;Q!(#BBx z!r&eC#u(eOz2ZELraB_0hOU#eOV?N*yK)O^mE;c`+q-^6YSBC|gE!nFv;f5SbGR## z8>&^%weBGB&P%8D@->T`k4GGCSpM#?uOuM$|ah%d}gR??B zPGvSh(is?U#(<+5vBw9%$RFi#01oVr5sX0$lW$3pdfn3vqx5iRDV+&Y1T}m41%$_^ z$oJb%U%X1ZDoeA58>VoayI?DO|7BMm)uV&E^hK0!LS(>G(p+=QLs@kx+ujdubRqv- zYJobCy~f5Fv(ZZ!Zk?*xUVFE7kExIol1LpULlouOGqkyD@{9PK%2rLS^o#O!wXI5uwMWr6UGUe0$~!@W)Bw|EuQdfU!jR1ESq_IE5#>C(=5$$x;a_; z2v~YX@rJs{`_ysCe3LF6_rBTU>0yq((&i1j;%|1f$tAv3gsHVH$QTc&0+<2<)Ley{-ID*#o zZ`D#s>w5{-_~KG^HYbU2XH3^{-z_e_z17}=TyyrcFU|m73}USB8O8RYpo3o+e|`gX zx7vCa&qQ>P1#A|ZQTH#}T$mJ)91}*)RMkmv;bWcph9xq~^x_d=DQR2h{0@%PTi3Yp z!Q#$+2UB27>;qbvz#xjX41Ig0xr8nQ;Uh+g?$T`DaZsACGpJ9wTkL14Ficp_nKu^Z z6<xo@FB8h6Oq_J3TyrOm2<5r&?gaz>Xim4<-<*Q~E!Nwkd z8{)wXXdsLE{>)g!MP>QNlj47GP=Ec~E9n94$rVrQ&0^K}+lhZ{Xy0togtzYeF{gTB z7uKf*FZ3-c-oFzQEdaTE?LZRPUAAnDm$;8rfI+!#>8~E!KcDQKYs}Yc4Cd(}TW$KZ zh*@Tgt_~exdb@2ZP+;#gv$jXlcB#~`We@E{PC>hdOQy*`TFi~zj9 z*~L=L{H$m(7)|l?6h)`%O*@#W{Q-}~U7^sUZ{8Of)zTNZ@KI;A&vi#E9hcT<+?jcY z_%ynVUuQigqV+$Vk=L_+TO*!73G@0!hra9$WKXdTuWIvSysjXj^^-=z0~tHoFFv-! zYXOXD#~n<($~M}s@58ucA6TxP0uQ1|a(&mc%A-^1k+CGN35(W2F$T4eQ;nW_I^hrl7lO z7IZq+KwKc5yCXp^#l8^UD-YqE`Y+JDBg-K}tx+1wVeDv%k~jM{6n?G_7XI|XB8_Pq zWFjk8s7&5aX3j1FUIX2fePxG!uTK^p14w|BhA4V+BElHGae<_NTC4uf_232XM%=k^ z%R$-L6^Z$?XsX-tlJwZ(xU-;&MC`veOoxNkL6OjtSc&<)*@kC7x1+inN3-g?w1d(D z5PLgMq@*Pvzvfsz5KiJ)H?I&|^I`W_pgXq<%7))~q)zF`Kr&{$C8iDW+avNBb$Vu> zaUBY_QU<&r)^VLgxq&5!^U|Re=_bhK^gF#=N#PthUD4xH8ak8}C;gp;%V}#`H`wdn z8~2Jt5+SiNo$o31$55#rissiGLSbJNTNjavR2!>OqD z5iNP}uD+ek#T!#x{(aF$zE&1kRi|+}i-1Oo=8(jIe_Yj4-gW4zoBxe+rA_7CHrzQW zfDVse@S3i4rOZnsAX+|TJM3&klfaT}Q+RzPM;80Tb_{og&L~&OeSY3U_$`sZgnt|b z2sp>jx+(mN&w>G{_)S0|hM73NLm0AZcU3-r8n18Frh z6QjUIC0v?C97|T=-}5cM&`y##hdIVF(Z!ERcSf1#opjSxQ@X(P{6{QC_X#u8A4SuZ$g> zA0R~4`2t2xfdX{BM$M?Xe^7sVuO)xK*o${Hj;5k zbNxqkFX5s4!T72;MKu|}NMKH9;`NYoccAjlYUt!=+u@qg1K{`H-EXKd9=A(8#44Aq z>s+eE3c9;<3nqWKaP?X!xL<;3p6i9Lp^Mj;QGMjeZ{Xw(?(+V#a6vz$wtJ8CuiSo} z*emXi3QJUWm+4K2SsPNLJNh=#E#R{P`G}!)R+il2Yff z#-Livwb2mUjYfSL+wM<>m3CV;2wPf)vS0kCK;YU5cue_9k_27N9N)}qJRRBGAHSv; z5`R`vU5p*y`mcJXvRzE1(Ofef$$r)#pP-}^Wz8TokDj`(T!qUcAswtNjX_&}0BKU4 zgI+K#?0=u+Vof+VrJOCX^7n6MU@gg?WUqPrW7|kqrP=-UU-)XU57;Ocn~^9-HeHo} z1HG);a?V zuyI02_zFCu`6*+sENAKs=$rEsdWc)6tKSGR$E1~s(U-~MYE{=>n%={xKS~d=u;0CV+#{*4ZHi|CKyfT9#R_r0pc)Cm$l|R~rs;xL2KVvakJR`anEL4# zYv*x_l2NIPRW{86)8AOX>Bb3vOHocbP15|RUiL;ryx{0dxnQv^iqKpWx?nUh7V{8q zQQ^Sb|2{AN)l;Kd;futtDiL@OB{B41w~Ht^w>~N)Dtd+#M?ui=MPIl*OGFn1cg!X! zM%#&`b6M6fTFC>DluPdSXb#|Mm!Enj`N^wR75rrI!TubYA5)&HY^ef5RsnBctfumn znZGk~;U3-6x2x9hu6G$~6CzD;6O%UW<$xjksY|cFnl*Qsux3l+2#V@d zSs`}jiYr*B?rbB4FYc()b77jVq;Esf(lOh3I!Vy~C)j_dvnV=58osp6`7BGv#IeZC z#Po*TG%U`<-uv;wT=n90*FTrVKs&HK^GLqs*U=l*8pSRMb(LCkay|6m{uAOMOV3!8 zbrGnTiTAAJDknA}8VjvOOlwt;cvr&=JSCrj13>r`VmfdB<-XBW?hkzXzWPL!h=t+Q zOw)L31fhvvwR$d+58@2?WnceI`_b4?%NGY!$Q+ra#QU~YA!0A@1$%}EBk}pjHt%yb zZ*~LP#*tOn{*Qk1xecaoRkl0h`UZ=zV#zTSoOY5gL6^nN5wg?#&KLJVfKjJaC^;s z1_JT(u?-zXTGVActz?my3=n|U2c5rr$5hSzqNw09I$4oBHpwQUna5e=PW;6e(s}+p zDZs?9Rkro$Lr`c7y2)*TOSp~%X?b9K204O}sIGcx)UB9%NUK2H*^Z8N=!&fJ;+YxU zeQ&d@D=Is|pd3;v-P>n`IdCe zFXjY(&-`&46`VriNO?VXua4YD(>rax_qTK;=kOJ1KxKA0tNmDx=ElYj=NU7}Tavh~ zEo@l(^kX^W;o|W7%a9csSj>oRxxcG!=wcyHq@t9pB}^xlo4_OFpJ3(%c}f~L1=QkE zx_puh5*I-oEmJLw|23Kw4p)iC7RUg!))je+mDIc5lb6k7r`@EbC&E*E8a>)xVWV8@fGW}3Y)G+G0NA8|e|tGZ?m+?Wtf zPdN87aPq5MLfF%KwXGY?{+ca{>K~%T@#)Rjt9rM?;hKCaW1 zzBRSE6)V6}l$IZE#SshSnyP6auq(tx9z9(>Rr#((bvtV}WMA1STbmYWND!1_m ztDcWNO7mJjJZM0Uo3aY5R*L6~WM$dG-JqhAmHifQnddFRms!Kr&V*s1HQ9MU>*gc4 z=-ZjMGEicn5;;Xa)^XU+LU@CBbtcah7-S`58W zjv|*6GN_Gc+1U6Mk?$aO=C1Fr!3;(>hH3qK1ClPFe@xk{@p6)aSxMzK;5VY$4*dZ7 z*>V?kP@wn@ns!*Y{ldB~)+!yoA`a5i1AqAhsbcu^?&dCxyv7>7#}j*8rkvmuxuD}V zUDz`Lld^n;83lz+yklI&QMOd7mA_WY_gi~I&Zu4S+keIPI3??K;w2Z>tNvrFSfbEy zrBaHw#I8ABf>JQ&R@hVK-K-!8!4ufo!7!~~vQugq@z`a|>8a21v)Lj@!<++txGG~e_^7~Jy&-jo>%u;8apJ+ z!274t-F?&p8waNFiQ;jAhNkkA8fc@--qb=^Tw#3q*Qu*^zSD*dmGw^W_0Pluqtr3c za>bmF&WZBIwDlf3@^xGh2*(p4;MG1f(HPwYYGQc1>JrENw-e~hN8M!7zswr6%U4^P z^Bxzqa}CcbVwZJ1Fy0xG2&dyz-Lkc>r|O^Vuv{)FJ5_K@3As2PGMtyshjM9t3p1?Y zZJxR8DzQHPe5Ck}%E?{ZPGE9}ysmeMJfm4WTB_#=`zt@4hW+-Ysr91IVl6lIoZ1wC zjfEjf&1sc9W8nf}#OQU+r|5xJtek@WF+3kG_zMIyKTtH8 z2&3Pzr|oZHfX(?)i^nWUZWj>8b+xG`Oj!R@ z#;ehf{M%O(P7R;?jhP8gAN#fW#5mjk zxTx!**Y}b{WF9;S!d-5jt;f?bl5L9abCyaPN?bGYJaK)+qY3^D)|)CahKO1Z+UB5fo??VD|$ zp`Mk_M2KGAias4ypeQ@E1ahLJ@37WgX6^z!k#$Bm{bb zOZxYQ9H^g;)SKQ9BxbZLe`HU)l+5o-8#y1JlBHfXx00IIB9fA%_*)=*4`ZY<@D%6^ zzn`DAJ~T)JkKa?Q#3|z!>FS3dFh6HpwZv*&@0ib1i6g8j4+K{~yQ>HN+*r+YL17r- zUWr1?%6mi{Jo*GJpH(oiso;Hy61B!UJ_bfY9>6!eDZx$e*#R-}#p@fO6{N)C_ThuX&vaHKRW6_OxFa z0i`=G(nr_+^1rCPEWXUaX)H{C-@@I#N5U!|BN_0~NY8w;(2UQe8A`(U9;Wzy%u zPDqNn#Q?xUr7`YWVztUXr}DKoGJ)?Ss_Jvo$VzR7&o{CzY^{NT@1Lmwu}x0gjGP8P?Jps%o%nrGmyy;Tw?Ai1bR|l02qCKqYM!Q*RYP z$5FFOFK|07AdMQDlm9X$Pl>35JH60R%yEr83x|RaxZ^8CIVy0~DW4`ofr0t+7tPp-5 z^y{k!fI$4~(wqPJsry6-6MCp9cit{$hd`Updgj0JTgc)W7kr`Iep^wYsG2$NiK#qC zL0E0^G_cp$elSG`HmPp^!g9FoxGMc*`QG_$1KJLxWAu-m=$xG;)M4+(U3Z|?JjSy= zt7fFw;?W-w@lvn-#aMmL6)fa#t)1HmLA27e*!9Q~?{`89g*eq~KQMyVK2~YrL-Yw> z_!b|lyZ!mv;ZeG_sQG0oq*0VB<|~BPuUEb(iZdN%(+`^N-I#o)Hc{&f_*rVH^&Rsw z=0_<(hf;BswXNPI`XYIem>J1aJ+Z56tWb{8!KG>SG;ufe6T?qRJjT3!FT_}g@7{p~ ziK-#yq0%-&xx+i~;fm^N@3RosiAlR?w^g6p0??Hsf?t$kmvt%D5JkUY9SIT7zlm_M zIxa7bD9p&CJVh zOW9jEWtUC2N5HIk{-=rDR=%XsJe|H2xe)u~?EfF=|EVxgbyt{Iy+@W)>Du7@?|(6& zjS@kjwD=1ZQf4S?S+8K{Rr5<}>XAykgAB`G|+9 zUi+-K#Q9rF2+_cCoqg80j;sv1P2eQ^+%?Tt&o5!Uw;<_CZ43=8zPo1C?@)%)*r{8YxqRVBNlCd7FxJKX3}cYY~FjTu!MBx zksoNhPU!|bo1Y{+h?9^QFxP~lMd z@`u_*(89___XjLZ7hpf>vjf;U45<)HgshHf;omV1L-%6lWS!YCz71#kqd@L%g+a#cG3sn<*Fb7Nd2q7NK;G_?Pxn zoCvLs=kFVPx28j`QD%ZPUHwZEeyWc?pgi;{zVd@5yR$RX=ykc?{A%t3lr;V*&eSJ& z5Iu%;dOGl_{a|K}IPZ|U1_&wIP_&&w8l+Ji!tcd@J}S{rme)`ex=W6(UeU4`2=TMq zzd2U~-;)r}IF$z88h_t7oPRd(wnRP3==C#Ob10BGHe5?RPN9~MWVI{mSvB68cKJz& zS{B1m(cmrdPxAXDMz99ZHVmH;Yj3t`x4VVK=)@Zx((!s_J|W=cHfQF3I(N)8|DU?x zp%yQCC)TlbQ;Ei5DfQfGe3sH29f#F%d&8jhqw$- zp8m_ogvA2ik9S#Z#fUHz_qe#_tJ&mH(|LBtJ*=@TyhFxr+k~S%CHzhf&I>6OYhrw+ zMTkeH_zAN4SeuDTb0?)$Y;P!h?h|1$I?8 z_&1s}WLex@)&w}B1GU-vvhHkX1d(KpvwV@5OLwC@Y`B`&zahn?d=HUPSA{T&h$rL; zHE{Y^43aFCP+U(YC0eAIX(a_y@rv#msjkO0XO_P3%iczHZgvHgnYxhv-E_P zCJyg!FW#hLkFw5eem-+RpR+tJP1ZQ&#y~O@l~l>y={(q71#cTRziFFgXa_$SlE15l2i2ppx5} zHqvCCruSOZf9=rn|M^v_tXQ4CcpnzSmaXu|_okFZQ@xHn|KxtF;huiX9kx5=^TRSQXUD?;O~Logi+w zmRB#WBNoQQ+AszzgDrh*ItzOP#zqzN&hHu~kMjmY^;J=EM?$>2y8 z4my=1mC&Hrq!TN`N6!nMmUAi%lD}Y-VAdLY30{8-y%ZbP<@?CN3>kSw{Xx)3X`5*F z=2V6uXhCwsFE^zZTMy}}Y zP3y0&I^9@hW_Tgq-2j@z5;f0jDQ_5IdlC^%Wl%Sk)Vr87RMxAGj&nzw1roVm3P_lF{ zbPR?ybrb>meEtmPE50acQ=W<%n!bm9Vq}ata(e0kp#HQwuF`;RyL39#q?@Wdg1t2o zfb41zI3D-aBrcJOlNLwH=J+GY-k1ke^uh-s`?73PjsM=z88)2?0@VSwuJ6@^bwP!b zyaj!2CS0g-t|TB2Mx8e|_Rou{``M}6nP(3bpyB3+Y|J-}ew#H_H9&%4xgz_M?+Dn# zonnS6$2#O>)GxQ0ewXQQBfNf`gS15!w15#Dl z6lM!K93x&O&=PLMAt3-S+IRg#m~*(U0>e6R(Esm?)*~#Wu77l7=tSeyJt72j5m0pN zAue_SHpup56PC8jC3An<7(Y=I3;w<0QKvg4`$}<{u#~|!gpnpwa9?Bt{dh4;Ahx4ed6g9<%emKTF+2Tg*K{mxMIoXMH$$Z&m> z7Fcwmym~_#nEP>VTwBF%4x#tL~fPYd0YI_RvNdh7n4*fMvhgH}nD> zsZ*Lx+AQ!*eLKB>gl%~wg|J*Q7W1GImrBC5m9vhesm*jKakgE&AJ-i?5K#zQIVomB zsBV`E2sw3=Zgva*5ckMlyW%A#l7a-f)CN+a!eHRWNmA6!dw$^YkK3lecdx}{jTM86}2p?02HMA!!TkNmaNwi}KRV`^T|UYD^F z)l+M{Z3;^}l2*a$x@`rpEl{2&DK#mv$d!?1&_8L&qDgYcGv42K^4q$!+}Dc<%6dogA>%h#~1Ei0_HR_Ds3v}5{F&? zy>YW_w(OCZv7j_|&-R*Pm*h)q7vB{B_eREl6%_TXGrX$h6WeL~-dt{t8D&*C#r7=4 zzUkjTi9dxDFlm3W4wUyOz71C#0{7U ziVHx;!GCR3=AOSbQzH=9`?8Ou)lNK7qZR2l3P=u$8qpVBB_yw4-`5FY-f?>D9C>pb zz0?${9QLrtZO4zb{HEaDNOx1;pahpQnifNlEQ`rs)Yi{R3s{ z;7{t0XyPQn?`Fgr4<g>&WbgnB~Fi7f8{u=zB ziwD9ak_k(vF%IL8qw}@o z38xqw(nB|{;Z1CBlpOT27a`sxhiH2IX_~1*R2A@4;GF;!AXQGo4NZyS!7SY1M!)Y~B#oZ&^lIFq7@Nr_ zs18)s`DHVZK_+w<&8_I_37@vIP(+%ho*dfGSw0^CM*Gp$pG~^L64T%jmdl@vM+EJe zWlEY={fix;`0J6lUw0zX$JUY28O*=WbX|8%uG1C@IqD~C1foOpO0wEW`-Ehr1QWlE&3Cz_P}F) z;54AK;9$q`KFn~KPABfBcws3&TLeX;O&ta|`6AxaFhN(RLDs8*D|aYFX`pTRl%4G3 z+S9@U-oDo^c!S|s)9x4(eRA?R5pDsK1Bnk+`+e9_m+_U2TfZ$=&)FFY6LMzT)cg7=C3b9?{EBz8pBE*-`iBz#f)CS z@CW@qj?oBrVH$AWQxxK+vsS8c}|gP zClZ?Z=RaV7a$QhcBAdomV16{*i1q%-`|D+Y0RN?Gao7J!W`l#)qr1q#2vvKVvF@7>;C>H{BK=wTvB6(Vt@1pB*)B*>0exNzP^=-nQ=wEw|E9M|3D^? zBBlSJX;bDcYkx30hdp%+m3!Q`kbLCTmDaJ=LD0TQ= zsHk2f>~{t8S!snb0?&UY9GA6sbCg#{n_v2rjY9MmS4S#k!pDqWR8dHhq60#$TIv*J z+YrOJBwq)>NNpKU+&mpq-Xca%2|y>Gjc}e-vS9dG`mtv+FJe_qS z`dNB%gg?3Y)F6pl0p80Pf@zvdEs_|gDzx9VeYoC5Us~FyW=V;{72=tU$>Oqp;oct( zJ~YWnlymLmnhL>JmA?HTFkG}DVl%>j7g@L9_>julDslOsiK~@1Ia=hb$SkvDI{f~p zzRy;{c%8;-6?VI|Gp{|Ix5`=cq|AynCsU_#rVnL@lCj7zU)b)mV6<0V=DeZlmumm0 zeivv~yiyYh0(I6{>T24)P)3hmisE-TrOPzeYf^4|pY2OtV)HViSF4F=_*Hla_PG|8 z-Jk`qN2Z6lIN{hQmNH4^$oXx(x>L}+Ce1fZU(w(5C_Af^51l~W?nU(|T(`Zp=t;&` z32?VkO|y8J#JH^nTUNb}kAL}8Ra#4>BcR-5EN#IUz2&4!c_E$lvxE?)K5-cIc;niQ zy8|IH!a?pg#sVGM-ubl!gtkhJ-m1}(Pt>grsfv0`Tk5>U?`HJdqcm%=Edk`GR2|dp zKqxZrnd~JJR;=|nmm#iCCHG9g!i4<@jQIqiefDK3+2b#!Q%v?xm88RUbJgJ~6N|}t z_n@*uO>+LW^+A=LVi*R)ipCMe0@+yfx}FYhc(fJ>CQ$aF&e}#Qm?a1BG zOv(5Du7t*BF{-BQUl{^;?l*2%%h$@%8dQ?^F&{zI7t=~Ykkl$4OT3l6u0~_kCx9_z zjOAXGz2@rv@AbH4^#fF>!@JtMB{oE5bR!9Hw-GGh$6^n7>bH8 z0g=ujNSAbjgmj2>xGB;pAfR-Q967pcbTdk%L3;FnF@N{_`)m8`-p{#b=X36R-uFDu zd7e3BYDwU#n~O|cEdQCYPAJcM_>)M^wAvIqXEkA@+Ig1)2%Th{y0P=tcH8UGMY^+a ziSS19ib|=ii9HJ})j)LWKIss<|F`fzx834I9iGMBo-h>q<(4j7@{g$IR))n?VczrC zvdRg;$rT&Eg%q%^DGNQ@d$P+s{?NNseRhJnlV{Bwb2iHZbTxz#4KIR4l;Sia< znzXAZq0ZN|4Rx2(3=vvl1;wN5lA&0tgRy<}ihRP5P$|WpT^8%s2q8!lFCcv>fcbpi z98dIo{;YO1=4$csVcAHrtUP}~GsPs&P~5osZq8LX&gdgIzw#6KW;mb@eO?Xw;;T2< zO3M8<8T1{XzboU5x9HRdv#V1SXD;mIWz3D3wCh1qBFDU1aipM#*r*fsmqaT$FBQ54 zvw60ZR-X3eCla~7ixZptaLvIX@x@J07-TSF?y(QJ(T#D?Y@P z??qQyGg@(w$$vzL(aCdVfBs)_!|=G2pAR4x$ACg~bIT%NdT(^IY&H)7%2ieGJG`w| z2lg|=DT2@MyJA=6zP1KGyNN9;7k}nH@PVq;IpNTyW5WvjJZ=OKc}1@$UkP>O(^iK_ zPF;RL*>pL^?v|VNJ!_e5ungaPTAjGBYs2&)q4BBmnbo|8;_JE0W3jQ%_v>}n86C=Y zas5zGVNO$$Qr8L6`PgZ|*>tKnd+}M3qX7D|{Id^kHt(uF)L4U(5PY48#&eLR_LQ4* zZ`w|L?PQ*jXHesKJhOniMm;Ug zM?C@7I%O;$q z6bZ%@L~B-OXK)RDxW{Ptp(#%31H4fK?qF(Q!Vm9MIp*2Rpp1z}X}4Z)2ekU4bj-^y zBD~^c+{ODulTAccLfDQ**q;hi-NCc*p>>NuQ4@i;&9zOlZ2 z#+>BKw(=YGMUo~Ot}!xkwPP*^dHDcuOE$Qe5*-veF~y)GU`xmr1NdzVC6 zK^b#XMcfLwMx9W1T4Hrwtok>?9u1Yqwx_zUc@@Tpavly%EO@eId?LM`$E$uWuQXNe zb?o05QzxqTDJ>z|pg39c;?;a%L?I~n(q%5gZCV+A>o({LYRKQnSkD;S?N;wDwEk?| zJ*)P1eoK{~9d)NkR6>MwYIqms9)i^!3G=)&lDM*Z#XATX28Ait_$};v-)@#R(f8rG zkM_;8$=!;duf`Q=cjQ3kN}v`;ImOBH$&{F-bL6ugc|^U!Nm(@9CJfZXXD^RG0>qq* z=W3Sj?m~oat&OCnR0>S}LeLcti(xD(42VGRq5dU|Vk&0TQ;iom1o~CE-kxnb+=43G zaxSlEu>bzsyqSW=+fqeCjX1mAKPcl~St~nQH-Ow(z$zjvko`bZSDWkDRd2Q#a>U68 zRV%dc_@t`;8^js@P`Hr(ieW3BFVa~jb{Ictt)4=?1yG&J@E%#E)goq(lynzb67WEb zZ8v4-!qOUP0C3Vc04E7t-}Nr|>g={7y?YT`Kia!JYRqB@Ro5d~Os@Q-MdK@(Nj;M+Z{L-{ zFk@Gv*IVT!`8sCFskuKd3^vC8Y-<=Nq?c_Sn$Rcf{k4ZbwDme$GJQZ?Jm>z`3G-T{ zTd9(kBg`Xz!5*nOjdQT+7%A8i8}&psz8SL#n=@~-U!?I&F_FekPR3Jqsrq;o{=NYZ zoOVF;E@hpRIGDWZ*_Wr^G7|k9)O*!anehG5j$rII_dU-d!3w(uH1aX5*NY9pmS5p_jIcnCuYw^zpxoc8f1Vnm?9^ zr!?58aC!+oTK-2A!TsoQ@F-*ul_qm_W2@xgD|bskGp9P0cTX?(Ge*B_$awXAzh$3q zLx(C#6X|(D9>@JqhDSF28s6bkvB^?Wd);m>|M>z26j7KdT|GaWSWYkBb?=RLaU|Bi zw_HXn%RWSAgDox8iUa|2q+r?npR{6?ab|vL!I< z12eBFv)iFf55rn8=K1v)G*4ux^_#09y4=jKU_}BSY|?JTZUXZb>08&YyhGigh$_0ls(GP2PE8uHlJ#( z`L9p{$S|Rhr-aF-EtC7z-?u~dcuxeXEq^04^p{S#1hT8C*bVb?^p%dRclM{<3XH_k z(t@n}K3k7P__+qFRjY zM&)F`A}-G<$~;xUjUZx7$@^r8%yM#1VcXHTub^_YbLzr>A&^E$mpoUSD)Vu9-K+0j z@SZ9@cmwt0hXz%EFSAfke*G*fTnV+j6Hr@4dRkE3Sod=58@YAiTz{+yAL39iZ7|Pr zG`BYFWV5p`!STb*W*Z!fZtZnWHTxx?n30Fwuo!)PITghL8@Q74hRLqRn^FFR;&hcS zXRh#-2tj3`IrN#^72oe)P3-n)f)7yTSM4(LelxoV>pPe93fLRDerN=@qK7rf08 zB2irK_t-{Gx5y~ew{o}tmv2d24v!K>On;nbL4<<3W=D>tr!@_VgCtBRVdV{EfPi}h zB8sd0R6L>@X}-MEEYWd$qoL7-VnNl`It>l->iXWC zDg~SpAG)1a%F98HaLF_Ahise6H8z*{>XZm-82+>-LRmeJrKUA=?w#RO;OA1>lyMmE z5`@Glr>uwUR!XW`GKr_7-xlZk{x(cZk!X#7NQCi|E`Cn9gH&5pZKxQ7SL@X0VvWxk z@;125*7L;qb4o(U-?Dte8+_u~?M|!h*UiW7dfuaweZ8JkYKyCtdnosz7GUp-b2+i8 zb;yB67`P3ox8-vrq!wp$Erf$PBj=)`3SJ9?h8H1S7g}vd2ct>y&WtXgys0tu6VHxI?E$~ypo_a`)F&8I`4pn*VLu(0 zA0oJ<)S~eSIV~_oC%bFOVX*7kAiJ=b4zGR#Indl`AQx@&t|nqQ1yj%3t)K<%K|tcn zKg09*M^!DgC=G2*L!~?;ZD5lY0*)@3mo@m-@rSoX#3(vp=J&Q1pO2D5_UO~(J0@!q zb)_7d;YN?>?cDmfPblbSUPUWMZj>3~9E^D{&Ocd^%}shJ@1#kwC$8O`LzOunBnm{+ z{S1)c=uh=3munP>u;uss@y3hJX3J!~Ht1x%`gm{ko~dUaoTiwmXIwR>473BC({nR$ z48W+R1--R6D8E>g`5uQ+7-1%nN2s&UjA$r=5H^me$-x}Rxc$+BJ{eA=GE&c z;SH{)QFQXw(LVj^>Z6xb5-bN@Nu0^+a9~1dlppAOs0{qo{&iRX3o34w+CaCI=SnX4~y-Y2B%j6YjbhwtT58L=Y?=A}E z;Ia}ihB_IDxmuehF|-3cfGvI{N!W{wudH;_J*B90Tzapgv0O$awecqJw;Q3a0Lc#r zDK*|s0y0SdX-18D%)*%UUvWcD~y7mTyZMa@wN{L1H5Zc-m__QpRwg#Mt%)0(k2oI>p5Nu9dvYMDC6^7Po#&7yN#qd~4=X=lb4~ zSERd_(5uc@JbMP#bp5ctD-lRqoj8f2BIozfRdKx!AIJw2+qfa2$1BG=r1s~`>PzZT z0yGQjgF=3~n6Fnaa7A0>m0@o#^=|P_zRwxqQVXYdepavi1C6~k@0ls#V6iD^#^f)- zpZdUIOq~8o5LY>29PnskZ(++!+B@m&=^;-j;mKdg$EalemcC)ytdUax9;f^dWGCq% z6CZfpj~c2u<%TlG(7PTYP1t8mu|U523?Pk>=KLc<=QEDC&|Js)--9o?+>){MzY6am z`P`W|vLTD|M%trvPQ>%FE9xn=FCbE&qn)SHygM-KTs$(54%A%Ro z)5^*mtM~7xk3#?^1I%`}<>8JVw_JDak5z=Mz%}d6QORG6n=YUWn=3Oy$EDex!Lrx^ ziNkSuRddwJcL=Dzyq%s7&GO@&opyv(^}1>|cykMz&n07qjZTq>Bx!lmks%X)xhN}o zmgGPeuO3^mXEw_I1%N@r;pBt|ndAJ5bm#{P6ptGWD>=foVC|c;ym_mj^bnT z+<6Iu1p11(fLV$qNW`O_@}#;!S1}8(VZRw zjvCPu)d+C4+UFd!u+WUe!`sC^d1+73N+~{cZHCKBIQVK!{Gij5|D&gWh2CDLi}Ph| z%e-^2dN1}~uL#~jyUfRq|CWOu)l6d0R$j=rQ~$R0xhUf+ex>!Bznhok+3j&>!E4Ua zvJNP-0gQm@@R0nXcakp7ybQXK%K)!&IEm9PduD7@MDmR4v904`l zx?pqAmnn`7RT(TRHSQT-O<#q7f(^U^T~oIP_(K>Iqx+$~T3Ohn;6AopTPU&dPE>^} z&ccc1K?nE?rT>jXiTNPgXUJj#y^C%`AZ<{4*}@%v_DmTLsFiwusp9eiy?mc@Yr-ex zsKY8xREfKM;Kz>(#&{-hrE|k7+0z@f{Loo>y^0@HOw;>M4WEDi2DUcq&lG)-Xa)Ye zr0WuX7#zDCNp(fKx;MRB?^h`I*!{h0B<6GLu1&Gb;3q}1YWX)4)T9r~Ko(p=>hAxD zG)a_K1}zq5P@Ue;btp3z)UwpVyhbti2Fj_`LWDePJvQaAY}!0`rfeZn8$SEdrB`XK zu$ykWI!f=slI|h&WzKbYkKo&G^@2j^auD)xBX8o{Y1w3cRTyJX;cE5$_ARAEcvNCj z&sg8b?=deeUz$JWDGa;P3YM<<_=* zDKB$eG<^oHLV?Ac1;*|E5mlrG+&@ZPY1y3>8;E4mxo}$7xy`3&{S$FEsB0pBK@m!O zraA^&j&d{d%&58@mEV2*Hov7-aH1-tovv_@YPenDNmZg~t?L!1!`ql203Xzb`G(g; z=`>ggV>UbAk%+!A#CHNZrJCy_nv0M^{jB{tpu98H7{Vpld$BRGldlpRy7#n8tus|WmaqM;;6r_pw}+(J*Q#}F<+yN7n;!F`5+ zk+3yAnFIZR$>20k?5Zjj(Y6Nfm4sfcXPh%r=HW7YLO%QbZON|7~|shV)96wodD7cq*_v_ zxqCtOmksB(d^^Ur;th`Mv}<&Yqc>ZEOw)BdL7}VFVPN#|3}?42K!5cFCLM&O4SEz) zfmnmgl7>9z33vPh=HxYyTNu#KzhqI8WWg!6xKyyOamarWhY0O=CzJb;o!Xu#`jI7F zG5^5ws=YhrKN~_C=^ydnr10Dbmzn8kQQhmLK)etY$Vhf;%VcYGp}yPR;IC@=K1kqI zd0L8%gh6XSiui~s*7WOj4pX6X=C>`=e?(4Mb_&@3$MjOXrf`@1q1(AXwIZic^3LmQ zbIrx_bcn$QrqX%k3^_c1nqik*D!JaMujSKMSjgrLRI z(3O7RU6ihTGXOH`T$*inhe+3GhGY|zTGWTs^=jWLM29ka??@Xo96^vYQl22zS{lpoA!@a#5Za@-0LgvjZV@+N@0u)zw z-Q$@o8Z5pN_BjSz9$#hW^LPGAT|6Vh@@a8@NqGe+q7iy2se?dfz=V%>a%LV49@KZH z-QiMX2@E`PsCI&A!eSX*&YqXfaZb$ILhY|oL-1^4Rh9yL-oC6U)YF;`EOoxp_H9@~ zW6UY(QI^K&$ecqxx_cS-eJwfTicFBn)g;iAc^c567x>`MhpYuY^F*H%YsrK%kUL9_ zDEYU56bU^Y6+JB!ea@;cu;?%fWlIz1+q{V(Pm;ws)Agy#?KoV{>ycCb94oRr zj`kfmbr5a_|2PnkH=QIQuVyWxmNUpCsEJ@McP1dg=;5;qAI?# z-c^h7OpRgAaR9A!?e@C6TyDvGph3`^a0a9dHl6AAUwhpf#)gBh5LE=cseOQ+(zK zkTPF$?R_Rus+cP?H=PByCka0HSxi_Vchu(Wf{EbPu-(;tWiEs z0a+|AS3flBeY_d&AMCz;WYMmG7T8g&bW~iZ%9SqLZmy3JkvF(-R37?amjo@rOk? z7fou*)_&whH-J{@~A;IiBq?>X6{20EB=SZGT6P;)V&g&aa_i4WZZM}qnzXluJh9xAG za|@@3v^ijF-}%G#%Q~^qTj;@zyw896?~p1?9~HB`4dnR^P7sm;d{4l)yRqh}>xm^X zUzdj!%IYWHs$7rh>B-jgkd?Y!F~6-S!F>XMeHVM|YDno%RL;I3aMtk`>psVUmJK`s zDSb6FkrazL^BrsJvu~!@J(;hX zW>JflG@4h@0SIL}OmZf)wD;oA;!^sLz>mVE>NstNw{PE*op;wpek747wdhfG|Ewo% z`dmH$-cd5~Xukko*oU20XSxMW@GGy5aQ?^%t?Ej1V1Jw8!qz>pyb$+^&8)u!y`*dm zXvv!OHzy5*A_g=<15T({Lq=hLKALR35I4j4F0RX@WoRg9bYmJTVi9; z#~0CzE-^pU(`Xu}8G3xom%_j=yiB?-eBuP{WghXs!e!Q zZx5%4P7oWb-8&w0Rj1eFHlU)q5&mevf|~J>b)?*c1>08@RS{8Q!v*1d^NEc&MoEl0 zKO0A%q|6m>UFyZrOK&s^ll6VaYf#(g)o56%S%2in&dl(6gF$Uf#7XCAGwDec=zo_L zq0H7=kDNSu>;`grVijR)Xs!z={LDO0vRnDN+ufkGE~gom^xFE3CV~_sM)KNCg^i+{ zsUTv@;%*W1Z;SloVxfgRA*&~Bn3P!McPywO!COIJwsd9#Cbk2+zI9vVx#q$Fa_Z00 zwf2qbrWJWm46}*cNm=JsAGn0(8`I=faxwDHDZ2Q z9M5c=vuP>ok)A~~XM?1u_WZf4fe0%EZAX_e3@Y|Zs+r%@p8R6J%(?T;pvHQtM!45W zp}rTnLeiylr{tvO-32Ho^~JnG{)7yvArNp3OLX0+A8xRoAXu60Mc*lGX&2cVJbo!k zrqY`C!Zq5KJ5yW9Jt`uav}XH!!sjB_Dv(@7R4nT+ z(i_=x>H?HHOcPyA76Eb06@^}}#K)cOrQI+QGghl}2{}KL1Nlxt{GZf(F(=z!#x{8R zeKo1?V{&8XnI?%xe8UupIGx1Z3vGX=*b$TFNqaJPvH~^&Sx-_0O@DiTAIZWrUb>$j943cnFbLH>VEzdst&z@Of=|ShXTi2%f z@aJhflp4zP7CF_yPbPVZ%`;@f$O()b-qYAj0VYYyp6Zx zO3ttuIM662vY#Q-4dHvjPfCZ_GuLD_(@;`ivaZ^wy!EFxDtMFmRQKuK0zM6uhuF^DU z!+s;8y(q=D+n6S?q8|2`{DR7ooi~GfPgP z1X~eC+Tl%d(R0PM(eAbv-Nx>6A-cLU8q3|TasEo96WPgy0OBn##lD}M{!xOO22DNj zm`RT)nMJcpPQ7%2RVwFN&@xGBuDIEFrUnf5Pf2yNLRVc+UCh zv2tj)t4+7y(77zXLx6gtTAQZ~`N!&AkN}!-C)jL(!Dde2(Vy?kNWnAoeElg}w+g42I$=Y(GHfeW*^r=8%3v=)F-;E>!rGr0yN3*Tw58vo`u~3Y= z?5e;EZ2k}V;U5!A*VIRrxyJ9M#l3RUPcpEnwo~uD&EA-bHvl$dF%-axCpzp_krS52 z9#w7sjqpg~+3;kk92WC+R9?mGf%j($N{s8zC$|n&gLze4C}(*~!MhSn4`_;O!T`*Y zY>e}e-o#@^MhosRC%p2*79*2A`AwxteYvDL`vzs95zCJlSVpHVremaX9F^9vdZSQ< z4D4)8XP76jFazw3n~=`3-u$Y>wKwU%QlrHXHO$~Q^!^`FanS<;hs!B@_{q)tt=Mqy znQ!+r)LmKhL~)x0gXFGXFt*J#XULqD*I&-Y;q6Ad$%KG8mgk5SbEnLyEQ&5UI?pn_ zW1EF;3W0v4n`Ukagss0jBP!;yQ?EO{f%>lKG%S7?9k1gozK@D<$Q9ZA(+3R_{e8`AajElKtog z@^*fW{8ATnt`<3l_zk=EY@6H7EE`r<8m1maMdnVR62UthXyxgKyAV&Ph&-3p%iShk z1ZTdpLYs-0vDs=N4cdflJ+6U2h5C?JX;^DQdetVY?>fSi^cSaDxx>)8X~X12J+-1_ z=U0P*D%o2NsXr%Biz{&{XZPKN^^nCQ*1$>-+QU$|=a6jYz?KrDiV0oc*2uG$KR~%> z@48c?(?VA&=B>$MhXtcNI}fx-Lk3)*-*<=bv2d&ovU}PUEsT|xR*b+q1RU^6oEpnR zPbt!Qjbu}VpQji9o^)HxMDgl%nz9gH$giuhP-6Smq@bJ_n>XFOq)^|&`RvDq)U+Lv z4B(?Sm^PxLh3>yEI-hX_Uahy5Y#SacMK4iq+|UQ!_Cj=DU^;dus&q{9ESx?zD82a& zF8Q-YeZonE^u}Ly2`M98tW3qm#e#%(9i!1B!A;O=%hrW;ET^jNq*s-rEu5v2a2A8E9Nmt z;aV>Y`w6qP8j(6)Q-C-Zs3@>D{I!FLs;g_=M5r~Bu8MD7@~RsbaQ0&&o2j?@n$Yd{ z`m#FKzf|*!z7^(?cf1n(=ioP%Q$uz7zvb`&ZvbQN(P0=3QBC9ZipXP4l>h64l){~N zUT;Pm7mXZ3I^U%`Kf!l9$vJ`qy{vseAEzG1_6~iF^EbpMmkqhnZg9r?TVMm=16|Oc z$RL_aGR#=|x2Kg4M708AqE&rl(LEV=ZvGYb za~icKAT~5|yUAj8wd|W~P<;sD5|l0}h%+GUO|lVgajpl)adMnX%=AV~PTu#RWF#(2 zKak|deI)1lZFWA2^mU26q?EhoMdRn+QSmPB_n20a+xD$c)`9YZ+;kG*-;mWl*z5Ux z_(Tr+s-xmLZr-{nr5E_u?22a`(&|}fRqB*~JF-n0Yn|!J0+Fq@5D5IjSADp0{W_oj z%KMx#2pVO%`v)U%+0+QmL=HsLX)7#&e+5I1!(HZSrOW>jnXn2MQ4tpZ+?kkubR0gm zO!ZeFB^L8K-t{=dckl+V-(E4bztL4^?z{1ak z#lBhBatMXiy6eG2+lSYu1+ZEB;=zcE=_Or&RZ3!kja=la)`HzxTsD>kI8Qh3wMQth z|08N)#B-*;!ZZ}LoEWxi1E-u^_MutzC}k-)2C=FA;gBWC`DfzW5lm~UUm5O2e(lHE z0FE|*^;`U4=WP2BmPg2Y4m6_rX*i$OPFs1yi5yv7{m$kj>>trxWN_Mtm}8K%I@s2z z=9eYD8!j+(rG;0YjCP7Fk_;|rMXN`imxkKJ!f)yiFn$feH_OHLx>D?d$0z*XVs~Y( z9pOKV#{HGB6G-vD|A-E<%!X82#Bo;zJo_$baM9wkgE{G)pQpOU)BIKbq)G*U#QDPf z-Vzcix+o|tA>w8~OHwjIMQPyE-T#O_lM!5ul&=er@X4KSI9Uwg;m91)>?iJfOh;v+ z-xsr^)6YF)%S>8V>;Ua7D$7={B)gP;8GSis7Wta6Sl%NdhcAAxN+4r*pkM?am36*& zzo>OKVhx)som6t&2yaOG4s1i~qi`z!h_ITu3{4z!Q zvQ_51z!XU2)r7E~-}RQrTd_Ke6ztE1fDyIAs4Qajc2Jo3R37ZOnc-QovzK2{iLa_!J#KzzS;0<3S#d8=D$hMZbOg!fxF}S-WB3or)#mx60 z-9}4zimm(pPWg!b4{vm}x|8`ge~rQf@I$}h?P@PQ9&kkfkZh&H4fs6Z-q`ALqPV7V z#bCO4-zBuchx^%oe56aDi-8$$f_xeyVFFMcD6anM z*ui+bC2yhUGl6jqg1ipM%?Ps-xxd$cvC-N*a_ zHwbfKG$B~zfh+zIbtksP_Xyb>Qs<#(O;fH1@I&V9!U^-27vND~WcR$5-LGn6UH2i8 zlW@l!rPy5>4s@@c`~AI^Gsv2H57(k04i#wVTi*FPeLXXtuE6WFRpJ6#Oh z>A}0}ikJFH(R<$KNJF!E%|C33sSJ2b#dVUu>>a9s;@&I2M@D1#+mozpgH7yAdW?q@ zJgxf^)+co=WxB8ciurTJ2}bv>8i*zjj*}wLQOmJh>@)RryCD2_+%pz8(o4)3 z4B_mSOPj9SA>-|q_M)XQIPaMR8V_=77aVGuRg$l^!9zp<6c(9&OT8HZY_-DX@sN!a zvs<|Lf=??$W6ovm%L4JtcQ2}OG5A}0%<5F-JN6&`5f!u!iLu~YvHI#Ev3P`1-9MsM zE&_=9|38$SyBB9%9W5@KveU#a>DM`NKl(>R1oD)CwnZI9ae}Xw;6ZLY)Ew}UT>CmQ z@bO`K4m%#S+ICDXp9k#nEG7cscsPYX*0<~z{djy|{<_iyiv+zCzw6L9o(07ctLrzB zsWWIp&)Z72afQ$FM3uVBC<+98RR0l~v3VX4$b!s$0Hu~W4`Au4aUy_v7lCC&&c!)q z|0NQ75Dc_FLqoWGGwL-D02Nt4Z*SgI{SQ{^XwojMt>TX%9yR1(@hTuJm~v0f3ksg! zfi4IPn87fuR4)IBCQEiQ&Q$#>e|t;*CgKND;DcBV#X_74l2$zu-X^PumrXSs=bws# zPC5!sOWM?3XTn}B!VA)^z%u$m&_|M@rfFl_R4?!j+wvKm6X1o}Yy$Nc+nu%=vwl=e z_{_+fEcfkBFG;x+t?MiobP7$?lbiF&S!Wb;uMfVi+@)!0i`!9+S=W~WQT;Z^sAqvB zn?76hs>jsHkTrmr!Njm}Js<$FX@_!tz(KDR2l;?CBQm=M^>g#bN*5;nY8Edwt(Y(gBBDnM;qVjzU79?gvY=4`QRjgiYU{MLbF`7Wh1 zV{-dp4?v<>UoKj1qPn_s^ay)%)&BZ`rgt&Q2!*3P^pt#l$k)1R+VtIi`?a&tabZr9 zMpVGBxyr(xUrBiU2N~9OIH!0$j2tf;aSf{(!A&bx_mFbd=!&N{b*1iGl z`C#>Nw>kH#LDOSV^Eh*wG4UmTo~hpi!ls99H@1bu?Bq-7DROW@_1+T7L^Ncc+NgpB&3_kfIv+kEo5r&ylx6Wg|CSqBME6FnJPg z3)e;95z##D>LFtfp_N%c^dV7YZ+8jVmE*6I$KQen`EhGg#_EzmR&7f;EIFaoP)ibQ z*Wuq53V-o0{&U9A^WIW@7`?w(B)pd!e6^;RtLT!>Sr^Jh!C#0yzNdx;)YQR_MPQ7E!g9+>6xiE;G9bPBA$%ss1bSn7~B*e;v9 z)~yL1Lx@BX>fqTY;8zIE>UmV>TetGRmtd?BF-{w;JVCoJO}cEvS)|USrK6MLWVQN4 oqG8XCWRGig`4Na>w(8s)Sz=I#h!SQ%&mTEQJvQN~g#VlUAKI*HmH+?% literal 0 HcmV?d00001 diff --git a/quarto/integrals/figures/johns-catenary.jpg b/quarto/integrals/figures/johns-catenary.jpg index 3dbffca7e39bbcd8de8396149dad61202f7b3dde..e165be64a6015877fe99c6d427c6ab1a9571e497 100644 GIT binary patch literal 30906 zcmbTdby!=^wm%%)p-`Z>7mB+Sm*T~p;O-FI9okYza4k}bySueGEtcT!4#i!5`JQv{ zJ@=pYdG7PBWbgT8_L|wVW@hiql6jeXSqJRN`r6w704gc~761T%4nRd903g8;0{jC& zAOoQM3j+YU2;~2RO%UGw7Y`x;fcW}9{GaRr$p6I;uk&9N|6jlVoub1NGZyB0wNM1G5TMa0DkRH-hcCOb8+P%{HOLj#Q&}xu`LhjzcJES0qTG8zzhFFQd%;K zittp+(%stH#lz0k699PGKzyetBV($rsV1lRQ5JvzmkyD{+|AADp9VX-c)DxKOVj8Z z7}B8a{9E>aYBIOCA`v(WYF`+xWQE>cE{}I~$8yEb8E&h#L{>7uMDFf&E1IIL$cIH-a zd<4gA7XK^X_J3h#C$E3*`w#uodTeVK9W8iD2tOzQ9{~0M4}crM31ANJ1JD4}0lfbo z{Cxl7mjSrKx%~j{@N%{QJAfy=j1)ZQpBk*;SP|d?umo@b*x;B4zy&{e{^@6USvXx! z|5yL9{U@Uje_-vlo`0DN?WLV ziiVDXiS-JAh=7EIh>V1Sf()PA2tofmUdZ?;1hm{zsIN85(da;gJYfmn(dngYe-dd< zo-*)SxQAn462BoKC40-r^p2T@k6%Dg=)JJa2U$6J1w|z-Z5>@bc(+?xS=-p!**kc6 zdU^Z!`uRshMn!-6{3RwaDLExIEj=SM|3^V#QE^FWSzUcYV^ecWYum5hzW#y1q2UoI zY-)ODc5Z%QePeTLduMlV|KRNW;_~YH=JxI${*3=y9QgU)68$fE@Zs_xA|oRqqx~Zf z0;119!r~*N&~l>^NNJ#%gI?3|grO5kCw#B{i9ye+c}isAK8Z=pz_>rW-CDH#) zpz!}+68(=r|0B=K0stEc0X|?z_y7sOBR$MBD;M`sX5#XJn?SZ=zVL+JxNKyy1)&y_ zp)}jPWYj=h{q|3mAu5aXE_GU^Z$o?DOfbpe&?4+8Gdgp#%4EUTC|8v{=Bm}RzEq75 zQEo#DwlB+9wa5WvB~<%db@Kg!R$g5KS3)@M3S&$10rF5Bs}ntGQHw*gR6&vxo~J@o zWSh9W9D0OT=+r7gdZ6gY8?~gy)a0PZ)H=r9Q)RvZUV6o3F~5hW6uYc+j&8w25D-1^ z?na8fQV~c3y2tEqOSDOfypwOVQuJWZBw5)j+3$;jtm_rS@|##(IGL6GWFLt(~-_R|)?uJKvkB&9ll>7xPJy-JHbJMElHRAvS-nYdPrG zkk7g17%V2h6GQC1%h*Q$%L!(}petcj}KTvdG@!jT*0WEfPY$?!SF@_E##u`zm^Sb~89)E@6x z7$H;l+RrX39DL+I*(}LyuwXoZkX(;6*ywl%dN)k%)5QGM(Gn|gL}84(BW6;CZd_d_ zW?^{>Txi8ae^LBhz={Paa0zda@UEawNF@(qm@0w$t1{xXIBKvHI3?8yN=ml0xa)eF zXm2`XPiPe|bGPAoTEOl@cburJWvr8v!cZbwPWbBu;FpeB^Gsr(VEc9CT35ur9JN@` z)yE47eVqOfIBCqAn|`*hix=&j?zGBgOLB*!Uw_JQuGFQSxaP$+Ncf(kmw-7-hy=?p z;c2X~Adtuv6$%q4*`%$JV5tdYwZyjmbCHN91PYVqA&Rx9NPldU$l=!Kst1zN%2EgB zrctt%rO8B}to{!7OL#(>;V+JMasIm#9?*)bqjd2Cm?{1ff5j^H=L^}u+AB50ilCxr z&X$z_UlKDKA~Lvcd9$`-ti(E-8r8 zuHH^_f~)1r#3S9L3p+m^x*b6;iOr;_W$B#KYTv4J9r!6Y=Ic4c|6X8-ZguIEyD?J! zW~mYAPA}^5U464S#@Xl>or?|JWS;SdRV$Dk; z{fCtn%W`pUkSI#C2t`$p#Qk(a;H++vyd(ZRxfQn0xMAyt4VY5WWPV#CKhW}&uAY%9 zPy?8og`lFXNZZi_w9>|^7n3z#HmrinDGMv))$VBPn%W4LR+V>&K>;>31=JDCZAc^| zXrq!vw`Tp0bnlF^n?I9C^Y>HxtKCc-SG;>C_FFYw(98nK6(74~BX7pZ z2hC)iXQJ}7*Y$IEcFRIrW)^kog94ya?A979+U@11j%VrsFn+CH+dMUtDmvI4Et+T_ zC{?fCMr+NA<l`o1!^B@Ol>6YaVu*WNYTM8} zf~8r9$OEh?<@78~#S@J6yeo?;P&ZuU%ZwmmFxT+UeQsn$U>Tx|mnXr-^Pdn@0MDi|J`oxT2803AIB17?~nhF zFLHa?wndgxbP0AE-pgg11JVyPkBzY567ugmmIz`Yh1P}nS#wayH!Qpm|M>u`(BqIR z+pt5WHKXP(0^k>au&{38A$xYMi;3joq0to2K$@wIsO`ySp*R=Waj1y&Ik#-+p>tK> zz70il8YKBN{&awkcvS zhX?tUk4?NwiI>Wp>BZM(#Oem$o~O5BZ(+}VOa;jY5o$fHW+?2>z5u2b!@loSa0_O% zCk@Wzqtj@C-eIpQWs2D4r~8e4mzNOyi0NH&ru-(oJlT8zk1%u^>TP12FB`E#-Qn9q z>4>EsB0g|%Jo|;oxxgRU;lMMDs45dy48W%d^aqDgguVd6VUN5}^*hJ)7XbO5ZbCGPHN!Y7)0!G+iID)hE zbsabF-mpVnCLcfEa=TZCY*JEvcmY6;ake#Lv#%3>lu(E9FBv+wpB{bWl2n%<+N6$h zr$$^G?-z0AeXpW4sL||(_OS=_Sl)EHWa1sMZCO!ALkSSYzt^}{s(Hu)aW&D$en%4< z+Pu^8bJjKtsy|tm69E~X9)lTUR1HOiE|I>M27YF22tcQ(QJ){T@W(6E_GUOgJlxBN z6;6rUaLkPfd-tnPG!iB%a$%OmOHegVE$!WK*>{_t*9p3eJM3I`3Zu025BC8xR;dfs zDN*{M%Rec6ke!U$Hp2`E8aOspC6N^bDMI|SJ*o<3x#eikKMNs!zpVHUqF)+j8+jAp zhE|9#DeFN7ESZn@e)dYe&is@{vsOdnOyqyPUucu?ytju|_;z$@!XU(s9-G!82owVt zN(mmy$nM!hTclQ+u1DOUb*pWanoLgkpr^VJ___`e8iIa=qs;pG(}*EkT%RYAa2Wg3 zO0E}$`Z8a8K8gBd6Wac$XPfnQq`+OXd^_`R$}H)7T#!M{;sCqsQDS`L=5}duwneP0 zz5BV#VI~{0&(6jHUSkK_*Ov7F%i1c70KeFB&z{63gv%;0j*NQKaEz+3**P~bQ?3y^ z>$q6|abN%o8}GT14-KEa@V6w0lxnAr_?Hm13v_;sKm_x$5~wM_LEi;o@%JIzc68SL zX&X(5DL1`I`A#vcN6M9;L?da;-`}%x5&cB_j=T3fr}$8p!})pO7ly8=ph|J(Zl%j# zHdktdEsY$(HH)His!wLkZ@9ioBU*OXlwH>+wa}%S34F-j6UQ8jUL~$XAxj$ftR= z6zp4u@32NHTG-kek>V9!f&^ST~%yaq&)n;rF!T z_d4ihloVU6G5~{SmjRq&5rVe|4V=udWa9yis#CBjBrJbO6o8Thu@+AiCj0TUEC>SB zntaRc*PY+34R_?OFKb^>zRg773gokmI^b&ICr0`azQFwbm64eXWl*_@7Ff1W zIW0;e8DHRECI!xm?kQ9EAem|fkJqr7p6Z&Sn-Fe36E6j*mAwG$lC|zEIy}}2+*BSu z`1)vU1%ase}T-O%DY@lpRFdD!-I%H|3|NA&N$vjgZmJS*cH zs<1R{w9Tfjl(XlY)LcTZWj@4k$LK8jybFOL33-xYqVFHqBIOqKn!zpvR_Lpz;u12t zPN@MR#VZH~+Ji3uM46a!-!#pofwyM8V>FgUBvv}@(Q_Yf7xKQmIglg2Di5z^)U7Pt z)o{d6{TYg4wOS_c;JkqpP_w&Ga?ot6+4@GHV=cFr=&NX2y=uJNdk)@9LN2`pt<22SBC_8yezk4{R@W7%EiU}RGPri{g}N5F>cp}-L_;W?rpgM%R<1IyRwdnx-KUi)>r^n_sOIipR*a_FOtGLMOD1w zLX#;C(cHR^PpG^GP1r7c>#S5UU+>7iEjJvEEV$#s5Dc*O*BHdl9UPkJ4GFtTG!T{3x?~1)clDibBu8=YJ=yPrF{^(hW$1$r~ z2VPQ|GbwKR>#)UU5wCV#Ehw7Z^{b7X+|eiWuf?L@ZfzcRg8@^!>-)0-)RO{OK?++f z(cJsNBIa3cAlrkVW)O~1V{PhX6#soANeMVm(2|1+&(L zKBPU?q{;7@Cw-$u!{mf6l`v;d;v3XeZ9)7J5ewQcaHS2PKh4gN>H9p$_^U)~4m*Tp z&xEyIh5A*8J6jBqs)Xx6_ks_91YPRA1a?aJBT8er~NL1?I8htRB| zXZX5|d5Y9^#nT6%@oi{Fc|BWC5RWAPn0;I1jL!rEN;Ju9-4hopXP5Qa@T^dz3GAsF zdDrzr-(#ok7BpXEPO(~zIF%LYF56@gYKK~9PjWe4o}(q+oFzR9uxnp)MfyAGN^Keu@QJoJ0XU%D3abfa?{}J63rz^n6KuFJatEV3q&Mn%=dC%9mw2l-5k3~XB6>tl2TZ{@B*-DO0Z`u zql^9P;!7?cVxIX0S0p1BOp(#_K@RVG_Iy@@zS)HU2a?tzBnoH zmC=V5D`&BG$J|p~!}@-1rjqh2TT~RdX+P+4nY;jorXQJJV2?tlO-bUl{G;H#J{}bc zXsYoTN(K*#YA?u%`~9829;}<_it%}M@~Ut=qjWD-Pm|a3CXuAi`gRZFbEt4A>uv~! zlKHf?EHzDCs|VySPIzbUfQivF?kQuz7E4Kcef;KMl-%v{bta?mWYPJom*N!7V#^1WDuQnD&eZw)aWiT3+S$I$wbwdeI3FvI~ISj^9>1> zq|{@xbbu%D0ucS?{1#9F;?q1@Zdfx5o5OQh=!_#<`x$oLOmf+8ZYclSxNe|>KpTdn ztkev7V9)0AZUQ~A60Dr57R6w(xFF48&T|rhGd%oA-oMq6H=k|vNYkzBCm2~}eMKrC zY%l!>D|2)>_3)2O`X&R?0dwko;Z=Wu_fc-gay*)wP{r4h%==lVPmbWPF)CQ98AY;R zJ!4IOZ#ELwkZB!8jByQ~TUJvbGSAH3WHwBOW6F}=frW%he>B7LGB-pz;&a_fHhokb z6dKWwl@@?iS`?W@$W_EM$M}Q_b+(kXFMxxpA94{4y%KVNjXIMP9Ns8VdLNIE?uQw! zF0iA_CgmV*1!y9sJPw;n%nvQozF&PF^)PHIcY8Qy55?!&U1|3E8kv+)+fG8EM}U-p zWcF^bNT|fj^WOM)IT_zzjs_W$C7#kQ2J244w&Pl*C%idwOn|vMZCbyPjSkohTWCnr zC0g_2mCp#lTkQA4V}FM*r)rx0PN=m%myoYDnTDjl)}kGl6)h5@y}}Dy2-HickncC) zNEJzyiaf8}0D8@J5)MGkZ@8biX~mEIswnEGYvQRj4t^}0P}e6_I^iRzOr(P6Wjsly8ZnKl)Oivf%WSYIcWqk$$CAMb#;5q14%g$+w zzfw-9an4I$rz5|VNn#L;e!bD!Iu_<%_lxm--iT;qi-Xkn?X!9TC~aF~ z`*QhxfYtk(oEwo|$&gZFxxq z8+s<912X5Mw0-393v+?QAcSAjsjwXw--VA$uLm(hvo%v8muRo-L}RB8cnGJ3_f1zWhNCa<1zMBm#uVT1iu}F#q7&lhHl8UxN)lyPJ z@!EJlo@uUdnJxB>-GDHaeSNtDLFyu%Av0VNRJv}%MMns%jx}OS@6=js1L>)R>?EY1ywl$(G_r2)~wXVNGlerHQ zC`7W6{Sebv6hu-qbQ6{X8W(RV|4kM3?rAulvUNaM*L~5+fT0ayn-IJ1%io+|n1Fdv zia9svq){y?;5;*X6k@Be6wcjFQl@ol!08?Q1Qw1($aVeIcC2)cq1|tWI=O`Dl8c;c z`=Hzfd*su{HSc&61%CI}`>|z7i^ig;{WQq{g?DBAxy?QSDh1KZ1mpQb&6?PYrx#Sj%J3iTByZNJ3H77_{4R5-gy6E>il09|c2jg`OMq@x7{JGZ7bo zZ5gpy>1VB>6c$a7*b_#J6v`O{ha@zXu1KOj8!hdu$~IpE6Y^M;UO#Q%dfb7GyL&^h zW#|^ipQ}V#oh@kPAFs3SE);1v9Q^6*(0=BSS`H++=FQ(z6b!*N3F+z|9i;Tsx2(81 zbsT`(-$j-$03@fwUyoKa(~r)@%RR#{0NO*l=+(PjoRNIUuxvo?r@wEWsoHLE-x?SL7mk6B2lrsn5&PqGC~F z<<@?LmeZ;1hqn;UFca$2`DEW(^D*qUpa~L6?>ny(ar~)ms5r--Ab#->C*7C1>~4n2 z?PiISCh-*S2|VJ{KjKdDKtTdlMdDSR@$-J8v*6VD9kK6YYdDH!_P_Ie2Q^^yXkKx> zP9y^Yt&polpm1l|I3Lf)c1x?DKQH%n2+@$gvMluTVWeMdA5&1g0Pqe?!q56k_uNkw z1nA0Qy(>hP>Gb)cea9F+fBG2U@mK=QE=-f~eh;74LCpLmRC*;0iADoGK`=;ACM-xgOuMckSrpKg-Zza(hWvRd2NE67Lh?O~^IxS!Ym41)J<{GA& z{yd@LoTblIrmE_{(Me|YK+sXzYfG(R9VQS<|^a zW^*ATCG1d&2pb&TH}HQKjcmsee8Yx*iB#yca4>#t&u*a;!z}5PFhuuNACqg4vcsIt zGGT7!`_jGo>T@|$)L~M;Rek?!+xO$tO3k*x<}G0IoR)}}e}_$QnzsO-752>2ALr1nQ)EkGSb z1najxsbzKzrHWqSzN-xv(2U2DX)b2dIkx%W1OqGa%dKv7mXp&713~s!j|=61uy&0| zHh17q-+6OJ&*6l3rSj*@6K-S~=6LiM0A`<{53w$}86@D1>y2d5g5wR#SIUyLE2cs| z)Q;4FSv!}Ays8C0;(Mm79$T>|)^Cd0g*;pUGpn3$TCUYQU6>em002uY2MQNkzwk%@ z3)p$FeOgU#nLP8hr5IkIE(3Ui19pCvBgR6Fw`ej-Z!&!2<3zoCnTLDdzp`^JN3uM7 zCaNv`Y1H>@EZE9KZ^|k-#bk7bbVXo`Y^DLcaK51So_j~R!T%Vtdat#Ibn=%@n3c~h ztk%kkel_O0JIveTfQkdkMxB6s&9j-;$EN=r-l1jEmmt>Fl=7cimiSg$e!gbv0g8=I>9%@0z z=}iCz1te_~HI;i&!li!SlqT>sG)Ik+Bcp^a_vj%7EU=dS)eUVQ%z8ME*&+XoxmhS( zoO0V=aP|UlP1T9iDSM17Jf(h_d91y)sly{#Y>`x`Ym>T7zYiwb4`ylo(4w3U!$Cqw zSf1S-Uk@{S6DC+4OJ}|5TsTM_>9y#av$4R+D0CP57_l0wQyG@}%nB@iWIt@5(X2In z0TklH==5jQ4xuesQ2A(=SG9P~GzlTgaQD>TM{6IP>FQ`vDX*Ior;R3t7eEkFU&!Sv zxX&=ExifX5a5C+JLg#}lj_}^FO}T)TF0JniK#Y~=mNCgc%nvMZ`~-U@O2}{AdKG+m zDv8~7!2QI`Up{5c=@`D32dU&pqXrUPEmAh8X<0fsevzUaVEuY}0nFB399gAD)blZQ zSTt5W)QI*y!QCC)9;AO{P5Y#hdn|3*rce@bD(}ru*la^#DNx#)!Bv0 zL7usc%6xVD#b}SkSt2$}6-NflBx6LPwqPRAV{Uw1Y{_zW7ptK583lTR@lp|R2#qpK zj5$G(@w+8OlAD1aQcVGxtR;U@)OV#CB{DxCM{rk4F4@~ZruNt$A}xL3>Lw%fvUu{u z)%xpAJP!+=#e4n?Gy2qh;*hrY0Dp_mVR}coHZe39HgQ1qrKFW+L#t`glJr-oyfNFJ z52YMdrb3r@A>-?YTp6MpTH)N801o=_UQ=%SHAO^AQ4hG!NF1o_$N%1G1L+TUxKYkd zvw59kKzGAPUSfwOxU)1(SaW`LNzKJ)m_cby>tF0p4B?4^CN?=(P6s|d4i-6yllbNqobD^Qaahm#X z+v}QaUI3|OA;^YTm8c_|DyQJ~QhRsw59fB*L$}N6E2oM)Aw5tov6kxGDoAz-trmot zbnzLbKVT-|deJmW9q9GOr`k;w)O^%sNK5^diW&Wk8sMNcf7^Sc%T{u3BG2~Cx@MLh zQp9?Qhxn_x`&3_qB>e?YdfQqcAl5$}R!swSx^xy%pIFW&zFq^Ct&5-Y55B**jJe8< z#&gL%HBhO2H73r!x87MVkZUk_n+w;3&|YRA`%cxUxW0gf>swuilr(G~4!!IL>WaZxz?I&A5i?jUazV#DNT6UKRDn znLjfcK<7DPhDRU;E~fOmC)%&rxK`dHi@rYY7dk@KfvH?i=S%>00k%=&->EiZ7CTxW*h^N4VM|m$ zh;lt#5#{Iui?)n%dmruDZ2T%s)w=I;IMXCn2%F4o`Wez7+q>AGa(Yqg`R%Npdt;Id=i7*>!u0YVB&24{sT-&1-9sT}!i z^=-S0m1jN}uVj}?`H>yXheoxo-4K3Gd*lC|iG5e#sM$o7nGvmEDD{Rx)Q;SLcZX6W zI3)$w$!UT1!tCO*lJj-1SYt}rB2RJRJcAEqzwU)Dv#H&3LXN+OCMr zp>RN$C1xCENfy_TuX`7^9Rwko7aJ|Cu(=(4=*6O;ucjyJ_k-Aaj!9~>c&H%AYfVND z$hETt&OQxk4A4>H%{0+A!PnRyNR~tA%M%fva?;A?kzGv(V`^=&eY{IaMe5w-3l)fu zt#oTU?*jW}bpWRr^qIv($Db37y@x0T@=VR2m^s;9;E{EIABx6Dga#zWiN^IA$`L!u zPSAM(g9zPE+B8gwBO=xY$@l|Do(uk(g}(+%WOGyWV<8T!Q<2X-)stF{+u|=e;%DP)8=0##hP4`%Rwaa22wz zhns?osuT4UgWOI+$=ytLorFm^^3)ic>Cg6qxxC?#pcwS2wndiQ!c;S0nnY8Xj*T7k zaQSrT&s1n%V52WSO(hVv{R_?Azj+-RPl=jsi_-XwWzHr8>l^4;cfP=9#O)g*XkTaM zT_k!!lhTJWyvxdmQW|AcZu7tDjd?ek&*0oomw7lcWQyfR@>E>J@1C?IU*Z`*Tm@u*IM(Plbd02>APK*#%{*igaiK;2At zgz?9Kg5c~r6(l(hi1&SSG#{nK2t%rf$mjKxnCCNTl?W5aX%*quXDlus?6lk3A6zeh z?u8(~7eMdtc`sLle%wrAW|U3R8U$o{A&SjHe@NKTMm^lZ+p&KWvnskud}+LT^W0;3 zS&?=oE*lg9j}L*=rLWwE6G0*<|XSuq#BzC z{%LM?%-9+NEL6yf)-R}~1b$Hn?4f4Rf%Q1NbW zeINK+=;8la&&Ie&5E3sBT0O7YRhGQgT*%4oy($U(oU^=<65^XqI(T=l!-w{1d6d88 zTlF8PrTMdtvTO_R)`VFiGH_(2^KiNmg59`<|C59q=>ALCvQ_Pwu|*AMVUW1;ZC8Py zlDKpL7xlU(YHD?H@7t)3MZGN&mL~C5}08&#t+4 zm=->IutvVnz3t|fM7g?*Vd^0HLigFsqH1%VpPuQucRcb&K2;;i)6oyd9|aO06vU#r zjmBd+7+YgWp7m3>K#%f5qZkfoI)pQS5f@IqOK9#Rj9i**$#{9(7JEz`S_?lWMq3Dw zYyhZ@^b{$-2`DX3wqBM=S!qpGH#}5wAlRR?u%IG6^tek1tGpIymgq9UcUPbs zzt!U}WEmnL;HvxR4KasyngG8lOL4WZV33eJ>j2yqsw&zd(+Ggo!}x%jdu>yki)#a> z^l(+QkEgA0wPZ7h?2MwoIjrX}>4;TA3RcC}DRERAf@sT!Adq6qM#eunJ;?JW;S(i- zsHTG>$6|m}NSW05O!MV{4~)e4RNgQzQQk)sQCgto=RNK_q;HPS?F#!-c!h8KJ9vAF zHosAXT~=0_5Y|m?#7LdTqoPoy&cC@H841Si=hTKS= zj0Kfv#msNk(@@9nLJEP=u66sMVGxbF+?q)C5(;7#IMh?Xdq+7s&VQ&d{hsLsU{YnT z{-(oT;22$?>=Lo|@@uQJCvr<0#B!JxLI2`7x46(Nb<9$Y9sRx8pMb5HtT3P7`1o`o z82iWC1uY-y1JY2!!*`b*)S7!ZKD>V@Mq6p!KM{w*_G-GQ&8ef*Ddp09fBhnVXt=XP zUJt_i_)LUZ#=cV=yo4U9P~0T2^=v>SkXdK2&&?XIlV*Qix-wmG+BowagAF!zmLs&D z4EMKvbd+hoHU0hqaJ>E9E3ElSW%Q{5?2l8np?6J9m+P?5G7B!RbR=7{q`N-YI94PX z%wh%&ZUS>L&B234Sq7lbm)YV6py!*E?dpT+ofXE(zb9;kFwbF}nzo1`8)|0kIfVK} zYG{$NFXAMo3yRJUdx8ZBzUKwwscymD{GQbHoS|2LBfH_w>E7bU(r2tQl5LNgna=H6sX~6?REmwgRDSAam@v zMON$Zk0r~!j%`!U1>7?nNY;g~0} zBKz%t_cQhKbt$NZTu(!-?4$+B?u)y}@stj?%d0r)_golrXb+8s(ECk|xJk@{iU3dR zLF51(kU7ZF<66-V8%=du0jE7HAZuBCWef3`+u{$JBrZ#U7>j}DVV3d~vRqt4vxOMT zX!huIGCnYBAtCP5S+VhwsYz~qLX*(h#Ch*Z5>YI<7z@zD5qS`c5kY(!<^ZZ6aAtSY zFijK&srj)dLKn6dapTUbYejVwPZ+|ya}-^CzjLQ>hz}+G7|3(l2d51Brs?I-U#Gua z=&v1h4QhMuOq1!(E1-??0whH^jnN5;U2a;;;+k3V7p^Q;|Gi*~lRB)td6 z)C{{pX0ybHt^kP+tl?urT}^rM7)z-($5(>rce&dK$*6>k^aegULmCu;cCq{tunW$zc?)ipwd^GB2?QNgwzu zS;}KM)qYjrk2?^%nJi)`mr&VEUn5oWj--8z>zXL`URv|nXnj)ojRMUJVZQhl|t(w|CvY+7|WcqGyDL$y*q3`@j-Ps4;pPN}lBL#Z-pkXzRv5WU>8;lkyQ# z#I>44xbKFZP(Xrz#>iH1eY_ssSgSj76#3I7Nj_*wUJeijYY$wDMaW|n+=}$Q`G}YW zsWYewV(_33fk&(6v{|M-h-+f}rlq;Zbu61r73!j%lgz*?HC{Hpm}0MEZ-W&1kX;;> zmqv9);>SU8eo&?Jzb25Ws)Vlg3P5EzQ{Eo+zp^Fo7{1%wH#O6jCky$h{>bGfDA;d| zRNT0ocB}V0mD_OTHb55aiS0oc1FR=~(3`rqUQ&U_-m09AmNUNq5=*~(U$)nmJ~&P^ zbi&}#H6=9-j@d3EeyW0VCIaG+S42zt)bt;kVrX9gY^yH-nJUL#-V9NtsG*;=a3fKN zepS`a%7O{@ttdBjA&HDi%hMu)BaapMvM(&oJs?8(GJA4IF-yj_+3-u~sEO2X$uyB4o{*=!W-9 zF;s@K*bPvmq|`#>4(xpu+4FYzo!~>q(9bb7b7)Zzi!l|@_Xl!%7851h zT7Rh1jXz(w{pW1){Ftl-X~L_RmeIyu)R?{OvGp!VtdskhXtCg{_crI7oi9G|s7L`Z z+iX7^^6slAYP101)$O@o5 z;RO7Yp9dTc7L%=Or>(8tDe}I%YP!Ri6sf?)TLf6JUEH1}g zgZsYUdC{c*ZM!n#Qt=AW&>pxpgS~n@la{4yCEd`yV)pCX!c1HD)S>-B#7s$ndq-Ah zRjjw$>Lrg>yD^KiA@y6)0m&}rn7FW*pEII@9v)@DgM&GRN%4;%WM{^zY1Y^mOP>JI z>bLj@KU=#dSSplqN>0KRT@hc?q*uUnB&>GV*Xd`q?V1V4N=V4Xp&~p>z z&d+Ll`i65|J)ZQYOEo8P7}Qc1Jy;~7dK?;UbjX_)sV2C*6j1O#1y|a0h^Q(Q-wqwXvQe{mAo<_LW(6sdHAU3R&*R96o@$NTGo#K*b3bR?5D! zT>qo|uj462q<6D^*OX{`EEUz1Bk4rJwlu`lY1ERe`70A2oGp<&zk)>&v_pR~{q721 z!o@rh`n!D*V*`Q5%#Zq&IR6@O`!WBX?nm|lj=2>R3%%7Nka}GwP|#d#z8+d3hg<2H z5dC~G`cCU!qh-}{?*$OU{*>*Lw}(0Co%F*{hUv*Wjd%FIrJ7v0zafV{e4d&>p3mX; zeye}5&W}w_(x%$YV_WUTHd0Oo?aTQ$C&_qDH2v<~HsF5%CDn>?F^Us zCL^g?^20kvd0^V6*pE^${($tbn3;nG`M@=XfF=dCNs`@#w!??_>{lQF6AnB;5vVtN zd_~?K$2@6B&mHyKpJ$^{TJQx>hxRE@c>@nkm{7ncv9D|iL=jsiU(Yy0HCGipZ_|`{ zK)BaS)idjSrn9m+VCWOaJ|@w8d8pi)?&%ozCV@i?bx#jGU!|tf+|pj(61bNB8{0n| zMS9Mf(u>{V+~o#MU;1QmqKYeIpt0g8zoTxweoMMv;crC0NB%OIf#;(+vV-Jc@Rh~b z&PLGNiac`B?&L5S(hHzc%^5HuP7?jY56%^n(O@@ zH?*P3vY{@K>Od%>#lC zrrs*jUx!rz=bj4X(h+5RIN zie)Y8vsvgce_$B$rP5?$`IRC?NC#yYVoi1xd*wQD34YO5?d1#LN)xHhk&^}4BfI(= z*(lF=@d^-UZ4EStMl?A#v|uEKc3+5L)cD8yZ|OkYd)Zic5R^1hHPq|ECx|92*mkSh z33K;~bzJyTg{98SrPoaoz8~jTl^vC2YvRf4$ZbhVF1XDWTus?O+6&*qL8~7fGDp3I z3HW_FbzFkvo6F_%7C0{C91hf&FA7#v2Z zhvd3oggiLQ*d`I38T8wisej>50s4r6UCmnHiq}$+{g>m0Ub0y4JUODK)ESU0${Le9 z?O?ZVIb!cKgX48L;kP=Exf}_l$=ND(1}D^C5lB?otftBn7X?8zJCZUqR1wY}ugTu4 z2zaW9j|piIwjc0u86wEKA}fJw}tfr z_fXLA3Hj9PF)`$)#^r~R=f4{{k>5gQr&BnaImM=|+97 z&w;}m4_^(y_^}+SZm2453q~qWWf~#d*#%D$E8xn5cIQVO8&|y4W({VvT7;HSr?l1! zwAOtSrm3RLr*G>DBCn|;iuXD2`5bny=RBu{LYhy;c3np1qCl%yp(+y z(WrCM@AI8ahAzVav)sGZ)0q2QB8tA=Pu^fq0`V1L`yNs2vDQ(BY( z&pc8hIR&25H8rw*okJUEOaoWe<8Pv_D6})hqL$<+s!CGOtzPDWop$rp{t+BI__qK7# zF7EjK*$U|hVl8W&Vw$S#=`eq21b}onoC;o%&LOrA!U!{KNc98~$syt%omLdPQG zk8X)w>hbbgaAsW`8@7$*22l33r_pxp6Up5V*fo_PuS-2K=TTybH?uWepFue2&_@|7 zDLVw96?^)gEG#o|{6%S$Tc4tnBOrN1zZ`sK3C7)=rIL9<1z<=vczE`u0mvX)$q|Z( zamw<-&m}3OpO!aTZf(1uoT8mp>De`M{?7Z5UyX57%|3O?&f)mskUT=TPREHekawL} zv@N@Y2CWm-fP0SY&S^mW5updg|8mlb*Kq9Xs^>1SBSLsSBhvPdNXb3 zJ##NT&tv;@KJ3pnL)%1L>l6aBz@K>lH)WZGt9DIo?d=&d=`q>M^i*onPhXBBUo>VHl7ZRUO<|=JKMs zt}EbH;fPh5E99XijCZ;VPZd1?iQVv)d-8_=?%zL$x7jy-XlhD535iGETxfO0UNT%F zex499**30Ot`V2I)1Jc}n0X)eDZmUr>9T@d?ktPFBPye374F~`RG;n>s&gSzkd8jW z;7`H17{_VACW`9(sYiKTEu-4waJ!+L^>wI%76BzcHlQA4a_C&#pBKn5!PlPlboF@S zUk*-3BR+e5n4-(NZz5q6@B2wN=Xmi?Tum4#Jg~86aKmw_BfuUv4!(uq7xx?H9|nv1 zrDA^+uwDS0f9za({Ls&{PE*^2hYun!eow>TxMiyXve|~Fg1n(Ekm1@zBp}$v=)|t=SRKiRXo?hN3n8+8mZyi`uwClfo1)PF8)1J z8F*k&`NUSZinqH?-HZ>ti`Zt-?bw)s$;!@Kp5tbP=WJnJpHGea&MqI8NJYOU7FQ|Y zZxZcmNpj@JnTC27PXJi+GjGH#0)G7A8`aW)R!eyO%1w@#3()g%i~sGC=sfUVe7=gQ zUpKB`WfA@PDDg-_W~II0RdVW>z-n0ReXKgK$okFy3&Ax$%83`l-a69n+-|m8m5yxv z({DKPWe3}1n&xKHPlx0I5BidhO{ z0%w@+a%T+a7m3w>gzo3j^NO~+ur&Zr6e$fe5Uqx`;IuymB`p#wu^Za zXKE`GyBX>$qVXl>@<;&3pcJLB8h#jL z3MHSPF`HTG+U*exM;9NQn_5u3uol3X>^imF~=Ki+;)MApgRp_XlAty9~f5W z*whv&x;SHAn*ig~3e32+33~SOH}2t!2j3N;HSp7|{J;U)qX2s1rGngxTh(K^STt1S zreaP%=bH8Xc&A4cF<$R(6;u=L{zY@zYC7tZYYsEH0wVtabw_W;w!Fo;o<=_?Bo1i- zAB*f0?L^viOaKSJOjTV9&{`$Lvu@{j1|`2G08p3tS7EpJ?F*cR%;a@G6$It+u{zHOritbt#_Z zw{UYaZXb{!W7@l)0p81|cyGko++0f)m-cK7IKyACMkCxpjnAR4G}PQj7R{iY*c*MS zSR`N3(A?@N0-P2iy<=L1?X4IgsgC zpVE>VN6uPasPPnmvff?C80nL6E9~Eex^!*-00nlixF}eki#jpL=rM6){uS|Mn#bl^ z!2E8wT%Mc@fyI6C@Rj3-{tB~Xl?dr&XDqnBpQ}|E+*8Gpsp^Dz>()CPH zuvfP7x)u_Sqv^1Ib=>QB*CH%i$Ow$GfI1J|gZ%4@@YI%vR<&n9S#559WA#i{+|b=L z36%%VnbRY(s)1b;{s4QNhxXH7-r9YojDi+f1_u~k4r{pn*R@Fu=j0PO1Y;XVrE+4% z&eK#&Xa^Bxv)J*|^sPI%Q&Y8;LUs~C{U~c-F#K1p*`)sfX0qHTk1~Qs7#)8~?lq{b zb>D=#ez^!yHjjMPZg%ATmsdbz$DkK656--ATh$?lRDq|95eN(?)w9KWFT!i>JH&QZ zUk=?smsPNm$fIx!@c#M(_(+2vS{{I8={mpKqhB-kX9(L#>Im&ny}Vahbimwmp4hBQ zVH7i$op$`ZXZThAU_am|BDx)}EaNP|@zRB;Jq;8L+8paMa#gwfE5$r@X(oj%9wf9@ zF}zFWB7c1MuV~XqyV9f>&cl!M_*NdVr^R-L)UM@p10J8@NuWH{7CXH-U#orP`M$qO z+toDtxpau{RF9Bw2lq#M$%@eFer1jG0hure^!^?ze@E2TLH(U53mf%+QR(a{C4GS7 ze$Ll+;_rS+{J8Y@s}|P?_J+bTcW>xL32e~H#dr*QRY>D?o>kkp2LsU4upQo^6uLF6 zmiHJ8=$OZ)aXP+X(z9+(^CFRtLtVbJJ6~Cuu13(i6byPUE1hVg@(sj;d^$3%OGEXZchBKBKKA#i7+TDdhW+#UUf`6>8H~k4y`0BhCoC)P1ma+Lj2UR~d zIp&xaHIogjl1fHPMn~X9aWjj(UiQ|={IN0HBdF+r5B~sFX!w&+l6VZ0AKfaQ+j{K- zrCQKjK9>{cYnclidKKV)6*mIUl8UlMP84n!{5YsAo?;mX&N)3kl_bdW$jBH0^s62i z*(6s}k+i|RK=n8RnggG{m|R~AcSE<2Y$zPpS)dvG_#nGpq|AM=Hmyr>1V-P;DmP5& z9&yjjOtEW={X%cCMqX$A#1X0G-F{UA>gNKR1LzrHO$*`fooWxytzAie%Z`x31Tp^r zQ(jefD3bVF@$xy&J*&}vC~B6Fc-C(ZX;Lh@qLc5d#g3FvHr^gJ(Y;T@CNZ z+e!QAIO)KyTStJrnx*>;Npp6fp2rL^k8UYi!$%9^TXeCymf8ut$nE9xRnX*otg~Pd z{{Vc~)&BspZ;!72Dt_I+41OzWmPbdt@mItz4{Dbd2YD9K&3kOE5c3i@`F0PyigVa> zuazw{Q#Og`FAiTHLGiLC~ok z575`5>v34!T(6SHgaDf4?mR!JETdGiMUhYyZ1I)HPs6oMz}i?5;Z%WgU*!aP)|P?c z`)KXH-*&s1_+Up*rFpiI;_0AKW*7H(JD3h~I}fF6+UuHxHf$}MI;SbU=$2(LKOZ}kIhcDG-=WOEqL zOm?YA4GZ@M=(+7s#-3zyNWDAOOy*77jz>MZVy$`PVF1V@wN}7Vw}x{#c)sd9WaIL$ zQ_$FXb1Z-Y&5`(5Bc?_YH_GtEPCk{nrf=TGxW^+kAU9Z+8Kzq(-BpKF2cKS*jjP>S z#E`14<{X@M?OsdbYl;5=;V2rGo4D;ic|BLCKGof67V~QQgI?SmqJfUs&!D838WO^S z)$T4%QyWx!nuAamnpMme);ysBkI?@B^{cbFvA6pxTNMm8AFgRNr?zOKRZt{Y_UFFZ zRs!@E);HQpUu_`gEslp3n{5ns_H#5`d6MTIf}_)~jkTkH z#|!?z{{Zz@tZ8;99pQfyG<#E~Nxo zfU$0hyT`8uMG|%zJpRUOSiCnQZutb?&fr(vJ_l&x{{VudXik9be~WrA>w9D?^Y>4h z-06`;CJbL{k9K_xz^}JF9jQqF0Ps}p0@ZPW@ry%Ik3oNBQ}&ThWB<_nRrqfz4NVyE z-c-x~0DW!$01Eee%ZY77akFg(S8M^;s1@^v!%vr7g5CcBia8G-Ma6v?c^lpEpYp`x z9gjo$3c7m(=x~~R_RDNU+)NpR0rjgo&HRyDTtyO7!|BNb*X z6yHg?-|k6}Gt>^%vj?#jnQ?R@TV^~e?XHsge?Gz+RVp#^umhUSlG->X@|X}@gU4Vf z^KIT^96eN2wg~Hf9z=CHZoD$5FRCNM7yW!g``CXytfT8(cAb44znK_to?Li9-`#JR zeqAd^Lu=hC=w9igZPmP~a{>PVrjU^8G4%sJ)l0_~+Faf+PZDV1hMTM)zPDEBk>6#! zv+3nazC3mqqQFtpE-mHUUHKsfLMnyjuh}JQD}?gc z0NVl1NB5W2ew`~y{{T}u#9}hI@`i9dY4>X{_ARxZ)O3twamt_eeLj^|w!l`4Sx@Zg zEaLs(3G%n5E2h#l7$(yU(~aoy0mt(dz>9m;luc}?#C)!}!6WHgIzQPhB4k2DW4Pr< zx2UPnfu%L#G}~iukYHeX*BSOf9o6B_%fE2?3h9IC#$$8lfUS+g+;prfo7-`5BzDSJ zKU~s|fMPp?2=054Pto08V3LkHRf{`wG$w0-g~=Rx3d1KgkKxC?K-!G+{IE9Zx_Nl{g*~)6ZzC6a(R|1EW5@&BH3$x8 z!*&;%j7Hv7Xw?NUQ*F;T^t}tS@ffHb_Zb&C~sw zb^`a>L}pA3=O9-#c5d!9*r0Eeg>TBIv%i;cf<+9(^!nDT+rtj1E*xW>pFxUXQG?9$ zOTPeB#*m9yEWeIYsK4+p{*86HV}B2PvRgtPhT#Yq=tOhR3er2jNt{7+6{BI{Xr9IZHh) zUsAhn-bJVqH?RckPsbI<_@`32)MmQXF3B@rO47LO8bh3Y2SMy9J%IB`uA?(Oz$0oL z;Bo9Lyb_)mbHa-A*3)J-=3p|8+>ZGAdsm}dTHu9mL5dOr?VhT!>Lvx~G5SO894c+i>I~9iu-TC z(r^C&g0kp{tOn1FnhziL{b64l_-ju~?K06%F}%=>eQs91(C{Uvm;V3-U(oKOkAIo* zlS9e>0CxVwarDJX_}8IQUg`hQ{80E`3O|djcYWgozcDZ8m#k z6G?yz1ODwLi8~_?Add%=p4Bzpp)|0_selRjJ@HHojZSGk-q4&345NBhfw7)FL9Vd7 zB#biS+*W$TZYB*nfg$4+(nMrY9i}omWbuJgfWbE@V%F_1IBsw=@}3XnQ%R%AsCX|? z)8b~5Zx7x1tDe>xiTQEu9l-np3c6y7W-k$3MsPFGgTbr57}WILC&iH7>)19ou*;=t z3NwbbxWbm>p;p+#=}m$>x6T(Ze)9^%WRFA8kIt<}aOkC#OPpi!{{RY2Ye%}!ykT*n z-u>v@g4n<>9|Yskh8mj7kXbNEKhm7MfPKZJT7-xtiC7jGApI$Ii=(G$=1HSpFFRL` zf|5w&j7IP{kIP(R9cxcmb914zGa!(R3P}OxdJd^;JX1v?gmTAkZYyTo$Pn9@%VHux z=t~ZtN};6b3ioXyF4Gjf{SOAZ->_Th(Tk;<4&4#Enh+jd+LJxNX`=Fo;ELNZT`}JX z-V_elHP}gIrrpAK?cPp!6+%do-c$Bdk+-)_DS^gbSjnmUj}ea@_T%{)<|ffEboqbM zrBz4!rLs7$U5{&^GLgF+)-Cil+MSfKec*=}^!KG?7C2wB+mH~xeb<$%Q0e+>9h1o` zFZWdKs%vIoDvy!1jt_HzU5=YSlNgas<&j75`c!~f!=hV4zhY#M*8`u%p?IyDR*>cP z`*BrgvAvd6OK6!8oSrJp!pUMhwFf60ngE>J+ax}0Y1`8ntVg|gtr}M3EOUp|uuv`zTkUoN-4_MPR z2)sM+y7k6Z=fv_Ym|Xt=%aw`WFa7r^R6fHUtIKY+3yYhNv+gAn41FuP_@8%SW#R|$ zzMA4oKMx?&V|D1w2m0s#0D($aA6!?PczWvHbzlyDVYl=D0P9g;CcU^e@y1}rBp)t0 ztsNrf@@ThA#Ejq(=qhyf%EeswAI;DJ^5z2}K$wgU|fyBGAru78ywq8|go z-`P4lw?lOp{H_+h()e4ch5rD8ujw{{Gv)kW(gFVffnQnw0F8V*;VZdLgQP-(t^K!_ zeT~9#UuS$WUoZX&)upoyyZ$if-1a!Wvi|_}RLA(ABS-L`|I++J_-SzG#TNG9fDBTC ze*x0H<6G1PjI*?yD3p*vrXw9Y3f_{v zHWGTF<5Wo%WoLgbMF(*sA-!>oV0u;+kBZu3B3;cSa-m_uj1W2zj+Kzwl$JLU!I8BH zP`$BDe=bY8CU1~#J-DQmfjte_JV5s-l18lOBoI#r--@>{iM4A-FKcZQqK=GPipaG2 z7k$6VgWH2v86>`T1Bk%qK9yQnHRxc!inaNh&9_KX1Z2jnPCpY-*=pMUp?DWREZs+G zIO&gEbgqL*(4&s~AZ?AumiXyZ64Ysz(_6p}z*W=^0Nc|QEh|_R&szA0YbL$nT`R}> zW!t>)GU?ablz+#>NxBDb`t1)H{_ zO|!pcXX&wI}Nh9neJQd zgAtB6Jl007;%l~jx@vEEt$lmLJp-|YWcV&?|aCZ~hx*4pb zTcWdZmi;PA?JZ68fj;b;RY;%==pOrMiVQqX4gtvR*j4zZ)h;c{+(#T+c;l$WbULP~ zZ(xwwgaM*ZJ@~6P(KNBKi2yFUkF`>04;6SdKko1>_^P0TtnziyMtfd+q-KBub${{^;}{)7KU6elybl z0O1$Wbf1S3m3xgfKGmub!2Z#?P)dK>yjvsBsWl*ZW8Ye`tPL5LZ!MG55Ai){d2NJ| zO=k+iQAPf^0OFnW0oE7O37w3hH0(B$S4`3{^Il`g z!hk6ww~Ry2IOjcuHf3oxTt)YHWO3TFXldBM)Gw|rgU9AEBLt2+8rJZ&!~Lf6{hM?B zj&t?-af;`)eYaN=%1+g8Gg{gn5vB;-G!BE`G?LIRN#L2hy(3CpybIe={{WKa!R4MqjW)}wnBrV0z~uBMv##ZW>P5GFWg7$OMZlg*c%ik5CENScvj>0) z993z>tlDhTTROBu0ZN_;2c=QIQ9Y0DWy`P2k5Vbx8bJCv)7$rLxSSe5ErGg&40M=_luBxMkKNAg_PN4{~)Yx+35&o`Z9X!Y%YmZgjF`QCkL{|3^ z>I|{Lj7yRT?d@1M6FW*R*J}A{K5tS#l~C1mU4Oy;Cb`fx5(KbIu(BVK&N3Z8*pWzO z_BicTtS-rEWt1G1$3E1O)I5#5NnJrZ$}6_=Uk{JH* z_4M|rOmf`Zm2tFZKHVy`6|7y-+G~4zX*&78A??8b0P9wa_qMDUgxj8j>BVJQK=58i zZ##9)2Wq`#rli5`vOK^rD>t~&1G_j$*N@@)Qr}!kb~>M%a9_%nOLw+qi(uNlz3ZF2 zmEe#+@RrU!DI_zl)b3!@uDtd@j#mP+CAho4(^B$JzTk`w_^vkbq;K9#<$>+}+S=3a z?x&@5n!NXu zEVsAv8D^E9Y2`fbX%1ZeC#8DFkE7GHJto89y`Io?`OV&=Dr5J$lP1RGbr47Ur2hbc zSC`4G#dBpfj?wb*{uG!P{vR=2>AraYO}L!@0J3^ky^XrDFE`EGnukxf7H27dhCGq( zDbPx1hwU8!G}gmp(_9GcS~$r03CfXK&`WgIix^+JN$ZX~RY|8aO(wz8Mg1#Q$wEN0 zFH9WJ_5_YD{ws)8;%=;em2`SS0`}5IxeTNlrKlB}0!g7v`t zDsS$GE~I`Fw=dzH56C#p?XG|DAuH|=g(DyERZSar8@@2;#2(yV+E>JSB(0=)0t<_{ znm5v}p_Wir2N6mnZ^vlB9<}xl?8V~^58{XX6eGYt68t{0HlN}ziJBg#Xk}fJ2qx4m zwg+F~VUC@vKeoLM8oUqx)ci5fppH)&Xp>xxphYua^vF;}dk(CF?5hf#Aeoqdx^Z4D zr(T(SPo+gF@?oE+Y@o$^uC5HaGg!vEu*^C9D@uO=<+>|RC7MqhE=Kr<1>7}M^YKF`BS3PcgX}S zcFE(iY32lAmC(N-RM*je0flAh3{EE6ew4g;MH-XMa zYUeKW@{b+GiwNX$RZqT6Yj|AAcc`@TjBoiz;ZxX)XP?PnTgqnJ+XpqGWE`P_>{`ot za(BoM7=K!#mSwc{)oj=~5>Ia@*_59uPnE}Qv@*{} z(X{O|$DR_COZkSYp+;UM##-l39F$-7?T^f(yhj72c|0~xWqkHicRfc!52bWEMx$}y z?P}it08g9Ao*7nXW&ranW00I3TeO&NVE_`gpEwxK=bmdpk)GwJPGA$0vh(XZwqV=u5bPJOC^ zUAjjcv9LcnqaUHg76Wef+TJZn0K*_N6YYR1yztAX-1*jPoSdIvM-^Y~fcS=WxIZE( z{#AeND|tI^2ao{AJP)N!fU7e{e-nV%kaLf5RP|eHWU@2|j*tGKxPv8LrdmC;^3 zytcn$VM4JieRg#fR;W4@sCKp-P@TZ!kf9V>DfEbTPMMR0tq9_X@m{d@QCs+bPWkTdAIe4R$FH_4wD^5*Z}9I+J~6yOHPmw( zT_QI7k9&{aT#?>8RX-@`WhywuH{sufV(}KJ`u4QzVGow`&2a0Vve_rzbNJop#-C+*I^3$C-k+yReY?C`mC$Exk*9i@YzX3~zAGc3KQ(ne1%+pO*{(iq zu>{9Yynp)D=eHRbTUV`iz71AMt|u8A{IU6(nx6?vb9caYvEFz(&dT(8(cIqKq@H2j zygbOHAE-D!)%UmT<)GN;{{Zk(pA7g{!q7!+r)eJ(^ti3yi~x|q9sRZ=QP?hc74RR! zshKqG3ey}*s9F5pgie+B-@#J0U+`322WiKd@n1{n#&>EK*{om0lI&=b<+=aW{0*^? zj~iHAM0i_g9P|V&eLD@SNu)<}bm~0wyNS4H#Q4B zlRSs!Wg@)#biC7JTUc1)avWp1=qRx6X6h2neEw9K3>X#19P}TbO6fd5txe&(!*VU&-Py`^iQp&ZjN_pP=8gXVgpNCkp{nTj+H}Gx-zH3+n8EHU+;+2SP@tKD+>e!8 zqOuR+Z*N1|qQTJVd}ZLhC&Tu(w|*9p^&JlQw3hO}4x|j2qckVZU^3h$j_Bnw$fVm;JTazE1$x&JXvuVyMce_k;uoT zZi{a#PcmAO=40lsEz~Y9?ysVG4-4|+)KtV}=-eGa(Z9QaSr&KX+&-Ie(RY58&??0% z$i;a<*mSIEw2OwYYqknxf4JSi^rv7Or=raln-~kp0CwZIwNGiLyUo5O^4K1Srfa0I zf;F{hkp>?P)6%vgwpo-;!)YFrm>AmCthdpE=jTNOk5OFjg#0C~_{&_59W4#QtKlR2 zs_wkI6aKNt9_#9KFmyc=}4I(4a6xVlld z?e}A&r}u5+=>qrV&p59lnhT4G!MNww(0hN4BooT_H#oy@+lCz!W4HO@ty>oRKTnkj z3VUbTpFl#~H{{TLfac&7ax`yKwj|J2&@rlk`cWS#j>^ou^P~ytT)R0SV;!rO*6%K&mMgW!)foU%1ERG|p(7t7 z993J$R7Z0z-gXmNx=-3BLn9tK;{2I*V0B#Z{e9y#^vpMrS!pS0CN{?)Iq!|5iq@CFRE+DTjbcG=Kx7#FY1S7J#VyU% z$KG)o0qy)mRc%V<+U8c8&4b6El=SITWR;_mJN6kvwBxs4l-Vamd3R%Nt!4h=K;JjF z82VQ__R!o$&vo-5Ps*T%{cm*{t|g3lJ4(g=_Iet2t>xQZdGG+qB8;gSP#cEmu!zpcBvMK2ohpT$?yIP30bTsWSi?9U zcLZXimrB2OlJXdU+55gbk%|#EJ&#{W^&7i(f&fvMJDBv(r{P?*Iv%aCvNmI0MtXt% z6*i%-YHuaImD>4JO29@2K*Fygx?4M$p2!)>?g~d-=QQq#I7zQ8=ehZAL^oe`X9Ft0 zbXh&U>fV{+uM&7$T#ok0{hmo8@@9tt;kbnLS3~m1hx?>)+XkVt@Wz=8%9k9!m=ER2 z{3}ZG$|+-t)^a6?k%T-PcG`PT2J~JZ@b8K5*6+ZVG3k1tY_^kkAV}mVwXDyDKG4cT z0nxEs=BsOWV|@Bft#0;LYm#A=N;lvSU#Q};rPFRyOqS4#xPc2jyrgd{+i5(0b<=5I z5$`lRGo)(~eXh-a<p@Hals!#TLZw&Ym zyK}VmuCn7+_(|d?j(Gn7wR}A?fd!twG?uCJS|2uNY`<+54+3 z%zq&sYT(j-9K#>_UZ<<;k_Q06q*<+;e18kv-XHaTts3VYDnP}a{yPg6x}C&p_d)mc z72J3$!rI=orT+kK_|;_a{)z|g?k*D-SB?GKTVwmiz#Pizv8Xv4yDg`Rbnge-nS3#% z+i0#ZR#_!&O85`@?I_#y*n^7aB=Jk@w+(-O^3OEieo2WeanmI8k3rB;a61nZc*O>T z{{RTjgzbivqlOIesmt7v(XJ%_0G62k6a!-Vbgv%NUTgPHG0wyCvG2G2pGxZ^RMtGm zava0+40?MC<+V#uAYF*cdSr?$23jh;jPM0j+l`-0Qcr5umT3$)W_I7V(Bhq7jcaxy zT)Q{Rj>5VLbj#L~G=~lW>^fAiIb9=0xQ^{4h-HX5_RVxgDQzwS+#qIR0ItHrNb+Uh zpW&{5T;FAMlXb@;qz*I1dZovsEdUIE_ zyK8d}V_rbUJ5YdX+s_rOO%Tj-W0Cx-_P1#E+BWG2-@sGsE4I?CQ|z&A!7-eQ<^Cb$ z*vBl&6KfCvG}^GyI6uc1G%W|K=@XUhzpcHHEAzbLN<@KvqMz9+Sf;$VSUx*nccuX(i8w3V`{ymmP!wtkh!&7*0S`slZ{+vKc)Rg)^DVz-(JB3<^F!QkNLnvRzxzJspx zq`J6Co^}VH)q<&nxQ;Y$hO5oPsc`n_G0kMcYVB@uRo*c5%uPm!yA{hi*Yb#HHXbv&{G=4?=Xa4JJBrP_IJcPnwU z43EmVKMvSyT8z+(xEe)!PdEKuK5f_?v(Q(#=za^*F8nyUmHoTLcm9$B43G6n3gB^% z@TQ{J+>$sOj|X^m8-KK3+JzlK*jMt#D-`%c!Fq~pZ!rKl{{ULc{cEd$&vL64puS{# zP6KWr5$Fv;{hJ--!b7Lp!IvO@@NEM={?u{-hi&2i00(KYFt%9tdXu>RwXU&P$#0u6 zyKi7X9D52pMg5(p-$7|yPnPt7)cb z>hW4c@{%!_fgl1mew4XbT!z}hT@q#0fXq^U?Bk{zrD`spzjT9ZbCHgf)9Eni+MV64 zl+mkM&gqf?0HGtNrYavC{{UuadR3*}iUu>>T%!Tb5;npNeLK_kZ+1^Hz)fvyWiT?y z5=u^RSPIgDXj%N&3IIEX2v5JSoj``n}8QY$lAyk^cZJ%W^8C&lHnwkrZU-?&GCYSmC)QSQPFdbH_L| zC^1~GWpRI{d4gzI!!J#^J4)9}rLUi68St(-JmZ60&bqM0DElk0432O;xY|c!Tk>f- zLfALhB`V7lB^a?K*@w@^wO1(5LA9C{vJV>$1~3h9z9utXAlvh(2aMGVOKDOA6E{5n z04_08XwbnG#E|P;#`sls<7*!E8|ViLblJEpfClOF59wVtmkL_g-FcfvA|Xb3C(^Y1 zIi^`$_@>=2!~)giVE$|Ej4lo`N2Oo!U5=M+;tQB`gA2hNU;_hg1%magr7wGsxFc8K z`SaVtcG&*_m)jrx9F_Gi?7JbK{tA2HC6ne0@jl1_>Hh!|G5!_v55edzqVW%ett5?? zZlkBE813iBUr&C>43~fKQ=bYhI($*L{{Y8zC;tEnNou_clRy8^_-o*I!taJ-;q*G_ z@a~u|UhGEKA7qQoDddy1w%p^|zO9$x$HUm{?TmU3oTZ{%;?^Ob{C2!E;D7v*n{WF5 zU;Pqm>T8eq^0fZ|t|?rVc7G$HJDikW0`M=AQ~MUf80q$C@&5pz)*PM$@H)E{(X2)| z{{WA8{{Z-!^oW1epQT{U_nxti=8t2Co521GGVeM)mhL(GJbs_<)oDBj;9{swjbp+O z`F4;00Ew=~U%&Z(Duw>yf4D12Gc(U7@JE8i;53UYWBz^P{{Z4^nZ58&gFLbS01}HX zzw_@O{{RzSn>SYaS2KJ5tW-Aa2NIqR@Q!i)m1oyK>J$7cOHS~og`>J2UYlyaNKk)D z(T}ZOu>Sy$nsIK#kn27b@Vv3c^69p$a(OVJ{*{Me;q3z1o;zzBc-66xu_y=f72RsT z>LdD^=PmyL);G{rmn7qLEeAvM3HuZu_w9ees@Z88Nic(J4{zWf;avu$_svkT`_TRr z(BDJQB+@jP?Bi(dVNx5iDE^h_de)bwOQ`%Z%h@vED55;;3R3wU8z1=FoZAM?dO z!n&(33iw^v@6&Bh2dpXn6;9LdkEL~5f8IYz<&$afe?QWi*?$(qabbTvU-|i2m^rdzP=U2d<2WSJfjbctf&)MVu0OD&~OYk>> zy8i%U*#7|FJb(O6YNz~q{{X;+YH4&`l+3w27vS$GxAsk*Lkxc5Kf;}R;C}^6WWrq@ z#lE)jpZ%7)H~#=qf8G@}&-{6{V|E=5B0q<|0T$_v4^JHb0ME34{7q;0qrg4|v(R2k zO%}o?44@RYjZQP{wcn@wd#Bd3JY)X=BJ`i{FY~1bt1GePQg~Cs)?X0(9X+0#YYVmh zoJ%Aqr2hb33?rv1NVd(CVZjcUv8A5sh2^9p1p-X8&8U^VTMY;t6r9`9=kP;*m z@IRp9^Stl(-FyEt9L_qc_ugx*9Wy^Cf6f6!>dI=$00aU7RKXAMa~|XFbw4M20MOC` zH~|2_0&pRO0160UfOisv=mN$8;WP*dfC@e#;0J)v0O%Jm0OUbvFX3kp_TN091_=L! zEH21b4H3A2$w0UTh4inU2qysTx1LcX%O4m$3B&>N9v&`mz)9E9-3?AJ>~)R;JwCVo z>#3rqrAsdW6@v-^fB+xAkOZHo1fMwlIUp#(FAM-zLNfk%hQc25_!8zrGXC`v$@UwD z0YCx(V1nAw_yh#_6Hxznj3SnRb_r`HLVoc;P*B0{0nU2{^a$l<63V51$0XECyhjq| zZ~b6@C1L-DL7^m^OMbs3vI{)e6DaqS?w;#M1@)pJlM*lNq6v%g0ZB=J;U|CmbqCA`<6+Lcd4j-#7p;;06GqJy7%=0N}?0fCN4O&;)=w9Kf3z z!y2sz6BSMPV$|uKbPPZ{U`%3x*04ePxlOQ3&ie)h3PRqXSLb8p2)^fmE&am-0ekfi z90|toA2=4|{sW(D`~#DotF`rTaq{x!-_s6u!kwtyXA3w~gL6A0OZw+UbXXakx+2%rJshu8pM)Pr~jki!wM1}p(9 zzzEO-G=M-r7a|0KgPI*LIGh1@kk<;7zX2G6^dQiCanQa8h<64#JOCF^`%SaeR?xPt<(+0GFyMQu89ufffgV0@&!yTl<02`2Q2z*w1?ayH`0)h9K=}wD8F&E7pHpvv?XUvOFX+0UJt2r9r>U4Yr_tJwP373k%qPX+RW6y#xB^ zb(Do0R)V=iwosf#Qll9Ur6vj3f`Y+c?XsNG;|D1tQ+<&HV$4lZrix{ zz)iiKT;X1JHZJfRa1VDoM`Oz>A0A;HV*<^{EZx!b~_cJ8jcel{Mw z{7^n#KuXrn!^X}T?oDqCcW`o(X4`Il%|`D8lV&p(*5cFhP=q@=sRkh61_9cJb^*?I z;xIN@85}7;2|rg4SGczgy`QU#o0o*2G~0!82@pO<^Rj_V2$;QuzLN597Vu4)?YA$! zzP?ajL8v>zftO!gT%4CrfLB0(2jt-K@^|yL@#AsxVgfbb&|m0K0*4#n=ah?cfWu+> z;3&H}c%8@3ulO^7dtSW!dUJ7-0T%^Qyywvk!WUc@FTdG-oB1EK{ffrxU`QIl5nkt! zbPm)1egPL+0)Iy6pFEfT{TTr;+`vq=aru83tV`DaQlbRv@OBD-!~P$%{N=q=qv-DH z0WKQdyuB3N-Rzwld=S4<cymR=D-&+~unx&MRiUp;8x=;I9r+QncQ zolgleLQrvj`s+sY?3cmB@p~BloAr(a~ z*`nqqDD3L0?BwDqBX~J>f3xf#wYrYzkU6mssF3+pCAJRUwG8J6|O?NkN z=J0{R_2Kq1#=iys6aPO7$cTw@(+h&PfB=;5&rJ7E$-e~)#LByPUz#xhgUJPc8}9Pg z+&{_xwDt!t!!ITV<>wRpZ;5|dIWGj@IJkPq2=M>fT0_t0_KQ^_=)_<0KW(VFI{XS& za0)-~1;f7r|4I7a#K0v4=$_C&O#M^xUoM_c`@dbhID$Ff^#3*ZmkKVHFRJN3R?FZD z`WFsdg@fyH-pf@x_;R`Khr5A`eZ={y{^tik@%-TG_l0u)L^;3E(9zLQP*Kq^G0?#a z0|Olc3j+fa8ygD?2MZko2L}fm7Y`Q`8}A%CSMpmB4V1(NnX&(WUq9OcVl1E+NI``V z11Q80RAR`_&hrxyQ~-j0xt{}H(a=#KpeHz307$0+4{FX$gPWn>*ZC0=IvU2$X#fuu z-02aaf*U5tmpk);=_m2at`B&lFq-l1f59^{HpN-wPQN)Ym-o1gpR3hN-3jwNf7w5N5A$lvbSb!s>R8_t^X0t!r(hI0ixaO_p)* zPtNpmGdaKh1QgJ{Mske%ZqMGD+}f?5ESmI*dDBoi>T(}zE}>VeC1UYF(kVWTUH6f= zdm(DmXRqQ_`?EX2ZYznQ6QhEPl2>drokg#QuxEYj-8f{xMhOVWHaXNB{D|#+(*K>! z!2xKcI;veKzgI>s#8Gcjpq5kU_bPC1brq$c)6L-Esm4Cx`#qYH4mh40#Wm(#x{6Xt()dI4%3q#J})Q7*n3ZjQFwS*1=fo^K3LUAm8dno+oNPE z{Fue~g1L4kazlaUc=Lo9*eF(Z+~U|m1!n?)~}Q%ncHW+DlcC%-%n1-4E&yI zRGi(?lZuJ_VCTf7@>GP0ll@qr8AkyFA?cX8m^{9Xknt>7=5n{zU&|SmweEeiP>S=2 zD&qZ0tBq?oOZ0P5MJ!wyxut{V{k`Mx5VNaOYnFn?!&`gbVDBP~Vk-w9xs>X=GLR6f zS)7=RXIqtk{#YJsW*)K)5pTx#>=@T7Q)L1M*72 zEVb^DR*2ohG>*u*!|&XyHN8ma?RasgmE^UfM=Rzx)f)S#hocq@%c!ZnlHoUuWqcJ! zc%jV?7&S8NV@p>W*2L<&{nZP|=~Q@22lL2QcDwB;O$r1D*1uz~UX#Ph8W4Jme|O?k zl%n2P?`g>mM`a_UnrF2`B*@lB>YEZ*wuVFer~5o%61h&9>EissOoJnM=-JSKGP3l{h&jy>_JCbajn6@oO;M}J zHiW!T2TXi!@Wcf?yg|W$pn)ea=NAM#cLh(5h)FO>DJU5jnFPqlnOXUX1VzA7fCW|v z2s(;kw1wGRIZpAd+r_tNa?5Gv%gMqRRP_avRR>5KpAw_W8;ctq>=MhK)h4T7^b4EQ1j!lsJU{@ zB2KqSh}glZ)36nn8K%dRqK2w%L_LLLen$0;51cVuQKKy@L%7sW{2s$7M?yVGWXGfr zH|**>zMwQYy6(H1`09>}+HR-3HhbGF^gZ^GiVkIXIqFD?SR~!r(XUR$m-!Dd)gk!j1BE8o5hU{bUDck^IKai zvGya6-c~J2W=&@tSfbAEs-K#PMRR|*FAT3k&|!oH*N~OG7pn?lIX1oJR%m#`Z8h$| zA_Z6dQHhl#4aM+Mq)N@f0rWlE z7Hj0}tLxSy`CS5IyDLUN0fxHLPL=#*n=kCaJ?MeAX4XQ!mMvj5T%$C7>1_5v>`{%j zz(mQ$cOl0ji$E!Zij6cEWxBX)%k<^@#;>YVS)SL`t2Y)g1#+sE)C|h9R+>d3#}2D+ z(ZqL;ETsD%wVlL#WH*17m#h_Ja7VygQFrs^m!1Y+iYWGQ#lvktV-QK^hSTeavvp!QvTz+(;A>{_Vs$wc~%Dv9)(Y1i^Pn#8oY zid}L{_Y?#jzd!iqH9Tdi(JZ;PF}A!3 za-HScndH+O!5UEt*j`aL+3rBol6U8%&E-GLj7oO%QD=%w1sb)<`wH6+e{r58jj{|tMb%&vRWffyt_{9TwCBxi^gjrMA82(sHO7ng)RFNNw)lK?URi{MXdO2-j*yzgFF2m+Tfs8|@-=)YGt;43i^2?M{p9vV6+89AfAC%u56O;sBP1rwj5 zEh0FCSs}5yedMCtfr}zp$e{g|=v1h^Aa&?q*FY+7xDCI+^4Yu;=|0q=>8`(K1C!AB zu3;*vRO4gjqsQDwkEM>HYKJm%$L)R6AKNPVE0x23un!O8B5k>#JH7L^ouTHKp@gmR zSuKi0;f&?uni|+rRLf!E7UoJV-Mq7BCOue>QYz2{0=QimnPR(<`8%nZ#{vAD0Ua*& zl-yEeF_ZQ}BtOW%=d=azakcEas_~$i)O6*U2$w`zbsXQVGLb8*vn6p#Qs8+YNbN(W zK@`^>^_nf{>0PRcTr|UP>5bWAPMMy*waU&Z!Y#C6t+|=GsToHCCKgp!3w2D-76OP5 z)0@YYe4zH^Nd8-5sWbMo2!~Utlf|7BE-X^Zc`ivx7IcJebyFZK@&)aT!3(CV&^Mtj zLmFQ=k9$_6T;$2`xqnsLR$5^3w)RX_8C zw(3ZN#`MKrnJ9E*x{YOPyF{9B{LJ02sVI=aODgB^YW#6ruN6}@!-6E|6_q#MjgM1R zzpN{)tTQeN`QgMc?saDQbhk683Ar>WFb*k@E9XQTiMezsFWj!JKJrzO&+Q#g=(^*Vu!Zd@TJi3rD-xn+H^Wr zg%(BAO4)^MQgQ6_Ddv85+9$`cB)n1l_M5~#%xVK^?Nt6RWIa{-*V;7i;Nc{wyGe74 z#p2!X=bB*@Bt~}2CqYf>YG_%Fj;p^~SM4~4(6Cqd5F@|QFfuE1WXF2(C66u`^CfY~ z99U@_@W-(*JZwQ@RITgxXiRj*N>W-(yq3jkR4?w&J2_&9jdGpkW07))^0koI=;%07 z>`>2UF6g&o-eC+;+jPc|tr2{zm)$Xp3(+u}xi`hDLGfSiQlpsB3f2tIY7`;u6xdFah!>|iBB7N19Fb> z?C8}VQu-WRvFv)vVbLjt6tY^DR!`z_>96KVY~X&{aB_wl;}R^5gQ(?x9sJIh_GM%r z&h;(U>yr=Na|H#wmMlI@4D1Dk%MdIJz=gLWV=X4akUfGH$o39pu2KP`*ZA}M{%1J{ za_MyUn}YLr6T-ZRKJf5hqUf!dr){}@(#hF~rZ;?Bykq_#8TS6w7B;+n!KIl~G#4|W zPBf&?{23jF0b4FzXglkz>tA_N(Nw~@wHIJMqM;eZKJNon+N#ld*E$F>Nw2=|WOZ~- zyxZCP^376n@lPNp$(_HdnvgL7R~tK@Xi0}(bgpE-FH8P>TZ?5ODv~aelnvXu+G|QX4q2?;+ z*}6~KVn@qSRh8Qomnlt34tYz?G?G{p!`>%qZ{M)Cgohr4oyv7G~jtACDsZJt+YVozT zce(Es6}3G0UL=)!)-xTKKG#kym1jR1@vC zo~p#=os|*@(&lSJi6^f|6^D3nT>o51cI~xQsEb~638&q2trkA4s_r^dLgy!DDt2P! zpUXA@T{v>=v^uNrs8GCaY0j6PzgcPZ)gGJGbZM*`JC2C@J`V=Ix!&&aJKZm4vz@Ha z)HjY9mdc7}vI=kdRg+!Ls1};@YI^VPI{sKD7T3(wIHW z4(e7gUm}b$RzOqyz)*cyQU6Uj33CtnwahJ=UCR;EPc=UR`8Mx7+UQp5SF567QEH=* z3ga|1@%D87e7jh6u?{2KwA%^%l?8)F>l`x44-n3^e3hq^e80Wgw9dCb>QmLZ55u>u z;v+`ImDAqN^Wx7F;a}CwBudPi-M4(<5a2Z7kGbCblvvJE2m?LBsCJDe#B-+HR-3)s zbG*gClh(+c@UbgXJ#DHn@kjFP39X=)BFP8wS}$R?Z6tg2clFoZ6qFZX=@WS z%Ek!(_qtO&+Fukhcy9?k9wg1svRD!ko6GnKjACz_)@n0xs(x!nT$xeXH}X3vo2v^V z?;j>Q68IeZMa8akI!0MSi9JvemZ9kQk&<(in%sDLOT&Wmannn} ztz{R_L#1V(fJlwX@O1XjQ>DzjKdYC+WjdR^yKrmO6e|i`B2L}ucUM84MGaY}rt;AF z_mlD6+fQ{oRvU6>8On2!8@0TBucQTQH0Bh@6?z~!$07NN^D&VTbAhgle4F$;>_L=m zjObOP#PQtvJNBVBp&lvekilxrA?gl0I>*d6ok0Ra$XRXEn4r5A-c@6FuDCV#adK;n zHWNH>R3?4QCzN5}PFonG-N?N10-3s|_IT~NcXVEKYnZ(!X_s{S3vb=27PxeE0n*6C z4$C8R!<1gs$aC^FGIL}>Hr)Z+NAYW|+bhOABi)%jj&-d_X{?YN)1(jlT?B9Ol8t0V zd9c<%ITdjyi<5UFo@%9q0~IY1q1Xjq^T_5o#J5TG@|)EXI>xnl!Yw*dxodori!SuS zwT|^gvx+H*i+k9EwhSFnBN&W z_&juetC*{insqjfBi=EARh>z@in;qSqvuS8$a55=1bm{D`9m^Wz~lB~7fsg=*1;zq z8XEa;Ya}p}#vC(C4(59MCBEPsYk4R6wk4oHTsvz5Q(RNkE6gu*NUSIl_r(O;b=+F0 zn4T5qCQ+F#GuX*!H+)5`K2{T353pmDLMI^Qx)ouqaQK(Sf(dXeUU%0Pcy^BjbA7qz$kr?@|M(LIZswu#u z`=Lo>S&_&tG|Psl#`|Of^G#r#h#gdf#~1cWe?^ zj3t_P4DQtfqAY58mJg}ib@Ux0A2vx$f9ev#E4;nc9mtNkEc!z05$#ct(1-jVGnFX? z?LRDw<*N%`X>^!1t30)O?bL1(ub()-6sQlgBM|)9USHVXmHcIp@5_OzC>23lC%uuI zw#N&$tCM9W&koQ{Hm|DJmoJ;E-KZ5e5}XJN=;}hKZCK&^2|?@i{n+^%3D`d#^Cy)V z#nsJEO_%Hhv9@$oR6Y~#dFn^Wi?X$CwgX4pV2FD^H!blKKri9puCl+fg`)zUjF6me zp{vX(b9jE27ua`R(RZyHVOdf@27RrSo;-`q=B<9U-r*KGjK27`1*kd zKHpc`cqS4B=eAXi?9A`}IW?1I=#e1Jmk`Xwb(LfqL5&=JOs6VXC_&ViQIpG_CKS>6 zqRB_N`uc)5!-%Ba*c!QZ%;)yS(Y1h7WrtB)e%g1VL|DkMyc12tY^^MFlcz-5K|#O8 zRH5k>D;6H**3B=E#%`Y^HezsD?{O3|5=whF<&-|PvJicNxhvt2dCv|jmwV!y9&1Z) zo7OT)pJ&63?$z8(wbo3GwRFq_5TC0?|FHaTjvyAWR z0*gfx_;R;Pgx}*rZMQ|yRgea04d3g!p`j=CiTiCo(xgdWL2X5|wV?0bFXFXP)NRodHoG=7Np#LS3rU=Pdw6~P3V(%gM@ z9?NZ%kC=;|-@-;`ZL(cCpAU4LDq;BEeP{bnj=KB}p~l0?!A80RYjJhN3}==>K{3S5 z3cF50w2Yxa?bRM+Kp|Cg(Ak)aSC~T|vvsHlGl=_C@hY3PeVW?x+s*msw@=xnw!xnf z4`@-@X*^1|D`B~zwH*De`S_5XRWam=#Qo=tsHyW1rqVbJeFR?SYNz%$m6w>@_>_{$ zv-0u16|3*WZ*mfHO0-Yf;9jpNQTFy(GrJkOfUSL1`E#ag)B#f3m=b2C`XL11-r2Ew zOEW~-s3VBg#-AFb7i4R~>axy>!Cyil`%Ia>d>uxnr+O^iCfljJA$Ss4=*;o3;#JYY zGegE+;Tt;S1g>dn?{199=~cJTv&>6xS^HbDrjAI*C{N6yN!v@03`o*xaEs+IaY|fU z7>Adub?0vpJ%}7|{R!~7q9?8a&W_dxCi73UxW?E@v;xf$q(sk!gPKZicB$444#O)O zH)6l(l=wB-oQg)JphgZ}*dc2~C;Vkbude@`jn=9W^)<0S$ zv{Zj9>Q9X=B`?G)cotrtI_G9eL_qL7E%rVpY-~l^S-MMt;!3QwE(Sa1!yIDK@7J)6 zeTb%c8)~Dv`8MDBr@IG9KQ5eXb8`u*Chq-D(rcaBjUB3W%d!vV_MefL&0RwiB9uh&`1_#A|7Y&Sm< z&aV_K=rMKNjYKSBeY9_3of*^?|AhRY7Ezjhdjw-yeW<|K{NQQ%t=er**6L5S(nZff}a4%cAB1d8;6$Z<9Aix@0nkDwM$iStI_E`hwzm#0b(u%;lrmz z+K+P+74>SdEp0|=eOT;?$jIIw>WWZ47c=ApGM`i1NtKBYb{Q+pxb7@??FRe=5_M{x zS?iDW?bm2=+lOYSKfY+>;3w{ zSyS^h5Mkf1RpH`e6KM{|^L8O#GLOq8N%LGeHIGT99U1n4rZKt3WbanmDaKg5Y=0B1B^*9=mp(LVT>^Km<8s)1QO*SdIy>mIq1a7OE3egZ020QRHZD%xkjVEF5l9`>K8x z!!&7p;_58k9l?6vNW0R{koB7WwCIge=W9KK8jfMzvo#tnqtV?hPo!sGIHPSROV3^9 zz0DIkBP?#dB1fPpXzk4fE!8{XpYqU3*R;fVL_GmG6O}x`n@{d!y79=KhMu`3t0_fk z`ld)IMw?Ktz5h|jrvIsOAc@4Qe z`Lri}`LusLoZ)tq49%e3Sf_HU0Hrf)B2B2tkKS;?*3=U;@lSX`UjmED`Oj+Of|yTL zZ0-gw`~mvz_){ zV#>!y)cz{3Hy+-&_F$K3ZizbA`Wpi>e09yw$(2I(D^pSFs~k0lgtPI*7ZI3WyUS$5 zkHIPP;ydT_-vk3hVjkMXc|;|9WTomoVilk2vfr6Ug>FRCo?6^Vk7t*F`N!?z2Cq72YitoRNd;D2Y3mLn~&{=*iLDY7ZzaDtGXc*8Ca*QhEn4o}agf=S~nzG{y*fXrYU~uB%8OD(HN1Kz#mn`uXzU&$sTs z76(Lni8i@yp1~vYJ5|3H2j^$ZM8--RZ$*9T8VDHgQC8U!=8EJVgz+M&5BpPQ!av-* z7oEnlZ`|amBY@U}z-bydif7mp?B5yeZT&#C82&x6USrSRO&!C!g#BpnC(xkPzk#=~ zd2isTUaV8|0o*@4;ac&6C##e?f!IFV4=MRI1|FL?hl9e=0)y*HY;huUHezn(2rs4- zzJ}?ZQ>hAZAE>qVc>28{A}70Q2Lt(3Z8m|WUGHwUCCQGkjbb3gqVW&-~J$3W7kxz8JiI%Ic)4Rw_@oVt5 zc10Iu#kA&6bl*;yldQ(@U4!&aEZX)2D5Fss!mAeK>njt!s%x97Tw|E33feyt(-`VZ zbRs|9?nKBWeX+Q`rJL{p?b>a*VMqD&t1y51Ap^r}jF~;w6Cb}TzLuwC?@ABKGUurp z?9L7R`Z=s?MKRqthbVxu3Ns8g+TXn0ZehBQ(VBbcG8uVYGR^SiRGTYx|J{6wv8P%h z@&O;PNEcZ<8#2>)WydMrdL?^^%;=2)S$SlQ_XeS<&MrHaOsDToI` z*?f@c?EheaP&=AtIUe!75z&^`8%K$!qx$~I)Ho}Kh{Olf&w+YAlw(R+^B&wx?P}Fe z4qk=E_0=>>6-f0Sa&Nuu`+R4$4;s2}>aqR?>ZdT=2L3pM^bY3m?rqEksZR|*!nZfi zRtKwnKAu+a-+PU`=5s6?Z8fp_y_xD{y1|;CZ^_qZiwV;zSnR&S3LeUE^uE<#jvz11 zJxCl^4|Kb1eYq1B(T8EumJ+}^m?BSg4`q{jpX=SKU^=gRr1WdN^6NW40isi`cB7o* z(6g?$iYqdygK_-+4%`0iQgqc)uVIwVbN82JCbC{{Pyb0&@M~=p^nWL+9uauvHe%yh zHNVsLD^bsnzKM8ELa^7kR<^r(xUIVTCNe8j5rjJTNlq!_M0um#CN0Xo_v9iKKickQ zyV0l*;O*VKJ>0q>Jz!m#RZN;eqS6!{8CFwwBgUVB)*RT^)b# zJ+HuMZpRK3gr%y9iKMto;OFF~vwP1xkx} zli*N-Y6~kwCB1iME*Bdh1TNePTuQ@7L8BavgLggwb@|365KJ4Qgz2g zKr=v4cH{U{%UxVXsm(qf;x<#hbmP+X9DUVCZTIQ>2w8k;ufwf3ZhfzDKS$(&RX!!b|a`Ro2f2fex*x7S5QCw^EsLC4QU4#;s zvk|6MEAi$l?ia6D=pqL)WoE@@$|5!ag+NMVo4+8%yALhdDXC?Xp^nf=sT~FOr}nZU z6{pAQQUmsEect^tJv$*HgRU4R`^^NZO&bou83W1Gep3zCs@Cp7v7Va0X+~J&kl@{| zki3bIbK;rcWi>8*P`|}p;g~dB7h;9hP}Y~tHfYy9G8`)#If1=z5S>F=ntY(Z{4I1{ zX^)pJgiv4{#;5Zm&we9i)KW#}Mq`l$)*ZPoMd9n$c3$$mFq3GN*b9gklt>>~JZ=um zi3o(^y&M&sOd^a_w3hktV3$$obSNH)iO!Hp6IJAO|OzN-E+N1h1d#f z2Cp%Z-3qMeS>Z2&zvP17o0tMZOb$(6vWW3CKTodI;kH}B+=@I3q~A;O-2VyOW0*T4 z6P6(w`4KeLdT*tQV@<)u@PG{^C-FGSEvI;e;NYb1^~}TVPiN(JBD8 z%lY;loS?z9|Gj^Q5HkQ5+jlVYtJ-!({;_)}O3<^ve)TG!Ze$##NUNPjD?T}$CZkvT zDq~tYVG%)wHks@d>AQTS#t*z)82$J&b?IB@G%{Dh?DC@*@$i`FNj3F1wh$QNy(mw% z?3e{2Fd$(o{kAug8Xi8;y#ot<6i<*COfW=ZpG+mHN19X~PO*^6VKS>J$mm(eGWS(D zq`K*#+bX*(j6^j~tZ#FYf5oSgE#GW|8Ns0HN`sl~BSzRyaSJQ|?R-l}Z!ob#LH6W2 zG)Y!}u<5}K;wymCmfZkJN#*^}qcSgg9ger@*DNx<71uw=>^`!oJS@|`DSrijh7W~@ z^hTxJw*f@8@dI`Sv_n=aOqnVDVrqtIN-m`Wz9R5IOfp$jsnp*KsTck1zIy#fSWZYG znpsG`_OAId18dJMc^ePEm(zMiPs~*tp6x7B0VvyOTAf>!Lzcm$l%FW(qY|%JbhV^? zM!OzquupX`lzV50pGh~8!t-XRJ?q0K>AikL$j7e{4#c`|0q>_a_)&a*;vLu_{;e6@ zQ~~rSo{67liE;!^=4jaG-(r#)Z%0{~N40*5vY3o8LCSB&Wg`3J`&z?i-7!Or71Hcy zB~O=>@>*?!+$jv-wlX>P{fI%iexrWT(xdZ^g@U|(rQ9|{d#ea#8#V1|{|gGH?11G~ zm={M*)OL)597h)A6{hG8y)s<=WE!VoWy7Srot23ry8`$K63rmpZlasfYQTl3Vxxy+ zuTXhmvEmzSLGSIY*Yn@{R13IrnPMsO*(>#nq*rCq)Iuux$bkKgc$`$Fa_gurvwB~m z+eua*7X~3w<%xk)%37Ty2Bg7kEU%sfiQnv-##ZELVZ$@ni$dEeA!trzwVyQi=G*7% z#qQN+JJ}h`pdhWl*T4w~coJvJYEUmjg0=%ek=rO!CYaJ+!rpvy*DNuP0Zoh+!3ryY zHatCSHNT0~_rcwFn+Scqn=-iOwJ}Rcnv>h-E)6w|Cz213T9eu+w4=CodP(lfQJZfN zOD&Ap3$Ks*OWL6^OHuJg+U6n?dS_D~rHLk7$%Ank-t+UgnG_l6Ku0{JNsqSD?Ob$4 zF5s5qLQ7>EHW{j%UyFbVH|llb=}N(+uj^Nxkx-BLq=)fEdHmj!FM{o!e~av`;nwaY zn(*yr=anhsmfh~=%m=^2Rr(1Gsuzene8E(0x{3*7KyS4hCQl0Nyc#9pE%uT!MP;0* zQkgHyT8wBrv@vI|PkmUdvhO(D3)96sU`&UaP#GHrElXqWR5}aKM{4II+3T2RQdX1M z#F$lYB>NZ?Q$ViH7f?MI+9+o(TeYTCvnrjm_~#A69u4 z9;OmEuVDP$$el3SY`vtdNPbS_x5db~^6QX=M>Be}S5ms&)nC~57ROu2xymEPzmrQw)4mGSh zMj2G@+Vt`wVQUd0t82ZhF^%{0hje{Cm@;ZU;EyQDvjZ||@}#GlZfjOQfkQ$;qxa@J zdl3`-tvHY8QrS-ZT_{(aizR!h^_Pj87k0`r|3>+$}|_=cMB#d8)J;zs8+vcA=0(u*T$#1 z5l+19b-#>@w3J9_SBH-HidV9)C2cQKsaI{4!6Vv9E6FKPfLi(4Rgv=AVm<Ydo+FBQ|f&3{MJ-+Pmn;^9KWk-zi=Xf@PpWxN2J_ymbAR6P+aEu+ae~3w|{cxS& zM~73N5w${Al^Mc^`fhbD#U8oi8(d_uMsu*B1s#HlBW4Wu)Q@DJnod zOf9F_n9E@OZp`MYGZKm0*bS-o(2YqudebeF9xl|So#kMlmcXYbXxB9^h6`;^ZFlx0 zk9+n?$y^Fc#dSBrfWQ0}O1WoPU!1OE_7MpWStqzM99MeK$Zrm?2~ktMd#N=4 zbGgrBfqOkvYMQh#^3Qt{Y$3(BG6IH&uoBS7bj`%_zG~>`I<1U62@c!K!Z<5S(*7Vo zR_^sQ1@Z9-onQY@mQwM@-q~6+++cA??~}A=j3{ptW!jCoE8iv&Usa2}#Yyd3G_zS| zii0Kxt4Kz1@rX$H_EhW4w)$mtnjR`Oeq@Hd^H;00#>QmkT=F;Hdc?>{Fz$bcwV3(H zv20y!=oWUh&lTI~vcV$2B=UU!w2vpf%#^W$OR+5p`d}?ZvrhRZ;8rH@h_BCqclYcj z$qjb!p}* z7Rnw!Fkr?+v&*0DQf_i~-5h^if#z}Zn3?QwD1wg}MZ3_%&m%%H8bEX=Bm z1*z`ZaYRrg7OX6!l)Vy)ioT4;f|u2XKc7dM*r%fH;ZSXxWn1w9GAp1f(wLg3lac8ywyfcTr@PFxa6~fmP^2*>Ig|4ZyoRTS!AvOf3QH+>`oc%%Zu{YLxR}Oh8VEZ z?lR>{qE>${6tWJCj!@q=)lQb@P_fvh^mlSBl(1lhc}pB2;k3#==As~$QnKs1t!**^wV_Zd;UwUOuE&iki=ekcm_udsH<{NX>E zSTdJ;ph>i5vR?H`A$*knifzz!kE4%Q&{|(g^b+7`s}BZHQ#GSgur%x{MtCpnG}r?{eD*S!!caATfID8l>NcCSm5_ID!t>>? zZ(#a8i|uJ{A!LL-TMYC+l1NxqBy1Vvlb^DGymi-^NJHT*xm{iwv0(3$#OD*C*D%Q$ zo#^)xD&I>^sn)#xic?s+fo|WA+_&EtqP`kj#>}((zCBAS8CyT`3J1;&Dk-E2Qvw_*rBVGsSU64A-OSHqRZ{3d_8gW{lY>mOFvwxR?a+w+ev-O0#shAY}2dVRi)b^binF$D3Ubv{Rss3 z#m`G&K(O1rU7@=>Cn3O3&8 z{Me@}Be+lEvy`V_2DK0^kEIRQb9Tk)?6gsMkFyhV#zL`S*K#Ji6)8srw>-ik0~T&z zc={f$WG;aL2YND#HLNnoWS!7B8y=)nONjyfIx2DjUvV4iZLY zQ3$nHeeZh0_YFYe9KT_P#q}O`OZ3tZ{m_X~7nEbgEP$0{AJqx4EcXpvAI)6|@{4qQ u7SRziS!7Q*yx2QuqBQW;nyRtbc=CG_y{q6g9i4+kTl*!pfs@srQ~w_<@!L88 diff --git a/quarto/integrals/ftc.qmd b/quarto/integrals/ftc.qmd index 48c104f..76a2bcd 100644 --- a/quarto/integrals/ftc.qmd +++ b/quarto/integrals/ftc.qmd @@ -8,90 +8,82 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia -using Plots -plotly() +using Plots; plotly() using SymPy using Roots using QuadGK ``` + --- - -We refer to the example from the section on [transformations](../precalc/transformations.html#two_operators_D_S) where two operators on functions were defined: - - -$$ -D(f)(k) = f(k) - f(k-1), \quad S(f)(k) = f(1) + f(2) + \cdots + f(k). -$$ - -It was remarked that these relationships hold: $D(S(f))(k) = f(k)$ and $S(D(f))(k) = f(k) - f(0)$. These being a consequence of the inverse relationship between addition and subtraction. These two relationships are examples of a more general pair of relationships known as the [Fundamental theorem of calculus](http://en.wikipedia.org/wiki/Fundamental_theorem_of_calculus) or FTC. - - -We will see that with suitable rewriting, the derivative of a function is related to a certain limit of `D(f)` and the definite integral of a function is related to a certain limit of `S(f)`. The addition and subtraction rules encapsulated in the relations of $D(S(f))(k) = f(k)$ and $S(D(f))(k) = f(k) - f(0)$ then generalize to these calculus counterparts. - - -The FTC details the interconnectivity between the operations of integration and differentiation. - - -For example: - - -> What is the definite integral of the derivative? - - - -That is, what is $A = \int_a^b f'(x) dx$? (Assume $f'$ is continuous.) - - -To investigate, we begin with the right Riemann sum using $h = (b-a)/n$: - - -$$ -A \approx S_n = \sum_{i=1}^n f'(a + ih) \cdot h. -$$ - -But the mean value theorem says that for small $h$ we have $f'(x) \approx (f(x) - f(x-h))/h$. Using this approximation with $x=a+ih$ gives: - - -$$ -A \approx -\sum_{i=1}^n \left(f(a + ih) - f(a + (i-1)h)\right). -$$ - -If we let $g(i) = f(a + ih)$, then the summand above is just $g(i) - g(i-1) = D(g)(i)$ and the above then is just the sum of the $D(g)(i)$s, or: - - -$$ -A \approx S(D(g))(n) = g(n) - g(0). -$$ - -But $g(n) - g(0) = f(a + nh) - f(a + 0h) = f(b) - f(a)$. That is, we expect that the $\approx$ in the limit becomes $=$, or: - - -$$ -\int_a^b f'(x) dx = f(b) - f(a). -$$ - -This is indeed the case. - - -The other question would be - +The Fundamental Theorem of Calculus (FTC) details the interconnectivity between the operations of integration and differentiation. Two questions come up. First: > What is the derivative of the integral? +That is, can we find the derivative of $\int_a^x f(u) du$ for some fixed $a$? (The derivative in $x$, the variable $u$ is a dummy variable of integration.) -That is, can we find the derivative of $\int_0^x f(u) du$? (The derivative in $x$, the variable $u$ is a dummy variable of integration.) +The other question is: + +> What is the definite integral of the derivative? + +That is, what is $A = \int_a^b f'(x) dx$? (Assuming $f'$ is continuous.) + +These questions are answered in the following theorem: + +::: {.theorem title="The fundamental theorem of calculus"} + +Part 1: Let $f$ be a continuous function on a closed interval $[a,b]$ and define $F(x) = \int_a^x f(u) du$ for $a \leq x \leq b$. Then $F$ is continuous on $[a,b]$, differentiable on $(a,b)$ and moreover, $F'(x) =f(x)$. + +Part 2: Now suppose $f$ is any integrable function on a closed interval $[a,b]$ and $F(x)$ is *any* differentiable function on $[a,b]$ with $F'(x) = f(x)$. Then $\int_a^b f(x)dx=F(b)-F(a)$. + +::: -Let's look first at the integral using the right-Riemann sum, again using $h=(b-a)/n$: + + +:::{.callout-note} +## Note +In Part 1, the integral $F(x) = \int_a^x f(u) du$ is defined for any Riemann integrable function, $f$. If the function is not continuous, then it is true the $F$ will be continuous, but it need not be true that it is differentiable at all points in $(a,b)$. Forming $F$ from $f$ is a form of *smoothing*. It makes a continuous function out of an integrable one, a differentiable function from a continuous one, and a $k+1$-times differentiable function from a $k$-times differentiable one. + +::: + +To motivate these statements, we refer to the example from the section on transformations^[XXX(../precalc/transformations.html#two_operators_D_S)] where two operators on functions were defined by what they do to some $k$: $$ -\int_a^b f(u) du \approx f(a + 1h)h + f(a + 2h)h + \cdots + f(a +nh)h = S(g)(n), +\begin{align*} +D(f)(k) &= f(k) - f(k-1),\\ +S(f)(k) &= f(1) + f(2) + \cdots + f(k). +\end{align*} +$$ + +It was remarked that the following relationships hold, a consequence of the inverse relationship between addition and subtraction: + +$$ +\begin{align*} +D(S(f))(k) &= f(k),\\ +S(D(f))(k) &= f(k) - f(0). +\end{align*} +$$ + +These two relationships analogs of the [Fundamental theorem of calculus](http://en.wikipedia.org/wiki/Fundamental_theorem_of_calculus). + + +We will see that with suitable rewriting, the derivative of a function is related to a certain limit of `D(f)` and the definite integral of a function is related to a certain limit of `S(f)`. The addition and subtraction rules encapsulated in the relations above then generalize to these calculus counterparts. + +--- + +To explore the first part of the FTC, let's look first at the integral using the right-Riemann sum, again using $h=(b-a)/n$: + + +$$ +\begin{align*} +\int_a^b f(u) du &\approx f(a + 1h)h + f(a + 2h)h + \cdots + f(a +nh)h \\ +&= S(g)(n), +\end{align*} $$ where we define $g(i) = f(a + ih)h$. In the above, $n$ relates to $b$, but we could have stopped accumulating at any value. The analog for $S(g)(k)$ would be $\int_a^x f(u) du$ where $x = a + kh$. That is we can make a function out of integration by considering the mapping $(x, \int_a^x f(u) du)$. This might be written as $F(x) = \int_a^x f(u)du$. With this definition, can we take a derivative in $x$? @@ -131,33 +123,43 @@ $$ That is $F'(x) \approx f(x)$. -In the limit, then, we would expect that +In the limit as $n\rightarrow \infty$, we would expect that: $$ \frac{d}{dx} \int_a^x f(u) du = f(x). $$ -With these heuristics, we now have: -::: {.callout-note icon=false} -## The fundamental theorem of calculus +To investigate the integral of a derivative, we begin with the right Riemann sum for $\int_a^b f'(x)dx$ using $h = (b-a)/n$: -Part 1: Let $f$ be a continuous function on a closed interval $[a,b]$ and define $F(x) = \int_a^x f(u) du$ for $a \leq x \leq b$. Then $F$ is continuous on $[a,b]$, differentiable on $(a,b)$ and moreover, $F'(x) =f(x)$. +$$ +A \approx S_n = \sum_{i=1}^n f'(a + ih) \cdot h. +$$ -Part 2: Now suppose $f$ is any integrable function on a closed interval $[a,b]$ and $F(x)$ is *any* differentiable function on $[a,b]$ with $F'(x) = f(x)$. Then $\int_a^b f(x)dx=F(b)-F(a)$. +But the mean value theorem says that for small $h$ we have $f'(x) \approx (f(x) - f(x-h))/h$. Using this approximation with $x=a+ih$ gives, after canceling an $h$: -::: +$$ +A \approx +\sum_{i=1}^n \left(f(a + ih) - f(a + (i-1)h)\right). +$$ + +If we let $g(i) = f(a + ih)$, then the summand above is just $g(i) - g(i-1) = D(g)(i)$ and the above then is just the sum of the $D(g)(i)$s, or: -:::{.callout-note} -## Note -In Part 1, the integral $F(x) = \int_a^x f(u) du$ is defined for any Riemann integrable function, $f$. If the function is not continuous, then it is true the $F$ will be continuous, but it need not be true that it is differentiable at all points in $(a,b)$. Forming $F$ from $f$ is a form of *smoothing*. It makes a continuous function out of an integrable one, a differentiable function from a continuous one, and a $k+1$-times differentiable function from a $k$-times differentiable one. +$$ +A \approx S(D(g))(n) = g(n) - g(0). +$$ -::: +But $g(n) - g(0) = f(a + nh) - f(a + 0h) = f(b) - f(a)$. That is, we expect that the $\approx$ in the limit as $n\rightarrow \infty$ becomes $=$, or: -This figure relating the area under some continuous $f(x)$ from $a$ to both $x$ and $x+h$ for some small $h$ helps to visualize the two fundamental theorems. +$$ +\int_a^b f'(x) dx = f(b) - f(a). +$$ + + +@fig-FTC-derivative relates the area under some continuous $f(x)$ from $a$ to both $x$ and $x+h$ for some small $h$ to help visualize the two fundamental theorems. ::: {#fig-FTC-derivative} ```{julia} @@ -189,7 +191,7 @@ let (b, 0, text(L"x", :top)), (b+h,0,text(L"x+h", :top)), (2b/3, 1/2, text(L"A(x)")), - (b + h/2, 1/2, text(L"f(x)\cdot h \approx A(x+h)-A(x)", rotation=90)) + (b + h/2, 1/2, text(L"A(x+h)-A(x) \approx f(x)\cdot h", rotation=90)) ]) @@ -224,20 +226,20 @@ That is $A(x)$ satisfies the two parts of the fundamental theorem. ## Using the fundamental theorem of calculus to evaluate definite integrals -The most visible use of the FTC is the computation of definite integrals, $\int_a^b f(x) dx$. Rather than resort to Riemann sums or geometric arguments, there is an alternative - *when possible*, find a function $F$ with $F'(x) = f(x)$ and compute $F(b) - F(a)$. +The most visible use of the FTC is the computation of definite integrals, $\int_a^b f(x) dx$. Rather than resort to Riemann sums or geometric arguments, there is an alternative---*when possible*: find a function $F$ with $F'(x) = f(x)$ and compute $F(b) - F(a)$. Some examples: - * Consider the problem of Archimedes, $\int_0^1 x^2 dx$. Clearly, we have with $f(x) = x^2$ that $F(x)=x^3/3$ will satisfy the assumptions of the FTC, so that: +* Consider the problem of Archimedes, $\int_0^1 x^2 dx$. Clearly, we have with $f(x) = x^2$ that $F(x)=x^3/3$ will satisfy the assumptions of the FTC, so that: $$ \int_0^1 x^2 dx = F(1) - F(0) = \frac{1^3}{3} - \frac{0^3}{3} = \frac{1}{3}. $$ - * More generally, we know if $n\neq-1$ that if $f(x) = x^{n}$, that +* More generally, we know if $n\neq-1$ that if $f(x) = x^{n}$, that $$ @@ -264,7 +266,7 @@ $$ \int_a^b \frac{1}{x} dx = \log(b) - \log(a). $$ - * Let $f(x) = \cos(x)$. How much area is between $-\pi/2$ and $\pi/2$? We have that $F(x) = \sin(x)$ will have $F'(x) = f(x)$, so: +* Let $f(x) = \cos(x)$. How much area is between $-\pi/2$ and $\pi/2$? We have that $F(x) = \sin(x)$ will have $F'(x) = f(x)$, so: $$ @@ -281,7 +283,7 @@ $$ \int_a^b f(x) dx = F(b) - F(a) = F(x)\Big|_{x=a}^b, \text{ or just expr}\Big|_{x=a}^b. $$ -The vertical bar is used for the *evaluation* step, in this case the $a$ and $b$ mirror that of the definite integral. This notation lends itself to working inline, as we illustrate with this next problem where we "know" a function "$F$", so just express it "inline": +The vertical bar is used for the *evaluation* step, in this case the $a$ and $b$ mirror that of the definite integral. This notation lends itself to working "inline", as we illustrate with this next problem where we "know" a function "$F$", so just express it inline: $$ @@ -301,7 +303,7 @@ This says nothing more than $F(b)-F(a) = -F(a) - (-F(b))$, though more compactly ## The indefinite integral -A function $F(x)$ with $F'(x) = f(x)$ is known as an *antiderivative* of $f$. For a given $f$, there are infinitely many antiderivatives: if $F(x)$ is one, then so is $G(x) = F(x) + C$. But - due to the mean value theorem - all antiderivatives for $f$ differ at most by a constant. +A function $F(x)$ with $F'(x) = f(x)$ is known as an *antiderivative* of $f$. For a given $f$, there are infinitely many antiderivatives: if $F(x)$ is one, then so is $G(x) = F(x) + C$. But---due to the mean value theorem---all antiderivatives for $f$ differ at most by a constant. The **indefinite integral** of $f(x)$ is denoted by: @@ -331,7 +333,7 @@ where $C$ is the *constant of integration* and isn't really a fixed constant, bu * `integrate(ex, (var, a, b))` to find the definite integral. This integrates the expression in the variable `var` from `a` to `b`. -To illustrate, we have, this call finds an antiderivative: +To illustrate, this call finds an antiderivative for $\sin(x)$: ```{julia} @@ -339,7 +341,7 @@ To illustrate, we have, this call finds an antiderivative: integrate(sin(x),x) ``` -Whereas this call computes the "area" under $f(x)$ between `a` and `b`: +Whereas, this call computes the "area" under $f(x)$ between `a` and `b`: ```{julia} @@ -398,40 +400,42 @@ Different cases explored by `integrate` are mentioned after the questions. There are some "rules" of integration that allow indefinite integrals to be re-expressed. - * The integral of a constant times a function: - +::: {.relationship title = "The integral of a constant times a function"} $$ \int c \cdot f(x) dx = c \cdot \int f(x) dx. $$ +::: This follows as if $F(x)$ is an antiderivative of $f(x)$, then $[cF(x)]' = c f(x)$ by rules of derivatives. - * The integral of a sum of functions: +::: {.relationship title="The integral of a sum of functions"} $$ \int (f(x) + g(x)) dx = \int f(x) dx + \int g(x) dx. $$ +::: This follows immediately as if $F(x)$ and $G(x)$ are antiderivatives of $f(x)$ and $g(x)$, then $[F(x) + G(x)]' = f(x) + g(x)$, so the right hand side will have a derivative of $f(x) + g(x)$. In fact, this more general form where $c$ and $d$ are constants covers both cases and referred to by the linearity of the integral: - +::: {.relationship title="The integral of a linear combination of functions"} $$ \int (cf(x) + dg(x)) dx = c \int f(x) dx + d \int g(x) dx. $$ +::: -This statement is nothing more than the derivative formula $[cf(x) + dg(x)]' = cf'(x) + dg'(x)$. The product rule gives rise to a technique called *integration by parts* and the chain rule gives rise to a technique of *integration by substitution*, but we defer those discussions to other sections. +This statement is nothing more than the derivative formula for a linear combination of functions, $[cf(x) + dg(x)]' = cf'(x) + dg'(x)$. The product rule gives rise to a technique called *integration by parts* and the chain rule gives rise to a technique of *integration by substitution*, but we defer those discussions to other sections. ##### Examples - * The antiderivative of the polynomial $p(x) = a_n x^n + \cdots + a_1 x + a_0$ follows from the linearity of the integral and the general power rule: +* The antiderivative of the polynomial $p(x) = a_n x^n + \cdots + a_1 x + a_0$ follows from the linearity of the integral and the general power rule: $$ @@ -444,7 +448,7 @@ $$ $$ - * More generally, a [Laurent](https://en.wikipedia.org/wiki/Laurent_polynomial) polynomial allows for terms with negative powers. These too can be handled by the above. For example +* More generally, a [Laurent](https://en.wikipedia.org/wiki/Laurent_polynomial) polynomial allows for terms with negative powers. These too can be handled by the above. For example $$ @@ -475,14 +479,21 @@ So the answer to the question is $$ -\int_0^\pi 100 \sin(x) dx = (100 (-\cos(\pi))) - (100(-\cos(0))) = (100(-(-1))) - (100(-1)) = 200. +\begin{align*} +\int_0^\pi 100 \sin(x) dx &= (100 (-\cos(\pi))) - (100(-\cos(0)))\\ +&= (100(-(-1))) - (100(-1)) \\ +&= 200. +\end{align*} $$ This seems like a lot of work, and indeed it is more than is needed. The following would be more typical once the rules are learned: $$ -\int_0^\pi 100 \sin(x) dx = 100(-\cos(x)) \Big|_0^{\pi} = 100 \cos(x) \Big|_{\pi}^0 = 100(1) - 100(-1) = 200. +\begin{align*} +\int_0^\pi 100 \sin(x) dx &= 100(-\cos(x)) \Big|_0^{\pi} = 100 \cos(x) \Big|_{\pi}^0 \\ +&= 100(1) - 100(-1) = 200. +\end{align*} $$ ## The derivative of the integral @@ -769,7 +780,7 @@ xstar, x0 The asymptotic answer agrees with the answer in the first four decimal places. -As an aside, we ask how many function evaluations were taken? We can track this with a trick - using a closure to record when $f$ is called: +As an aside, we ask how many function evaluations were taken? We can track this with a trick---using a closure to record when $f$ is called: ```{julia} @@ -870,10 +881,10 @@ This has the advantage that each "dot" still represents a calorie burned, so tha Sadly though, users didn't like it. Instead of a set of dots being, say, 5 high, they were now 3 high and 2 high. It "looked" like they were doing less work! What to do? -The users actually were not responding to the number of dots, which hadn't changed, but rather the *area* that they represented - and this shrank in half. (It is much easier to visualize area than count dots when tired.) How to adjust for that? +The users actually were not responding to the number of dots, which hadn't changed, but rather the *area* that they represented---and this shrank in half. (It is much easier to visualize area than count dots when tired.) How to adjust for that? -Well our engineer knew - double the dots and count each as half a calorie. This makes the "area" constant. She also generalized letting `n` be the number of updates per minute, in anticipation of even further improvements in the display technology: +Well our engineer knew---double the dots and count each as half a calorie. This makes the "area" constant. She also generalized letting `n` be the number of updates per minute, in anticipation of even further improvements in the display technology: ```{julia} @@ -885,7 +896,7 @@ end Then the "area" represented by the dots stays fixed over this time frame. -The engineer then thought a bit more, as the form of her answer seemed familiar. She decides to parameterize it in terms of $t$ and found with $h=1/n$: `c(t) = (C(t) - C(t-h))/h`. Ahh - the derivative approximation. But then what is the "area"? It is no longer just the sum of the dots, but in terms of the functions she finds that each column represents $c(t)\cdot h$, and the sum is just $c(t_1)h + c(t_2)h + \cdots + c(t_n)h$ which looks like an approximate integral. +The engineer then thought a bit more, as the form of her answer seemed familiar. She decides to parameterize it in terms of $t$ and found with $h=1/n$: `c(t) = (C(t) - C(t-h))/h`. Ahh---the derivative approximation. But then what is the "area"? It is no longer just the sum of the dots, but in terms of the functions she finds that each column represents $c(t)\cdot h$, and the sum is just $c(t_1)h + c(t_2)h + \cdots + c(t_n)h$ which looks like an approximate integral. If the display were to reach the modern age and replace LED "dots" with a higher-pixel display, then the function to display would be $c(t) = C'(t)$ and the area displayed would be $\int_{t-10}^t c(u) du$. @@ -987,8 +998,8 @@ choices = [ "``-x^2\\cos(x) + 2x\\sin(x)``", "``-x^2\\cos(x) + 2x\\sin(x) + 2\\cos(x)``" ] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -1006,8 +1017,8 @@ choices = [ "``-(1+x) e^{-x}``", "``-(1 + x + x^2) e^{-x}``" ] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -1024,6 +1035,7 @@ val = N(integrate(exp(x) * sin(x), (x, 0, 2pi))) numericq(val) ``` + ###### Question @@ -1054,9 +1066,9 @@ numericq(f(pi/2)) ###### Question -The position of a particle is given by $x(t) = \int_0^t g(u) du$, where $x(0)=0$ and $g(u)$ is given by this piecewise linear graph: - +The position of a particle is given by $x(t) = \int_0^t g(u) du$, where $x(0)=0$ and $g(u)$ is given by the piecewise linear graph of @fig-piecewise-linear-graph-for-particle-over-0-to-5. +::: {#fig-piecewise-linear-graph-for-particle-over-0-to-5} ```{julia} #| hold: true #| echo: false @@ -1069,10 +1081,13 @@ function g1(x) 1 + (1/2)*(x-3) end end -plot(g1, 0, 5) +plot(g1, 0, 5; legend=false) ``` - * The velocity of the particle is positive over: +Plot of piecewise linear function over $[0,5]$ +::: + +* The velocity of the particle is positive over: ```{julia} @@ -1084,11 +1099,11 @@ choices = [ L"Between $0$ and $1$", L"Between $1$ and $5$" ] -answ = 4 -radioq(choices, answ, keep_order=true) +answer = 4 +radioq(choices, answer, keep_order=true) ``` - * The position of the particle is $0$ at $t=0$ and: +* The position of the particle is $0$ at $t=0$ and: ```{julia} @@ -1099,11 +1114,11 @@ choices = [ "``t=2``", "``t=3``", "``t=4``"] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` - * The position of the particle at time $t=5$ is? +* The position of the particle at time $t=5$ is? ```{julia} @@ -1113,7 +1128,7 @@ val = 4 numericq(val) ``` - * On the interval $[2,3]$: +* On the interval $[2,3]$: ```{julia} @@ -1125,8 +1140,8 @@ L"The position, $x(t)$, increases with a slope of $1$", L"The position, $x(t)$, increases quadratically from $-1/2$ to $1$", L"The position, $x(t)$, increases quadratically from $0$ to $1$" ] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -1143,8 +1158,8 @@ choices = [ "``-f(t-10)``", "``f(t) - f(t-10)``" ] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -1160,16 +1175,16 @@ choices = [ "At a critical point", L"At the endpoint $0$", L"At the endpoint $1$"] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` ###### Question -Let $F(x) = \int_0^x f(u) du$, where $f(x)$ is given by the graph below. Identify the $x$ values of all *relative maxima* of $F(x)$. Explain why you know these are the values. - +Let $F(x) = \int_0^x f(u) du$, where $f(x)$ is given by in @fig-identify-relative-maxima-from-graph-of-xs-ys. Identify the $x$ values of all *relative maxima* of $F(x)$. Explain why you know these are the values. +::: {#fig-identify-relative-maxima-from-graph-of-xs-ys} ```{julia} #| hold: true #| echo: false @@ -1178,6 +1193,9 @@ ys = [-1,0,1,0,-1,0,1/2, 0, 1/2, 0, -1] plot(xs, ys , linewidth=3, legend=false, xticks=0:10) ``` +Plot of $f(x)$ over $[-1,1]$ +::: + ```{julia} #| hold: true #| echo: false @@ -1187,8 +1205,8 @@ choices = [ "The derivative of ``F`` is ``f``, so by the second derivative test, ``x=7``", "The graph of ``f`` has relative maxima at ``x=2,6,8``" ] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question @@ -1391,8 +1409,8 @@ L"At a critical point, either $0$ or $1$", L"At a critical point, $1/2$", L"At the endpoint $0$", L"At the endpoint $1$"] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -1401,40 +1419,37 @@ radioq(choices, answ, keep_order=true) Barrow presented a version of the fundamental theorem of calculus in a 1670 volume edited by Newton, Barrow's student (cf. [Wagner](http://www.maa.org/sites/default/files/0746834234133.di020795.02p0640b.pdf)). His version can be stated as follows (cf. [Jardine](http://www.maa.org/publications/ebooks/mathematical-time-capsules)): -Consider the following figure where $f$ is a strictly increasing function with $f(0) = 0$. and $x > 0$. The function $A(x) = \int_0^x f(u) du$ is also plotted with a dashed red line. The point $Q$ is $f(x)$, and the point $P$ is $A(x)$. The point $T$ is chosen to so that the length between $T$ and $x$ times the length between $Q$ and $x$ equals the length from $P$ to $x$. ($\lvert Tx \rvert \cdot \lvert Qx \rvert = \lvert Px \rvert$.) Barrow showed that the line segment $PT$ is tangent to the graph of $A(x)$. This figure illustrates the labeling for some function: - +Consider the @fig-borrowed-from-barrow where $f$ is a strictly increasing function with $f(0) = 0$. and $x > 0$. The function $A(x) = \int_0^x f(u) du$ is also plotted with a dashed red line. The point $Q$ is $f(x)$, and the point $P$ is $A(x)$. The point $T$ is chosen to so that the length between $T$ and $x$ times the length between $Q$ and $x$ equals the length from $P$ to $x$. ($\lvert Tx \rvert \cdot \lvert Qx \rvert = \lvert Px \rvert$.) Barrow showed that the line segment $PT$ is tangent to the graph of $A(x)$. This figure illustrates the labeling for some function: +::: {#fig-borrowed-from-barrow} ```{julia} #| hold: true #| echo: false let gr() -f(x) = x^(2/3) -x = 2 -A(x) = quadgk(f, 0, x)[1] -m=f(x) -T = x - A(x)/f(x) -Q = f(x) -P = A(x) -secpt = u -> 0 + P/(x-T) * (u-T) -xs = range(0, stop=x+1/4, length=50 -) -p = plot(f, 0, x + 1/4, legend=false, line=(:black,2)) -plot!(p, A, 0, x + 1/4, line=(:red, 2,:dash)) -scatter!(p, [T, x, x, x], [0, 0, Q, P], color=:orange) -annotate!(p, collect(zip([T, x, x+.1, x+.1], [0-.15, 0-.15, Q-.1, P], [L"T", L"x", L"Q", L"P"]))) -plot!(p, [T-1/4, x+1/4], map(secpt, [T-1/4, x + 1/4]), color=:orange) -plot!(p, [T, x, x], [0, 0, P], color=:green) - + f(x) = x^(2/3) + x = 2 + A(x) = quadgk(f, 0, x)[1] + m=f(x) + T = x - A(x)/f(x) + Q = f(x) + P = A(x) + secpt = u -> 0 + P/(x-T) * (u-T) + xs = range(0, stop=x+1/4, length=50 + ) + p = plot(f, 0, x + 1/4, legend=false, line=(:black,2)) + plot!(p, A, 0, x + 1/4, line=(:red, 2,:dash)) + scatter!(p, [T, x, x, x], [0, 0, Q, P], color=:orange) + annotate!(p, collect(zip([T, x, x+.1, x+.1], [0-.15, 0-.15, Q-.1, P], [L"T", L"x", L"Q", L"P"]))) + plot!(p, [T-1/4, x+1/4], map(secpt, [T-1/4, x + 1/4]), color=:orange) + plot!(p, [T, x, x], [0, 0, P], color=:green) + plotly() p end ``` -```{julia} -#| echo: false -plotly() -nothing -``` +Plot illustrating labeling of Barrow +::: The fact that $\lvert Tx \rvert \cdot \lvert Qx \rvert = \lvert Px \rvert$ says what in terms of $f(x)$, $A(x)$ and $A'(x)$? @@ -1447,8 +1462,8 @@ choices = [ "``A(x) / \\lvert Tx \\rvert = A'(x)``", "``A(x) \\cdot A'(x) = f(x)``" ] -answ = 1 -radioq(choices, answ, keep_order=true) +answer = 1 +radioq(choices, answer, keep_order=true) ``` The fact that $\lvert PT \rvert$ is tangent says what in terms of $f(x)$, $A(x)$ and $A'(x)$? @@ -1462,8 +1477,8 @@ choices = [ "``A(x) / \\lvert Tx \\rvert = A'(x)``", "``A(x) \\cdot A'(x) = f(x)``" ] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` Solving, we get: @@ -1478,8 +1493,8 @@ choices = [ "``A'(x) = A(x)``", "``A(x) = f(x)``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1494,8 +1509,44 @@ According to [Bressoud](http://www.math.harvard.edu/~knill/teaching/math1a_2011/ choices = [ L"Part 1: $[\int_a^x f(u) du]' = f$", L"Part 2: $\int_a^b f(u) du = F(b)- F(a)$."] -answ=1 -radioq(choices, answ, keep_order=true) +answer=1 +radioq(choices, answer, keep_order=true) +``` + +###### Question + +How does the mean value theorem say that if $F(x)$ and $G(x)$ are two different antiderivatives of $f(x)$ that $F$ and $G$ differ by a constant? + +Let $H(x) = F(x) - G(x)$. What is $H'(x)$? + +```{julia} +#| echo: false +choices = ["It is hard to say, as it depends on ``x``.", +"It is ``0``."] +anwwer = 2 +explanation = raw"$H(x)$ has derivative $f'(x) - f'(x) = 0$" +buttonq(choices, answer; explanation) +``` + +The mean value theorem says that for any $a < b$ there is a $c$ in between with $H'(c) = (H(b)-H(a))/(b-a)$. Why does this imply for any $a$ and $b$ that $H(a) = H(b)$? + +```{julia} +#| echo: false +choices = [L"As $b-a < 0$ it must be that $H(b) - H(a) = 0$.", +L"It doesn't really. The value of $H(a)$ and the value of $H(b)$ can be anything." +] +answer = 1 +buttonq(choices, answer) +``` + +If $H(a) = H(b)$ for any $a < b$ then $H(x)$ must be a constant. Why? + +```{julia} +#| echo: false +choices = [L"If $H(x)$ is *not* a constant, then some $x$ satisfies $H(a) \neq H(x)$. But take $b=x$ and we see this can't be so.", + L"Because the polynomial $H(x)$ must have even degree."] +answer = 1 +buttonq(choices, answer) ``` ## More on SymPy's `integrate` diff --git a/quarto/integrals/improper_integrals.qmd b/quarto/integrals/improper_integrals.qmd index 8194e52..e6bf22c 100644 --- a/quarto/integrals/improper_integrals.qmd +++ b/quarto/integrals/improper_integrals.qmd @@ -8,8 +8,7 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia -using Plots -plotly() +using Plots; plotly() using SymPy using QuadGK ``` @@ -21,9 +20,9 @@ using QuadGK A function $f(x)$ is Riemann integrable over an interval $[a,b]$ if some limit involving Riemann sums exists. This limit will fail to exist if $f(x) = \infty$ in $[a,b]$. As well, the Riemann sum idea is undefined if either $a$ or $b$ (or both) are infinite, so the limit won't exist in this case. -To define integrals with either functions having singularities or infinite domains, the idea of an improper integral is introduced with definitions to handle the two cases above. - +To define integrals when a function has a singularity or there is an infinite domains, the idea of an improper integral is introduced. +::: {#fig-area-under-sqrt-x-animation} ```{julia} #| hold: true #| echo: false @@ -39,7 +38,7 @@ function make_sqrt_x_graph(n) f(x) = 1/sqrt(x) val = N(integrate(f(x), (x, 1/2^n, b))) - title = L"area under $f$ over $[2^{-%$n}, %$b]$ is $%$(rpad(round(val, digits=2), 4))$" + title = L"area under $f$ over $[2^{-%$n}, %$b]$ is $%$(rpad(round(val, digits=2), 4, '0'))$" plt = plot(f, range(a, stop=b, length=1000); @@ -53,11 +52,7 @@ function make_sqrt_x_graph(n) end -caption = L""" - -Area under $1/\sqrt{x}$ over $[a,b]$ increases as $a$ gets closer to $0$. Will it grow unbounded or have a limit? - -""" +caption = "" n = 10 anim = @animate for i=1:n make_sqrt_x_graph(i) @@ -69,13 +64,17 @@ plotly() ImageFile(imgfile, caption) ``` +Area under $1/\sqrt{x}$ over $[a,1]$ increases as $a$ gets closer to $0$. Will the area grow unbounded or have a limit? + +::: + ## Infinite domains Let $f(x)$ be a reasonable function, so reasonable that for any $a < b$ the function is Riemann integrable, meaning $\int_a^b f(x)dx$ exists. -What needs to be the case so that we can discuss the integral over the entire real number line? +What needs to be the case so that we can discuss the integral of $f(x)$ over the entire real number line? Clearly something. The function $f(x) = 1$ is reasonable by the idea above. Clearly the integral over $[a,b]$ is just $b-a$, but the limit over an unbounded domain would be $\infty$. Even though limits of infinity can be of interest in some cases, not so here. What will ensure that the area is finite over an infinite region? @@ -84,26 +83,32 @@ Clearly something. The function $f(x) = 1$ is reasonable by the idea above. Clea Or is that even the right question. Now consider $f(x) = \sin(\pi x)$. Over every interval of the type $[-2n, 2n]$ the area is $0$, and over any interval, $[a,b]$ the area never gets bigger than $2$. But still this function does not have a well defined area on an infinite domain. -The right question involves a limit. Fix a finite $a$. We define the definite integral over $[a,\infty)$ to be +The right approach involves a limit. +::: {.definition title="Definite integral over an unbounded domain"} + +Fix a finite $a$. We define the definite integral over $[a,\infty)$ to be $$ \int_a^\infty f(x) dx = \lim_{M \rightarrow \infty} \int_a^M f(x) dx, $$ -when the limit exists. Similarly, we define the definite integral over $(-\infty, a]$ through +when the limit exists. + +Similarly, we define the definite integral over $(-\infty, a]$ through $$ \int_{-\infty}^a f(x) dx = \lim_{M \rightarrow -\infty} \int_M^a f(x) dx. $$ -For the interval $(-\infty, \infty)$ we have need *both* these limits to exist, and then: +For the interval $(-\infty, \infty)$ we need *both* these limits to exist, and then: $$ \int_{-\infty}^\infty f(x) dx = \lim_{M \rightarrow -\infty} \int_M^a f(x) dx + \lim_{M \rightarrow \infty} \int_a^M f(x) dx. $$ +::: :::{.callout-note} ## Note @@ -118,7 +123,7 @@ When the integral exists, it is said to *converge*. If it doesn't exist, it is s $$ -\lim_{M \rightarrow \infty} \int_1^M \frac{1}{x^2}dx = \lim_{M \rightarrow \infty} -\frac{1}{x}\big|_1^M +\lim_{M \rightarrow \infty} \int_1^M \frac{1}{x^2}dx = \lim_{M \rightarrow \infty} -\frac{1}{x}\Big|_1^M = \lim_{M \rightarrow \infty} 1 - \frac{1}{M} = 1. $$ @@ -126,7 +131,7 @@ $$ $$ -\lim_{M \rightarrow \infty} \int_1^M \frac{1}{x^{1/2}}dx = \lim_{M \rightarrow \infty} \frac{x^{1/2}}{1/2}\big|_1^M +\lim_{M \rightarrow \infty} \int_1^M \frac{1}{x^{1/2}}dx = \lim_{M \rightarrow \infty} \frac{x^{1/2}}{1/2}\Big|_1^M = \lim_{M \rightarrow \infty} 2\sqrt{M} - 2 = \infty. $$ @@ -152,28 +157,37 @@ $$ for any finite $a$. This is because, $F(M) = e^x$ and this has a limit as $x$ goes to $-\infty$, but not $\infty$. - * Let $f(x) = x e^{-x^2}$. This function has an integral over $[0, \infty)$ and more generally $(-\infty, \infty)$. To see, we note that as it is an odd function, the area from $0$ to $M$ is the opposite sign of that from $-M$ to $0$. So $\lim_{M \rightarrow \infty} (F(M) - F(0)) = \lim_{M \rightarrow -\infty} (F(0) - (-F(\lvert M\lvert)))$. We only then need to investigate the one limit. But we can see by substitution with $u=x^2$, that an antiderivative is $F(x) = (-1/2) \cdot e^{-x^2}$. Clearly, $\lim_{M \rightarrow \infty}F(M) = 0$, so the answer is well defined, and the area from $0$ to $\infty$ is just $1/2$. From $-\infty$ to $0$ it is $-1/2$ and the total area is $0$, as the two sides "cancel" out. - * Let $f(x) = \sin(x)$. Even though $\lim_{M \rightarrow \infty} (F(M) - F(-M) ) = 0$, this function is not integrable. The fact is we need *both* the limit $F(M)$ and $F(-M)$ to exist as $M$ goes to $\infty$. In this case, even though the area cancels if $\infty$ is approached at the same rate, this isn't sufficient to guarantee the two limits exists independently. +* Let $f(x) = x e^{-x^2}$. This function has an integral over $[0, \infty)$ and more generally $(-\infty, \infty)$. To see, we note that as it is an odd function, the area from $0$ to $M$ is the opposite sign of that from $-M$ to $0$. So: +$$ +\lim_{M \rightarrow \infty} (F(M) - F(0)) = \lim_{M \rightarrow -\infty} (F(0) - (-F(\lvert M\lvert))). +$$ +We only then need to investigate the one limit. But we can see by substitution with $u=x^2$, that an antiderivative is $F(x) = (-1/2) \cdot e^{-x^2}$. Clearly, $\lim_{M \rightarrow \infty}F(M) = 0$, so the answer is well defined, and the area from $0$ to $\infty$ is just $1/2$. From $-\infty$ to $0$ it is $-1/2$ and the total area is $0$, as the two sides "cancel" out. + +* Let $f(x) = \sin(x)$. Even though $\lim_{M \rightarrow \infty} (F(M) - F(-M) ) = 0$, this function is not integrable. The fact is we need *both* the limit $F(M)$ and $F(-M)$ to exist as $M$ goes to $\infty$. In this case, even though the area cancels if $\infty$ is approached at the same rate, this isn't sufficient to guarantee the two limits exists independently. - * Will the function $f(x) = 1/(x\cdot(\log(x))^2)$ have an integral over $[e, \infty)$? + +* Will the function $f(x) = 1/(x\cdot(\log(x))^2)$ have an integral over $[e, \infty)$? We first find an antiderivative using the $u$-substitution $u(x) = \log(x)$: $$ +\begin{align*} \int_e^M \frac{1}{x \log(x)^{2}} dx -= \int_{\log(e)}^{\log(M)} \frac{1}{u^{2}} du -= \frac{-1}{u} \big|_{1}^{\log(M)} -= \frac{-1}{\log(M)} - \frac{-1}{1} -= 1 - \frac{1}{\log(M)}. +&= \int_{\log(e)}^{\log(M)} \frac{1}{u^{2}} du\\ +&= \frac{-1}{u} \Big|_{1}^{\log(M)}\\ +&= \frac{-1}{\log(M)} - \frac{-1}{1}\\ +&= 1 - \frac{1}{\log(M)}. +\end{align*} $$ As $M$ goes to $\infty$, this will converge to $1$. - * The sinc function $f(x) = \sin(\pi x)/(\pi x)$ does not have a nice antiderivative. Seeing if the limit exists is a bit of a problem. However, this function is important enough that there is a built-in function, `Si`, that computes $\int_0^x \sin(u)/u\cdot du$. This function can be used through `sympy.Si(...)`: + +* The sinc function $f(x) = \sin(\pi x)/(\pi x)$ does not have a nice antiderivative. Seeing if the limit exists is a bit of a problem^[Well, we could break the answer into an alternating series with clearly shrinking terms, so this is a bit of an exaggeration.]. However, this function is important enough that there is a built-in function, `Si`, that computes $\int_0^x \sin(u)/u\cdot du$. This function can be used through `sympy.Si(...)`: ```{julia} @@ -192,13 +206,13 @@ $$ we introduce a trick and rely on some theorems that have not been discussed. -First, we notice that $\Si(x)$ is the value of $I(\alpha)$ when $\alpha=0$ where +First, we notice that $\text{Si}(x)$ is the value of $I(\alpha)$ when $\alpha=0$ where $$ I(\alpha) = \int_0^\infty \exp(-\alpha t) \frac{\sin(t)}{t} dt $$ -We differentiate $I$ in $\alpha$ to get: +We differentiate $I$ in $\alpha$ to get:^[This is a bit of a fast one, as we move the derivative *inside* the integral.] $$ \begin{align*} @@ -217,8 +231,8 @@ $$ &=\sin(t) \frac{-\exp(-\alpha t)}{\alpha} \Big|_0^\infty - \int_0^\infty \frac{-\exp(-\alpha t)}{\alpha} \cos(t) dt \\ &= 0 + \frac{1}{\alpha} \cdot \int_0^\infty \exp(-\alpha t) \cos(t) dt \\ -&= \frac{1}{\alpha} \cdot \cos(t)\frac{-\exp(-\alpha t)}{\alpha} \Big|_0^\infty - -\frac{1}{\alpha} \cdot \int_0^\infty \frac{-\exp(-\alpha t)}{\alpha} (-\sin(t)) dt \\ +&= \frac{1}{\alpha} \cdot \cos(t)\frac{-\exp(-\alpha t)}{\alpha} \Big|_0^\infty -\\ +&\quad\frac{1}{\alpha} \cdot \int_0^\infty \frac{-\exp(-\alpha t)}{\alpha} (-\sin(t)) dt \\ &= \frac{1}{\alpha^2} - \frac{1}{\alpha^2} \cdot \int_0^\infty \exp(-\alpha t) \sin(t) dt \end{align*} $$ @@ -232,7 +246,7 @@ $$ Solving gives the desired integral as $$ -I'(\alpha) = -\frac{1}{\alpha^2} / (1 + \frac{1}{\alpha^2}) = -\frac{1}{1 + \alpha^2}. +I'(\alpha) = -\frac{1}{\alpha^2} / \left(1 + \frac{1}{\alpha^2}\right) = -\frac{1}{1 + \alpha^2}. $$ @@ -242,14 +256,15 @@ As our question is answered by $I(0)$, we get $I(0) = \tan^{-1}(0) + C = C = \pi The above argument requires two places where a *limit* is passed inside the integral. The first involved the derivative. The [Leibniz integral rule](https://en.wikipedia.org/wiki/Leibniz_integral_rule) can be used to verify the first use is valid: -:::{.callout-note icon=false} -## Leibniz integral rule +:::{.theorem title="Leibniz integral rule"} If $f(x,t)$ and the derivative in $x$ for a fixed $t$ is continuous (to be discussed later) in a region containing $a(x) \leq t \leq b(x)$ and $x_0 < x < x_1$ and both $a(x)$ and $b(x)$ are continuously differentiable, then $$ -\frac{d}{dx}\int_{a(x)}^{b(x)} f(x, t) dt = -\int_{a(x)}^{b(x)} \frac{d}{dx}f(x,t) dt + -f(x, b(x)) \frac{d}{dx}b(x) - f(x, a(x)) \frac{d}{dx}a(x). +\begin{align*} +\frac{d}{dx}\int_{a(x)}^{b(x)} f(x, t) dt +&= \int_{a(x)}^{b(x)} \frac{d}{dx}f(x,t) dt \\ +&\quad + f(x, b(x)) \frac{d}{dx}b(x) - f(x, a(x)) \frac{d}{dx}a(x). +\end{align*} $$ ::: @@ -296,7 +311,7 @@ Suppose $a < c$, we define $\int_a^c f(x) dx = \lim_{M \rightarrow c-} \int_a^M $$ \lim_{M \rightarrow 0+} \int_M^1 \frac{1}{\sqrt{x}} dx -= \lim_{M \rightarrow 0+} \frac{\sqrt{x}}{1/2} \big|_M^1 += \lim_{M \rightarrow 0+} \frac{\sqrt{x}}{1/2} \Big|_M^1 = \lim_{M \rightarrow 0+} 2(1) - 2\sqrt{M} = 2. $$ @@ -311,7 +326,7 @@ The cases $f(x) = x^{-n}$ for $n > 0$ are tricky to keep straight. For $n > 1$, $$ \lim_{M \rightarrow 0+} \int_M^1 \frac{1}{x} dx -= \lim_{M \rightarrow 0+} \log(x) \big|_M^1 += \lim_{M \rightarrow 0+} \log(x) \Big|_M^1 = \lim_{M \rightarrow 0+} \log(1) - \log(M) = \infty. $$ @@ -363,7 +378,7 @@ A probability density is a function $f(x) \geq 0$ which is integrable on $(-\inf Probability densities are good example of using improper integrals. - * Show that $f(x) = (1/\pi) (1/(1 + x^2))$ is a probability density function. +* Show that $f(x) = (1/\pi) (1/(1 + x^2))$ is a probability density function. We need to show that the integral exists and is $1$. For this, we use the fact that $(1/\pi) \cdot \tan^{-1}(x)$ is an antiderivative. Then we have: @@ -376,7 +391,7 @@ $$ and as $\tan^{-1}(x)$ is odd, we must have $F(-\infty) = \lim_{M \rightarrow -\infty} f(M) = -(1/\pi) \cdot \pi/2$. All told, $F(\infty) - F(-\infty) = 1/2 - (-1/2) = 1$. - * Show that $f(x) = 1/(b-a)$ for $a \leq x \leq b$ and $0$ otherwise is a probability density. +* Show that $f(x) = 1/(b-a)$ for $a \leq x \leq b$ and $0$ otherwise is a probability density. The integral for $-\infty$ to $a$ of $f(x)$ is just an integral of the constant $0$, so will be $0$. (This is the only constant with finite area over an infinite domain.) Similarly, the integral from $b$ to $\infty$ will be $0$. This means: @@ -389,10 +404,10 @@ $$ (One might also comment that $f$ is Riemann integrable on any $[0,M]$ despite being discontinuous at $a$ and $b$.) - * Show that if $f(x)$ is a probability density then so is $f(x-c)$ for any $c$. +* Show that if $f(x)$ is a probability density then so is $f(x-c)$ for any $c$. -We have by the $u$-substitution +We have by the $u$-substitution $u(x)=x-c$ that $$ @@ -402,7 +417,7 @@ $$ The key is that we can use the regular $u$-substitution formula provided $\lim_{M \rightarrow \infty} u(M) = u(\infty)$ is defined. (The *informal* notation $u(\infty)$ is defined by that limit.) - * If $f(x)$ is a probability density, then so is $(1/h) f((x-c)/h)$ for any $c, h > 0$. +* If $f(x)$ is a probability density, then so is $(1/h) f((x-c)/h)$ for any $c, h > 0$. Again, by a $u$ substitution with, now, $u(x) = (x-c)/h$, we have $du = (1/h) \cdot dx$ and the result follows just as before: @@ -412,10 +427,10 @@ $$ \int_{-\infty}^\infty \frac{1}{h}f(\frac{x-c}{h})dx = \int_{u(-\infty)}^{u(\infty)} f(u) du = \int_{-\infty}^\infty f(u) du = 1. $$ - * If $F(x) = 1 - e^{-x}$, for $x \geq 0$, and $0$ otherwise, find $f(x)$. +* If $F(x) = 1 - e^{-x}$, for $x \geq 0$, and $0$ otherwise, find $f(x)$. -We want to just say $F'(x)= e^{-x}$ so $f(x) = e^{-x}$. But some care is needed. First, that isn't right. The derivative for $x<0$ of $F(x)$ is $0$, so $f(x) = 0$ if $x < 0$. What about for $x>0$? The derivative is $e^{-x}$, but is that the right answer? $F(x) = \int_{-\infty}^x f(u) du$, so we have to at least discuss if the $-\infty$ affects things. In this case, and in general the answer is *no*. For any $x$ we can find $M < x$ so that we have $F(x) = \int_{-\infty}^M f(u) du + \int_M^x f(u) du$. The first part is a constant, so will have derivative $0$, the second will have derivative $f(x)$, if the derivative exists (and it will exist at $x$ if the derivative is continuous in a neighborhood of $x$). +We want to just say $F'(x)= e^{-x}$ so $f(x) = e^{-x}$. But some care is needed. First, that isn't right. The derivative for $x<0$ of $F(x)$ is $0$, so $f(x) = 0$ if $x < 0$. What about for $x>0$? The derivative is $e^{-x}$, but is that the right answer? $F(x) = \int_{-\infty}^x f(u) du$, so we have to at least discuss if the $-\infty$ affects things. In this case, and in general, the answer is *no*. For any $x$ we can find $M < x$ so that we have $F(x) = \int_{-\infty}^M f(u) du + \int_M^x f(u) du$. The first part is a constant, so will have derivative $0$, the second will have derivative $f(x)$, if the derivative exists (and it will exist at $x$ if the derivative is continuous in a neighborhood of $x$). Finally, at $x=0$ we have an issue, as $F'(0)$ does not exist. The left limit of the secant line approximation is $0$, the right limit of the secant line approximation is $1$. So, we can take $f(x) = e^{-x}$ for $x > 0$ and $0$ otherwise, noting that redefining $f(x)$ at a point will not effect the integral as long as the point is finite. @@ -424,16 +439,45 @@ Finally, at $x=0$ we have an issue, as $F'(0)$ does not exist. The left limit of ## Application to series -In this application, we compare a series to a related integral to decide convergence or divergence of the series. +In this application, we compare a series to a related integral to decide convergence or divergence of the series. @fig-integral-test-figure motivates the following theorem. + +:::{.theorem title="The integral test"} -:::{.callout-note appearance="minimal"} -#### The integral test Consider a continuous, monotone decreasing function $f(x)$ defined on some interval of the form $[N,\infty)$. Let $a_n = f(n)$ and $s_n = \sum_{k=N}^n a_n$. * If $\int_N^\infty f(x) dx < \infty$ then the partial sums converge. * If $\int_N^\infty f(x) dx = \infty$ then the partial sums diverge. ::: +::: {#fig-integral-test-figure} +```{julia} +#| echo: false +let + # integral test + gr() + f(x) = 1/x + p1 = plot(; legend=false, framestyle=:origin, xticks=1:8, yaxis=([], false)) + p2 = plot(; legend=false, framestyle=:origin, xticks=1:8, yaxis=([], false)) + plot!(p1, f, 0.75, 8.25; line=(1, :black)) + plot!(p2, f, 0.75, 8.25; line=(1, :black)) + + for k in 1:7 + plot!(p1, [(k,0), (k+1,0), (k+1, f(k)), (k, f(k)), (k,0)]; line=(1, :black, :dot)) + annotate!(p1, [(k+1/2, f(k+1)/2, latexstring("a_{$k}"))]) + end + + for k in 1:7 + plot!(p2, [(k,0), (k+1,0), (k+1, f(k+1)), (k, f(k+1)),(k,0)]; line=(1, :black, :dot)) + annotate!(p2, [(k+1/2, f(k+2)/2, latexstring("a_{$(k+1)}"))]) + end + plotly() + plot(p1, p2) +end +``` + +Illustration of the integral test, where a series $a_1 + a_2 + \cdots$ is bounded *below* by $\int_1^\infty f(x)dx$ and the series $a_2 + a_3 + \cdots$ is bounded *above* by $\int_1^\infty f(x)dx$ where $a_i=f(i)$. In either case, converge/divergence of the integral forces convergence/divergence of the series. +::: + By the monotone nature of $f(x)$, we have on any interval of the type $[i, i+1)$ for $i$ an integer, that $f(i) \geq f(x) \geq f(i+1)$ when $x$ is in the interval. For integrals, this leads to $$ @@ -491,8 +535,7 @@ That this is finite shows the series converges. The integral of a power series can be computed easily for some $x$: -:::{.callout-note appearance="minimal"} -### The integral of a power series +:::{.theorem title="The integral of a power series"} Suppose $f(x) = \sum_n a_n (x-c)^n$ is a power series about $x=c$ with radius of convergence $r > 0$. [Then](https://en.wikipedia.org/wiki/Power_series#Differentiation_and_integration) the limits of the integral and the sum can be switched around when $x$ is within the radius of convergence: @@ -501,7 +544,7 @@ $$ \int f(x) dx &= \int \sum_n a_n(x-c)^n dx\\ &= \sum_{n=0}^\infty \int a_n(x-c)^n dx\\ -= \sum_{n=0}^\infty a_n \frac{(x-c)^{n+1}}{n+1} +&= \sum_{n=0}^\infty a_n \frac{(x-c)^{n+1}}{n+1} \end{align*} $$ @@ -647,14 +690,39 @@ val, _ = quadgk(f , 0, 1, 2) numericq(val) ``` +###### Question + +Consider the integral $\int_{-\infty}^\infty f(x) dx$. We do a change of variable with $x = t/(1-t^2)$. This gives: + +$$ +\int_a^b f\left(\frac{t}{1-t^2}\right) \frac{t^2 + 1}{t^2 - 1} dt +$$ + +What are the values of $a$ and $b$? + +```{julia} +#| echo: false +choices = [ +L"$a=-1$ and $b=1$", +L"$a=1$ and $b=-1$", +L"$a=1$ and $b=0$", +L"$a=0$ and $b=1$", +] +answer = 1 +explanation = "As ``t`` goes to ``1`` from the left, ``x`` goes to what?" +buttonq(choices, answer; explanation) +``` + + ###### Question From the relationship that if $0 \leq f(x) \leq g(x)$ then $\int_a^b f(x) dx \leq \int_a^b g(x) dx$ it can be deduced that - * if $\int_a^\infty f(x) dx$ diverges, then so does $\int_a^\infty g(x) dx$. - * if $\int_a^\infty g(x) dx$ converges, then so does $\int_a^\infty f(x) dx$. +* if $\int_a^\infty f(x) dx$ diverges, then so does $\int_a^\infty g(x) dx$. + +* if $\int_a^\infty g(x) dx$ converges, then so does $\int_a^\infty f(x) dx$. Let $f(x) = \lvert \sin(x)/x^2 \rvert$. @@ -670,8 +738,8 @@ choices =[ "It is convergent", "It is divergent", "Can't say"] -answ = 1 -radioq(choices, answ, keep_order=true) +answer = 1 +radioq(choices, answer, keep_order=true) ``` --- @@ -690,8 +758,8 @@ choices =[ "It is convergent", "It is divergent", "Can't say"] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` --- @@ -707,8 +775,8 @@ choices =[ "It is convergent", "It is divergent", "Can't say"] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` --- @@ -724,8 +792,8 @@ choices =[ "It is convergent", "It is divergent", "Can't say"] -answ = 1 -radioq(choices, answ, keep_order=true) +answer = 1 +radioq(choices, answer, keep_order=true) ``` --- @@ -741,8 +809,8 @@ choices =[ "It is convergent", "It is divergent", "Can't say"] -answ = 1 -radioq(choices, answ, keep_order=true) +answer = 1 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -759,8 +827,8 @@ choices = [ "``\\int_0^1 u^{2/3} \\cdot du``", "``\\int_0^\\infty 1/u \\cdot du``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question diff --git a/quarto/integrals/integration_by_parts.qmd b/quarto/integrals/integration_by_parts.qmd index a7e0a5b..0ea1770 100644 --- a/quarto/integrals/integration_by_parts.qmd +++ b/quarto/integrals/integration_by_parts.qmd @@ -8,8 +8,7 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia -using Plots -plotly() +using Plots; plotly() using SymPy ``` @@ -26,81 +25,82 @@ nothing So far we have seen that the *derivative* rules lead to *integration rules*. In particular: - * The sum rule $[au(x) + bv(x)]' = au'(x) + bv'(x)$ gives rise to an integration rule: $\int (au(x) + bv(x))dx = a\int u(x)dx + b\int v(x))dx$. (That is, the linearity of the derivative means the integral has linearity.) +* The sum rule $[au(x) + bv(x)]' = au'(x) + bv'(x)$ gives rise to an integration rule: $\int (au(x) + bv(x))dx = a\int u(x)dx + b\int v(x))dx$. (That is, the linearity of the derivative means the integral has linearity.) - * The chain rule $[f(g(x))]' = f'(g(x)) g'(x)$ gives $\int_a^b f(g(x))g'(x)dx=\int_{g(a)}^{g(b)}f(x)dx$. That is, substitution reverses the chain rule. + +* The chain rule $[g(u(x))]' = g'(u(x)) u'(x)$ gives $\int_a^b g(u(x))u'(x)dx=\int_{u(a)}^{u(b)}g(x)dx$. That is, $u$-substitution reverses the chain rule. Now we turn our attention to the implications of the *product rule*: $[uv]' = u'v + uv'$. The resulting technique is called integration by parts. -::: {.callout-note} -## Integration by parts +::: {.definition title="Integration by parts"} By the fundamental theorem of calculus: $$ -[u(x)\cdot v(x)]\Big|_a^b = \int_a^b [u(x) v(x)]' dx = \int_a^b u'(x) \cdot v(x) dx + \int_a^b u(x) \cdot v'(x) dx. +\begin{align*} +\left[u(x)\cdot v(x)\right]\Big|_a^b &= \int_a^b [u(x) v(x)]' dx\\ +&= \int_a^b u'(x) \cdot v(x) dx + \int_a^b u(x) \cdot v'(x) dx. +\end{align*} $$ Or, $$ -\int_a^b u(x) v'(x) dx = [u(x)v(x)]\Big|_a^b - \int_a^b v(x) u'(x)dx. +\int_a^b u(x) v'(x) dx = \left[u(x)v(x)\right]\Big|_a^b - \int_a^b v(x) u'(x)dx. $$ ::: -The following visually illustrates integration by parts: +@fig-visualize-integration-by-parts illustrates integration by parts showing a parametric plot of $(u(t),v(t))$ for $a \leq t \leq b$. +::: {#fig-visualize-integration-by-parts} ```{julia} #| echo: false -#| label: fig-integration-by-parts -#| fig-cap: "Integration by parts figure ([original](http://en.wikipedia.org/wiki/Integration_by_parts#Visualization))" let ## parts picture gr() -u(x) = sin(x*pi/2) -v(x) = x -xs = range(0, stop=1, length=50) -a,b = 1/4, 3/4 + u(x) = sin(x*pi/2) + v(x) = x + xs = range(0, stop=1, length=50) + a,b = 1/4, 3/4 -p = plot(u, v, 0, 1; legend=false, axis=([], false), line=(:black,2)) -plot!([0, u(1)], [0,0]; line=(:gray, 1), arrow=true, side=:head) -plot!([0, 0], [0, v(1) ]; line=(:gray, 1), arrow=true, side=:head) + p = plot(u, v, 0, 1; legend=false, axis=([], false), line=(:black,2)) + plot!([0, u(1)], [0,0]; line=(:gray, 1), arrow=true, side=:head) + plot!([0, 0], [0, v(1) ]; line=(:gray, 1), arrow=true, side=:head) -xs′ = range(a, b, length=50) -plot!(Shape(vcat(u.(xs′), reverse(u.(xs′))), - vcat(zero.(xs′), v.(reverse(xs′)))), - fill=(:red, 0.15), - xlims=(-0.07, 1) - ) -plot!(Shape([0,u(a),u(a),0],[0,0,v(a),v(a)]), fill=(:royalblue, 0.5)) -scatter!(p, [u(a), u(b)], [v(a), v(b)], color=:mediumorchid3, markersize=5) -plot!(p, [u(a),u(a),0, 0, u(b),u(b),u(a)], - [0, v(a), v(a), v(b), v(b), 0, 0], - linetype=:polygon, fill=(:brown3, 0.25)) + xs′ = range(a, b, length=50) + plot!(Shape(vcat(u.(xs′), reverse(u.(xs′))), + vcat(zero.(xs′), v.(reverse(xs′)))), + fill=(:red, 0.15), + xlims=(-0.07, 1) + ) + plot!(Shape([0,u(a),u(a),0],[0,0,v(a),v(a)]), fill=(:royalblue, 0.5)) + scatter!(p, [u(a), u(b)], [v(a), v(b)], color=:mediumorchid3, markersize=5) + plot!(p, [u(a),u(a),0, 0, u(b),u(b),u(a)], + [0, v(a), v(a), v(b), v(b), 0, 0], + linetype=:polygon, fill=(:brown3, 0.25)) -annotate!(p, [(0.65, .25, text(L"A")), - (0.4, .55, text(L"B")), - (u(a),v(a), text(L"(u(a),v(a))", :bottom, :right)), + annotate!(p, [(0.65, .25, text(L"A")), + (0.4, .55, text(L"B")), + (u(a),v(a), text(L"(u(a),v(a))", :bottom, :right)), (u(b),v(b), text(L"(u(b),v(b))", :bottom, :right)), - (u(a),0, text(L"u(a)", :top)), - (u(b),0, text(L"u(b)", :top)), + (u(a),0, text(L"u(a)", :top)), + (u(b),0, text(L"u(b)", :top)), (0, v(a), text(L"v(a)", :right)), (0, v(b), text(L"v(b)", :right)), - (0,0, text(L"(0,0)", :top)) - ]) + (0,0, text(L"(0,0)", :top)) + ]) + + plotly() + p end ``` -```{julia} -#| echo: false -plotly() -nothing -``` +Integration by parts figure ([original](http://en.wikipedia.org/wiki/Integration_by_parts#Visualization)) +::: -@fig-integration-by-parts shows a parametric plot of $(u(t),v(t))$ for $a \leq t \leq b$.. The total shaded area, a rectangle, is $u(b)v(b)$, the area of $A$ and $B$ combined is just $u(b)v(b) - u(a)v(a)$ or $[u(x)v(x)]\Big|_a^b$. We will show that $A$ is $\int_a^b v(x)u'(x)dx$ and $B$ is $\int_a^b u(x)v'(x)dx$ giving the formula. @@ -225,7 +225,7 @@ $$ \int_a^b x^2 e^x dx = (x^2 \cdot e^x)\Big|_a^b - \int_a^b 2x e^x dx. $$ -But we can do $\int_a^b x e^xdx$ the same way: +But we can compute $\int_a^b x e^xdx$ the same way: $$ @@ -245,8 +245,8 @@ In fact, it isn't hard to see that an integral of $x^m e^x$, $m$ a positive inte ```{julia} -@syms 𝒙 -integrate(𝒙^10 * exp(𝒙), 𝒙) +@syms x +integrate(x^10 * exp(x), x) ``` The general answer is $\int x^n e^xdx = p(x) e^x$, where $p(x)$ is a polynomial of degree $n$. @@ -281,7 +281,10 @@ So: $$ -\int e^x \sin(x)dx = \sin(x) e^x - \int \cos(x) e^x dx = \sin(x)e^x - \cos(x)e^x + \int (-\sin(x))e^x dx. +\begin{align*} +\int e^x \sin(x)dx &= \sin(x) e^x - \int \cos(x) e^x dx \\ +&= \sin(x)e^x - \cos(x)e^x + \int (-\sin(x))e^x dx. +\end{align*} $$ But simplifying this gives: @@ -324,11 +327,11 @@ $$ This is called a reduction formula as it reduces the problem from an integral with a power of $n$ to one with a power of $n - 2$, so could be repeated until the remaining indefinite integral required knowing either $\int \cos(x) dx$ (which is $-\sin(x)$) or $\int \cos(x)^2 dx$, which by a double angle formula application, is $x/2 + \sin(2x)/4$. -`SymPy` is able and willing to do this repeated bookkeeping. For example with $n=10$: +`SymPy` is willing and able to do this repeated bookkeeping. For example with $n=10$: ```{julia} -integrate(cos(𝒙)^10, 𝒙) +integrate(cos(x)^10, x) ``` ##### Example @@ -367,7 +370,7 @@ $$ Using right triangles to simplify, the last value $\cos(\sin^{-1}(x))$ can otherwise be written as $\sqrt{1 - x^2}$. -##### Example +##### Example: maximum error in the trapezoid rule The [trapezoid](http://en.wikipedia.org/wiki/Trapezoidal_rule) rule is an approximation to the definite integral like a Riemann sum, only instead of approximating the area above $[x_i, x_i + h]$ by a rectangle with height $f(c_i)$ (for some $c_i$), it uses a trapezoid formed by the left and right endpoints. That is, this area is used in the estimation: $(1/2)\cdot (f(x_i) + f(x_i+h)) \cdot h$. @@ -393,14 +396,17 @@ $$ $$ -We choose $A$ to be $-h/2$, any constant is possible, for then the term $f(t+x_i)(t+A)\Big|_0^h$ becomes $(1/2)(f(x_i+h) + f(x_i)) \cdot h$, or the trapezoid approximation. This means, the error over this interval - actual minus estimate - satisfies: +We choose $A$ to be $-h/2$, any constant is possible, for then the term $f(t+x_i)(t+A)\Big|_0^h$ becomes $(1/2)(f(x_i+h) + f(x_i)) \cdot h$, or the trapezoid approximation. This means, the error over this interval---actual minus estimate---satisfies: $$ -\text{error}_i = \int_{x_i}^{x_i+h}f(x) dx - \frac{f(x_i+h) -f(x_i)}{2} \cdot h = - \int_0^h (t + A) f'(t + x_i) dt. +\begin{align*} +\text{error}_i &= \int_{x_i}^{x_i+h}f(x) dx - \frac{f(x_i+h) -f(x_i)}{2} \cdot h \\ +&= - \int_0^h (t + A) f'(t + x_i) dt. +\end{align*} $$ -For this, we *again* integrate by parts with +To compute this, we *again* integrate by parts with $$ @@ -415,7 +421,10 @@ Again we added a constant of integration, $B$, to $v$. The error becomes: $$ -\text{error}_i = -\left(\frac{(t+A)^2}{2} + B\right)f'(t+x_i)\Big|_0^h + \int_0^h \left(\frac{(t+A)^2}{2} + B\right) \cdot f''(t+x_i) dt. +\begin{align*} +\text{error}_i &= -\left(\frac{(t+A)^2}{2} + B\right)f'(t+x_i)\Big|_0^h \\ +&\quad + \int_0^h \left(\frac{(t+A)^2}{2} + B\right) \cdot f''(t+x_i) dt. +\end{align*} $$ With $A=-h/2$, $B$ is chosen so $(t+A)^2/2 + B = 0$ at endpoints, or $B=-h^2/8$. The error becomes @@ -429,7 +438,7 @@ Now, we assume the $\lvert f''(t)\rvert$ is bounded by $K$ for any $a \leq t \le $$ -\lvert \text{error}_i \rvert \leq K \int_0^h \lVert \left(\frac{(t-h/2)^2}{2} - \frac{h^2}{8}\right) \rVert dt. +\lvert \text{error}_i \rvert \leq K \int_0^h \lvert \left(\frac{(t-h/2)^2}{2} - \frac{h^2}{8}\right) \rvert dt. $$ But what is the function in the integrand? Clearly it is a quadratic in $t$. Expanding gives $1/2 \cdot (t^2 - ht)$. This is negative over $[0,h]$ (and $0$ at these endpoints, so the integral above is just: @@ -446,7 +455,7 @@ $$ \lvert \text{error}\rvert \leq n \cdot \frac{Kh^3}{12} = \frac{K(b-a)^3}{12}\frac{1}{n^2}. $$ -So the error is like $1/n^2$, in contrast to the $1/n$ error of the Riemann sums. One way to see this, for the Riemann sum it takes twice as many terms to half an error estimate, but for the trapezoid rule only $\sqrt{2}$ as many, and for Simpson's rule, only $2^{1/4}$ as many. +So the maximum error^[This is a worst-case estimate. There are functions for which the trapezoid method converges exponentially and are discussed in [Trefethen and Weiderman](https://people.maths.ox.ac.uk/trefethen/sirev56-3_385.pdf).] is like $1/n^2$, in contrast to the $1/n$ error of the Riemann sums. One way to see this, for the Riemann sum it takes twice as many terms to half an error estimate, but for the trapezoid rule only $\sqrt{2}$ as many, and for Simpson's rule, only $2^{1/4}$ as many. ## Area related to parameterized curves @@ -458,9 +467,9 @@ The figure introduced to motivate the integration by parts formula also suggests When $u(t)$ is strictly *increasing*, and hence having an inverse function, then re-parameterizing by $\phi(t) = u^{-1}(t)$ gives a $x=u(u^{-1}(t))=t, y=v(u^{-1}(t))$ and integrating this gives the area by $A=\int_a^b v(t) u'(t) dt$ -However, the correct answer requires understanding a minus sign. Consider the area enclosed by $x(t) = \cos(t), y(t) = \sin(t)$: - +However, the correct answer requires understanding a minus sign. Consider the area enclosed by $x(t) = \cos(t), y(t) = \sin(t)$ in @fig-area-parameterized-curve-example. +::: {#fig-area-parameterized-curve-example} ```{julia} #| echo: false let @@ -477,6 +486,9 @@ let end ``` +Area consideration for a parameterized curve +::: + We added a rectangle for a Riemann sum for $t_i = \pi/3$ and $t_{i+1} = \pi/3 + \pi/8$. The height of this rectangle is $y(t_i)$, the base is of length $x(t_i) - x(t_{i+1})$ *given* the orientation of how the circular curve is parameterized (counter clockwise here). @@ -488,16 +500,18 @@ $$ A &\approx \sum_i y(t_i) \cdot (x(t_{i}) - x(t_{i+1}))\\ &= - \sum_i y(t_i) \cdot (x(t_{i+1}) - x(t_{i}))\\ &= - \sum_i y(t_i) \cdot \frac{x(t_{i+1}) - x(t_i)}{t_{i+1}-t_i} \cdot (t_{i+1}-t_i)\\ - &\approx -\int_a^b y(t) x'(t) dt. + &\approx -\int_a^b y(t) x'(t) dt\\ + &= \int_a^b x(t) y'(t) dt. \end{align*} $$ - -So with a counterclockwise rotation, the actual answer for the area includes a minus sign. If the area is traced out in a *clockwise* manner, there is no minus sign. - +The last line using integration by parts to reverse the role of $x$ and $y$. +When traversing a curve in a counter clockwise manner $\int x(t) y'(t) dt$ has no minus sign and when traversing in a clockwise manner $\int y(t) x'(t) dt$ has no minus sign. This is a case of [Green's Theorem](https://en.wikipedia.org/wiki/Green%27s_theorem#Area_calculation) to be taken up in [Green's Theorem, Stokes' Theorem, and the Divergence Theorem](file:///Users/verzani/julia/CalculusWithJulia/html/integral_vector_calculus/stokes_theorem.html). +We also revisit this in the section on the area between curves where this formula comes from decomposing the area contained within a simple polygon using trapezoids. + ##### Example @@ -510,7 +524,7 @@ Apply the formula to a parameterized circle to ensure, the signed area is proper @syms r t x = r * cos(t) y = r * sin(t) --integrate(y * diff(x, t), (t, 0, 2PI)) +integrate(x * diff(y, t), (t, 0, 2PI)) ``` We see the expected answer for the area of a circle. @@ -539,9 +553,9 @@ integrate(y * diff(x, t), (t, 0, 2PI)) ##### Example -Consider the example $x(t) = \cos(t) + t\sin(t), y(t) = \sin(t) - t\cos(t)$ for $0 \leq t \leq 2\pi$. - +Consider the example $x(t) = \cos(t) + t\sin(t), y(t) = \sin(t) - t\cos(t)$ for $0 \leq t \leq 2\pi$ shown in @fig-area-spiraling-curve-over-0-2pi. +::: {#fig-area-spiraling-curve-over-0-2pi} ```{julia} #| echo: false let @@ -551,17 +565,20 @@ let plot(x.(ts), y.(ts)) end ``` +Plot of parameterized spiraling curve over $[0, 2\pi]$ +::: + How much area is enclosed by this curve and the $x$ axis? The area is described in a counterclockwise manner, so we have: ```{julia} #| hold: true -let +let # a let block avoids the issue that `x` has already been used as a constant x(t) = cos(t) + t*sin(t) y(t) = sin(t) - t*cos(t) - yx′(t) = -y(t) * x'(t) # yx\prime[tab] - quadgk(yx′, 0, 2pi) + xy′(t) = x(t) * y'(t) # xy\prime[tab] + quadgk(xy′, 0, 2pi) end ``` @@ -584,8 +601,8 @@ choices = [ "``du=1/x dx \\quad v = x``", "``du=x\\log(x) dx\\quad v = 1``", "``du=1/x dx\\quad v = x^2/2``"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -602,8 +619,8 @@ choices = [ "``du=\\csc(x) dx \\quad v=\\sec(x)^3 / 3``", "``du=\\tan(x) dx \\quad v=\\sec(x)\\tan(x)``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -620,8 +637,8 @@ choices = [ "``du=-e^{-x} dx \\quad v=-\\sin(x)``", "``du=\\sin(x)dx \\quad v=-e^{-x}``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -683,10 +700,56 @@ choices = [ "``\\int (\\log(x))^{n+1}/(n+1) dx``", "``x(\\log(x))^n - \\int (\\log(x))^{n-1} dx``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` +###### Question + +Let $k$ be a positive integer. We can use integration by parts to understand the value of + +$$ +I_k = \int x^k \exp(-x) dx +$$ + +Let $u = x^k$ and $dv = \exp(-x)$. What becomes of $uv$? + +```{julia} +#| echo: false +choices = [L"-kx^{k-1} \exp(-x)", + L"-x^{k+1}/(k+1) \exp(-x)", + L"-x^k \exp(-x)"] +answer = 3 +buttonq(choices, answer) +``` + +What becomes of $\int v du$? + +```{julia} +#| echo: false +choices = [L"\int k x^{k-1} (-\exp(-x)) dx", + L"\int x^{k+1}/(k+1) (-\exp(-x)) dx", + L"\int x^k (-\exp(-x)) dx"] +answer = 1 +buttonq(choices, answer) +``` + +Does this show that $I_k = -x^k \exp(-x) + k I_{k-1}$? + +```{julia} +#| echo: false +choices = ["Yes", "No"] +answer = 1 +buttonq(choices, answer) +``` + + +If so them be repeating the above until $k=0$, we can see that $I_k = p_k \exp(-x)$ for some polynomial $p_k$ of degree $k$. + + + + + ###### Question @@ -700,8 +763,8 @@ Consider the integral $\int x \cos(x) dx$. Which letter should be tried first? #| hold: true #| echo: false choices = ["L", "I", "A", "T", "E"] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` --- @@ -714,8 +777,8 @@ Consider the integral $\int x^2\log(x) dx$. Which letter should be tried first? #| hold: true #| echo: false choices = ["L", "I", "A", "T", "E"] -answ = 1 -radioq(choices, answ, keep_order=true) +answer = 1 +radioq(choices, answer, keep_order=true) ``` --- @@ -728,8 +791,8 @@ Consider the integral $\int x^2 \sin^{-1}(x) dx$. Which letter should be tried f #| hold: true #| echo: false choices = ["L", "I", "A", "T", "E"] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` --- @@ -742,8 +805,8 @@ Consider the integral $\int e^x \sin(x) dx$. Which letter should be tried first? #| hold: true #| echo: false choices = ["L", "I", "A", "T", "E"] -answ = 4 -radioq(choices, answ, keep_order=true) +answer = 4 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -759,6 +822,51 @@ choices = [ "``x\\cos^{-1}(x)-\\sqrt{1 - x^2}``", "``x^2/2 \\cos^{-1}(x) - x\\sqrt{1-x^2}/4 - \\cos^{-1}(x)/4``", "``-\\sin^{-1}(x)``"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` + + +###### Question + +In [notes by S.G. Johnson](https://math.mit.edu/~stevenj/trap-iap-2011.pdf) we find a simpler means to estimate the maximum error in the (composite) trapezoid rule. + +Each trapezoid estimate uses a line between two points, $x_{i-1}$ and $x_i$ so deviates from a Taylor polynomial how? + +```{julia} +#| echo: false +choices = ["In the constant term", + "In the linear term", + "In the quadratic term"] +answer = 3 +explanation = "The constant and linear terms can match off, leaving a quadratic." +buttonq(choices, answer; explanation) +``` + +The worst case error in a single approximation is then around $\Delta_x^2$ times a constant depending on $f''$. + +The error in the approximation---up to a constant---over the interval is: + +```{julia} +#| echo: false +choices = [L"Basically $\Delta_x$", + L"Basically $\Delta_x^2$", + L"Basically $\Delta_x^2 \cdot \Delta_x$"] +answer = 3 +explanation = "The worst case is the error *times* the length of the base" +buttonq(choices, answer; explanation) +``` + +In terms of $n$, the error in each sub-interval is like $1/n^3$. The total error in the approximation is then + +```{julia} +#| echo: false +choices = [L"Basically $1/n^3$", + L"Basically $1/n^2 = 1/n^3 \cdot n$", + L"Basically $1/n = 1/n^3 \cdot n^2$"] +answer = 2 +explanation = "There are ``n`` terms each with error like ``1/n^3``" +buttonq(choices, answer; explanation) +``` + +This is a worst case estimate, of course, as it doesn't account for individual errors canceling each other off. diff --git a/quarto/integrals/mean_value_theorem.qmd b/quarto/integrals/mean_value_theorem.qmd index 527a05d..c8f1ca3 100644 --- a/quarto/integrals/mean_value_theorem.qmd +++ b/quarto/integrals/mean_value_theorem.qmd @@ -1,5 +1,14 @@ # Mean value theorem for integrals +::: {#fig-ice-cream-shop} +![](./figures/ice-cream.jpg) + +How to compare which container has more ice cream at an ice cream shop? +::: + +At an ice cream shop, a container of vanilla and one of chocolate have been busy, leaving the containers with an uneven top due to areas where scoops have been taken out, and where not. Which container has more left? We could compare readily if we were willing to spread the ice cream around so that the top of the ice cream is a uniform level. Then the one with a height height has more volume. The uniform level is just the mean value theorem for integrals applied to volumes. The volume of ice cream in the container is equivalent to the area of the base times *some* height which reflects the amount of total ice cream. + +---- {{< include ../_common_code.qmd >}} @@ -8,13 +17,11 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia -using Plots -plotly() +using Plots; plotly() using QuadGK ``` ---- ## Average value of a function @@ -30,7 +37,7 @@ $$ \frac{1}{b-a} \int_a^b f(x) dx. $$ -If $f$ is a constant, this is just the constant value, as would be expected. If $f$ is *piecewise* linear, then this is the weighted average of these constants. +If $f$ is a constant, this is just the constant value, as would be expected. If $f$ is *piecewise* constant, then this is the weighted average of these constants. #### Examples @@ -63,20 +70,22 @@ What is the average value of $f(x)=\sin(x)$ over $[0, \pi]$? $$ -\text{average} = \frac{1}{\pi-0} \int_0^\pi \sin(x) dx = \frac{1}{\pi} (-\cos(x)) \big|_0^\pi = \frac{2}{\pi} +\text{average} = \frac{1}{\pi-0} \int_0^\pi \sin(x) dx = \frac{1}{\pi} (-\cos(x)) \Big|_0^\pi = \frac{2}{\pi} $$ -Visually: +@fig-integral-mean-value illustrates. +::: {#fig-integral-mean-value} ```{julia} -#| label: fig-integral-mean-value -#| fig-cap: "Area under sine curve is equal to area of rectangle" +#| echo: false plot(sin, 0, pi, legend=false, fill=(:forestgreen, 0.25, 0)) plot!(x -> 2/pi, fill=(:royalblue, 0.25, 0)) ``` -In @fig-integral-mean-value the area under the sine curve ($2 = (-\cos(\pi)) - (-\cos(0))$) is equal to the area under the average (also $2 = 2/\pi \cdot \pi$). +Area under the sine curve over $[0, \pi]$ is equal to the area of a rectangle or the area under the constant function $f(x) = 2/\pi$ +::: + ##### Example @@ -84,7 +93,7 @@ In @fig-integral-mean-value the area under the sine curve ($2 = (-\cos(\pi)) - ( What is the average value of the function $f$ which is $3$ between $[0,3]$, $2$ between $(3,5]$ and $1$ between $(5,6]$? -Though not continuous, $f(x)$ is integrable as it contains only jumps. The integral from $[0,6]$ can be computed with geometry: $3\cdot 3 + 2 \cdot 2 + 1 \cdot 1 = 14$. The average then is $14/(6-0) = 7/3$. +Though not continuous, $f(x)$ is integrable as it contains only jumps. The integral from $[0,6]$ can be computed with geometry: $3\cdot 3 + 2 \cdot 2 + 1 \cdot 1 = 14$. The average then is $14/(6-0) = 7/3$. This can also be expressed as a weighted sum: $3 \cdot 3/6 + 2 \cdot 2/6 + 1 \cdot 1/6$, the weights reflecting the respective interval lengths. ##### Example @@ -96,21 +105,25 @@ What is the average value of the function $e^{-x}$ between $0$ and $\log(2)$? $$ \begin{align*} \text{average} &= \frac{1}{\log(2) - 0} \int_0^{\log(2)} e^{-x} dx\\ -&= \frac{1}{\log(2)} (-e^{-x}) \big|_0^{\log(2)}\\ +&= \frac{1}{\log(2)} (-e^{-x}) \Big|_0^{\log(2)}\\ &= -\frac{1}{\log(2)} (\frac{1}{2} - 1)\\ &= \frac{1}{2\log(2)}. \end{align*} $$ -Visualizing, we have - +Visualizing, we have @fig-area-exp-x-0-log2-as-rectangle. +::: {#fig-area-exp-x-0-log2-as-rectangle} +#| echo: false ```{julia} plot(x -> exp(-x), 0, log(2), legend=false, fill=(:forestgreen, 0.25, 0)) plot!(x -> 1/(2*log(2)), fill=(:royalblue, 0.25, 0)) ``` +Rectangle with same area as under $f(x) = e^{-x}$ over $[0,\log(2)]$ +::: + ## The mean value theorem for integrals @@ -123,8 +136,7 @@ $$ When we assume that $f(x)$ is continuous, we can describe $K$ as a value in the range of $f$: -::: {.callout-note icon=false} -## The mean value theorem for integrals +::: {.theorem title="The mean value theorem for integrals"} Let $f(x)$ be a continuous function on $[a,b]$ with $a < b$. Then there exists $c$ with $a \leq c \leq b$ with @@ -144,6 +156,7 @@ $$ So in particular $K$ is in $[m, M]$. But $m$ and $M$ correspond to values of $f(x)$, so by the intermediate value theorem, $K=f(c)$ for some $c$ that must lie in between $c_m$ and $c_M$, which means as well that it must be in $[a,b]$. + ##### Proof of the second part of the Fundamental Theorem of Calculus @@ -157,7 +170,7 @@ $$ \frac{\int_a^{x+h} f(u) du - \int_a^x f(u) du}{h} =\frac{\int_x^{x+h} f(u) du}{h} = f(\xi(h)). $$ -The value $\xi(h)$ is just the $c$ corresponding to a given value in $[x, x+h]$ guaranteed by the mean value theorem. We only know that $x \leq \xi(h) \leq x+h$. But this is plenty - it says that $\lim_{h \rightarrow 0+} \xi(h) = x$. Using the fact that $f$ is continuous and the known properties of limits of compositions of functions this gives $\lim_{h \rightarrow 0+} f(\xi(h)) = f(x)$. But this means that the (right) limit of the secant line expression exists and is equal to $f(x)$, which is what we want to prove. Repeating a similar argument when $h < 0$, finishes the proof. +The value $\xi(h)$ is just the $c$ corresponding to a given value in $[x, x+h]$ guaranteed by the mean value theorem. We only know that $x \leq \xi(h) \leq x+h$. But this is plenty---it says that $\lim_{h \rightarrow 0+} \xi(h) = x$. Using the fact that $f$ is continuous and the known properties of limits of compositions of functions this gives $\lim_{h \rightarrow 0+} f(\xi(h)) = f(x)$. But this means that the (right) limit of the secant line expression exists and is equal to $f(x)$, which is what we want to prove. Repeating a similar argument when $h < 0$, finishes the proof. The basic notion used is simply that for small $h$, this expression is well approximated by the left Riemann sum taken over $[x, x+h]$: @@ -210,8 +223,8 @@ What integral will show the intuition of the Merton College scholars that the di #| hold: true #| echo: false choices = [ -"``\\int_0^t (v_0 + au) du = (v_0 t + a\\cdot u^2/2)\\big|_0^t``", -"``\\int_0^t (v(0) + v(u))/2 du = v(0)/2\\cdot t + x(u)/2\\ \\big|_0^t``", +"``\\int_0^t (v_0 + au) du = (v_0 t + a\\cdot u^2/2)\\Big|_0^t``", +"``\\int_0^t (v(0) + v(u))/2 du = v(0)/2\\cdot t + x(u)/2\\ \\Big|_0^t``", "``(v(0) + v(t))/2 \\cdot \\int_0^t du = (v(0) + v(t))/2 \\cdot t``" ] answ = 1 @@ -389,3 +402,30 @@ L"The exponential of the average of $\log(f)$" answ = val1 > val2 ? 1 : 2 radioq(choices, answ) ``` + + +###### Question + +Above we showed the mean value theorem for integrals is used to prove part of the fundamental theorem of calculus. Suppose by some other means you knew that the FTC was correct and $f(x)$ is continuous. Then we have + +$$ +F(b) - F(a) = \int_a^b f(x) dx +$$ + +Now if $f(x)$ is continuous, $F(x)$ is differentiable and satisfies the mean value theorem for derivatives. What does this say: + +```{julia} +#| echo: false +choices = [L"F'(\xi) = f(\xi) = (F(b) - F(a))/(b-a)", + L"f'(\xi) = (F(b) - F(a))/(b-a)", + L"F'(\xi) = (f(b) - f(a))/(b-a)" + ] +answer = 1 +buttonq(choices, answer) +``` + +Given the right answer, we can solve to get that $\xi$ exists in $[a,b]$ with: + +$$ +\int_a^b f(x) dx = f(\xi) \cdot (b-a) +$$ diff --git a/quarto/integrals/numeric_integrals.qmd b/quarto/integrals/numeric_integrals.qmd new file mode 100644 index 0000000..ad39040 --- /dev/null +++ b/quarto/integrals/numeric_integrals.qmd @@ -0,0 +1,861 @@ +# Numeric approximations to definite integrals + + +{{< include ../_common_code.qmd >}} + +This section uses these add-on packages: + + + +```{julia} +using CalculusWithJulia +using Plots; plotly() +using QuadGK +using Roots +``` + +## Numeric integration + +The fundamental theorem of calculus gives an easy to compute answer to the value of a definite integral *when* a computable (elementary) anti-derivative can be found. This is not always the case. See [Liousville's theorem](https://en.wikipedia.org/wiki/Liouville's_theorem_(differential_algebra)) to read more. If there is no computable anti-derivative the definite integral can be *approximated* numerically, as discussed in this section, where we begin with a Riemann sum approach, but end with a much more efficient Gauss-quadrature approach we will utilize in subsequent sections. + + +The Riemann sum approach gives a method to approximate the value of a definite integral. We just compute an approximating sum for a large value of $n$, so large that the limiting value and the approximating sum are close. + + +To see the mechanics, let's again return to Archimedes' problem and *approximate* $\int_0^1 x^2 dx$. + + +Let us fix some values, $a$, $b,$ and $f$ are part of the question, $n$ is related to the approximation. + + +```{julia} +a, b = 0, 1 +f(x) = x^2 + +n = 5 +``` + +Then for a given $n$ we have some steps to do: create the partition, find the $c_i$, multiply the pieces, and add them up. Here is one way to do all this: + + +```{julia} +xs = a:(b-a)/n:b # also range(a, b, length=n) +deltas = diff(xs) # forms x2-x1, x3-x2, ..., xn-xn-1 +cs = xs[1:end-1] # finds left-hand end points. xs[2:end] would be right-hand ones. +``` + +We want to sum the products $f(c_i)\Delta_i$. Here is one way to do so using `zip` to iterate over the paired off values in `cs` and `deltas`. + + +```{julia} +sum(f(ci)*Δi for (ci, Δi) in zip(cs, deltas)) +``` + +Our answer is not so close to the value of $1/3$, but what did we expect---we only used $n=5$ intervals. Trying again with $50,000$ gives us: + + +```{julia} +#| hold: true +n = 50_000 +xs = a:(b-a)/n:b +deltas = diff(xs) +cs = xs[1:end-1] +sum(f(ci)*Δi for (ci, Δi) in zip(cs, deltas)) +``` + +This value is about $10^{-5}$ off from the actual answer of $1/3$. + + +We should expect that larger values of $n$ will produce better approximate values, as long as numeric issues don't get involved. + + +Before continuing, we define a function to compute approximating sums for us with an extra argument to specifying one of four common methods for estimating $\int_{x_{i-1}}^{x_i}f(x)dx$. Leaving explanations for later, @fig-various-integration-methods shows the different approximations. + + +```{julia} +#| eval: false +function riemann(f, xs; method="right") + Ms = (left = (f,a,b) -> f(a), + right = (f,a,b) -> f(b), + trapezoid = (f,a,b) -> (f(a) + f(b))/2, + simpsons = (f,a,b) -> (c = a/2 + b/2; (1/6) * (f(a) + 4*f(c) + f(b))) + ) + M = Ms[Symbol(method)} + xs′ = zip(xs[1:end-1], xs[2:end]) + sum(M(f, a, b) * (b-a) for (a,b) ∈ xs′) +end + +riemann(f, a, b, n; method="right") = + riemann(f, range(a,b,n+1); method) +``` + +(This function is defined in `CalculusWithJulia` and need not be copied over if that package is loaded.) + +::: {#fig-various-integration-methods} +```{julia} +#| echo: false +let + gr() + f(x) = sin(x/3) + sin(x/1.5) + plt = plot(; legend=false, xaxis=([], false), + yaxis=([], false))#, empty_style...) + plot!(plt, [(-0.5, 0), (6.5,0)]; line=(1, :black), arrow=true) + plot!(f, 0, 6) + left = [(1,0), (2,0), (2, f(1)), (1, f(1))] + push!(left, first(left)) + + right = [(2,0), (3,0), (3, f(3)), (2, f(3))] + push!(right, first(right)) + + trapezoid = [(3,0), (4,0), (4, f(4)), (3, f(3))] + push!(trapezoid, first(trapezoid)) + + l1(x) = (x-4.5)*(x-5) / (4 - 4.5) / (4 - 5) + l2(x) = (x-4) * (x-5) / (4.5-4) / (4.5 - 5) + l3(x) = (x-4) * (x-4.5) / (5-4) / (5-4.5) + s(x) = f(4)*l1(x) + f(4.5)*l2(x) + f(5)*l3(x) + ts = range(4,5, 15) + simpsons = [(4,0), tuple.(ts, s.(ts))..., (5, 0)] + push!(simpsons, first(simpsons)) + + plot!(plt, left; line=(1, :blue)) + plot!(plt, right; line=(1, :red)) + plot!(plt, trapezoid, line=(1, :green)) + plot!(plt, simpsons; line=(1, :brown)) + + annotate!(plt, [ + (1.5, .5, text("left riemann", 90.0)), + (2.5, .5, text("right riemann", 90.0)), + (3.5, .5, text("trapezoid", 90.0)), + (4.5, .5, text("Simpson's", 90.0))]) + + plotly() + plt +end +``` + +Plot of a $f(x)$ showing (from left to right) a left Riemann sum approximation, a right Riemann sum approximation, a trapezoid approximation, and Simpson's approximation. The error in the Simpson's approximation (barely discernible) appears to be less than that of the trapezoid approximation which is less than either the left- or right-Riemann sum approximations. +::: + + +With this, we can easily find an approximate answer for a definite integral. We wrote the function to use the familiar template `action(function, arguments...)`, so we pass in a function and arguments to describe the problem (`a`, `b`, and `n` and, optionally, the `method`): + + +```{julia} +f(x) = exp(x) +riemann(f, 0, 5, 10) +``` + +Or with more intervals in the partition + + +```{julia} +riemann(f, 0, 5, 50_000) +``` + +(The answer is $e^5 - e^0 = 147.4131591025766\dots$, which shows that even $50,000$ partitions is not enough to guarantee many digits of accuracy.) + + + +##### Example + + +Numerically estimate the definite integral $\int_0^2 x\log(x) dx$. + + +This particular integrand is continuous on $(0,2]$ *but* we can redefine it to be $0$ at $0$ to make it continuous on $[0,2]$, hence the integral above is well defined. Numerically though, we have to be a bit careful with the Riemann sum, as the left Riemann sum will have an issue at $0=x_0$---`0*log(0)` returns `NaN` which will poison any subsequent arithmetic operations, so the value returned will be `NaN` and not an approximate answer. We could define our function with a check, instead we avoid this value by using the right Riemann sum: + + +```{julia} +h(x) = x * log(x) +riemann(h, 0, 2, 50_000; method="right") +``` + +(The default is `"right"`, so no method specified would also work.) + + +## Error estimate + + +The Riemann sum above is actually extremely inefficient in that it can take a large number of rectangles to produce an "accurate" approximation for a definite integral, even for nice functions. To see how much so, we can derive an estimate for the error in approximating the value using an arithmetic progression as the partition. Let's assume that our function $f(x)$ is increasing, so that the right sum gives an upper estimate and the left sum a lower estimate, so the error in any Riemann sum estimate will be smaller than the distance between these two values: + + +$$ +\begin{align*} +\text{error} +&\leq \text{upper sum} - \text{lower sum}\\ +&= +\left(f(x_1) \cdot (x_{1} - x_0) + f(x_2) \cdot (x_{2} - x_1) + \cdots \right.\\ +&\quad + \left. f(x_{n-1})(x_{n-1} - x_{n-2}) + f(x_n) \cdot (x_n - x_{n-1})\right)\\ +&\quad - +\left(f(x_0) \cdot (x_{1} - x_0) + f(x_1) \cdot (x_{2} - x_1) + \cdots \right.\\ +&\quad + \left. f(x_{n-1})(x_n - x_{n-1}) \right)\\ +&= +\left(f(x_1) \cdot \Delta + f(x_2) \cdot \Delta + \cdots + f(x_{n-1})\Delta + f(x_n) \cdot \Delta y\right)\\ +&\quad - +\left(f(x_0) \cdot \Delta + f(x_1) \cdot \Delta + \cdots + f(x_{n-1})\Delta\right) \\ +&= \left(\left[f(x_1) + f(x_2) + \cdots + f(x_n)\right] - \left[f(x_0) + \cdots + f(x_{n-1})\right]\right) \cdot \Delta \\ +&= \left(f(b) - f(a)\right) \cdot \frac{b-a}{n}. +\end{align*} +$$ + + +We see the error goes to $0$ at a rate of $1/n$ with the constant depending on $b-a$ and the function $f$. In general, a similar bound holds when $f$ is not monotonic. + +### The trapezoid rule + +There are other ways to approximate the integral that use fewer points in the partition. Riemann sums approximate the definite integral over each "piece" of the partition---$\int_{x_{i-1}}^{x_i} f(x) dx$---using a rectangle. Other geometric shapes are possible. + +The *trapezoid* rule uses a trapezoid formed to approximate this area, namely the one formed by $(x_{i-1}, 0)$, $(x_{i-1}, f(x_{i-1}))$, $(x_i, f(x_i))$, and $(x_i, 0)$ with area + +$$ +\frac{1}{2} \left(f(x_{i-1}) + f(x_i) \right) \cdot (x_i - x_{i-1}). +$$ + +If we use an equally spaced partition ($\Delta=(b-a)/n$) and add all the $n$ terms, we get single contributions from the endpoints and double from the others giving + +$$ +A \approx (\frac{f(x_0)}{2} + \frac{f(x_n)}{2})\Delta + \sum_{i=1}^{n-1} f(x_i) \Delta/ +$$ + +In a later section, we will see that the error in using trapezoids to +estimate the area is bounded, for some constant $K$: + + +$$ +\text{error} \leq \frac{K (b-a)^3}{12n^2}. +$$ + +The $n^2$ means *roughly* that the error in the estimate using a Riemann sum with $n$ terms is similar to the error in the estimate using the trapezoid rule with $\sqrt{n}$ terms.^[There are functions where the trapezoid method has much faster convergence, even exponential. (cf. [this article](https://people.maths.ox.ac.uk/trefethen/sirev56-3_385.pdf) and [these notes](https://math.mit.edu/~stevenj/trap-iap-2011.pdf) for some background.)] + + +##### Example + +Consider the integral + +$$ +\int_0^2 x e^{-x} dx = 1 - 3 e^{-2} = 0.59399415\cdots +$$ + +For comparison sake, we define the exact answer as a constant: + +```{julia} +A = 1 - 3 * exp(-2) +``` + +The error of a Riemann sum with $n=10^4$ is then: + +```{julia} +a, b = 0, 2 +f(x) = x * exp(-x) +riemann(f, a, b, 10^4; method="right") - A +``` + +and this is comparable to the error of the trapezoid method with $n=10^2$: + +```{julia} +riemann(f, a, b, 10^2; method="trapezoid") - A +``` + + + +### Simpson's rule + + [Simpson's](http://tinyurl.com/7b9pmu) rule is one, where instead of approximating the area with rectangles that go through some $c_i$ in $[x_{i-1}, x_i]$ instead the function is approximated by the quadratic polynomial going through $x_{i-1}$, $(x_i + x_{i-1})/2$, and $x_i$ and the exact area under that polynomial is used in the approximation. The explicit formula for a single partition is^[The `riemann` function sums this expression, but this approach is inefficient computationally. Alternative formulations would be suggested.] + +$$ +\int_{x_{i-1}}^{x_i} f(x) dx \approx \frac{x_i - x_{i-1}}{6}\left(f(x_{i-1}) + 4f(\frac{x_{i-1} + x_i}{2}) + f(x_i)\right) +$$ + + + +The error in this approximation can be shown to be + + +$$ +\text{error} \leq \frac{(b-a)^5}{180n^4} \text{max}_{\xi \text{ in } [a,b]} \lvert f^{(4)}(\xi) \rvert. +$$ + +That is, the error is like $1/n^4$ with constants depending on the length of the interval, $(b-a)^5$, and the maximum value of the fourth derivative over $[a,b]$. This is significant, the error in $10$ steps of Simpson's rule is on the scale of the error of $10,000$ steps of the Riemann sum for well-behaved functions. + + +:::{.callout-note} +## Note +The Wikipedia article mentions that Kepler used a similar formula $100$ years prior to Simpson, or about $200$ years before Riemann published his work. Again, the value in Riemann's work is not the computation of the answer, but the framework it provides in determining if a function is Riemann integrable or not. + +::: + + +##### Example + +Continuing the previous example, the accuracy of Simpson's rule with $10$ steps is comparable to that of a Riemann sum with $10^4$ steps: + +```{julia} +riemann(f, a, b, 10; method="simpsons") - A +``` + + +## Gauss quadrature + +There are function types where the above approximations are actually exact: + +* Riemann sums are exact for *constant* functions (polynomials with order $0$) +* The trapezoid method is exact for *linear* functions (polynomials with order $1$) +* Simpson's rule is exact for *quadratic* functions (polynomials with degree $2$) + +This pattern could be extended by taking more intermediate points. In fact an entire family of similar approximations using $n$ points can be made exact for any polynomial of degree $n-1$ or lower. +However, by choosing points judiciously---not necessarily evenly spaced out and not necessarily including the end points---$n$ points can be exact for polynomials of degree higher than $n$. (Simpson' rule actually being exact for *cubic* polynomials is something that hints at this.) + +The formulas for an approximation to the integral $\int_{-1}^1 f(x) dx$ discussed so far can be written as: + + +$$ +\begin{align*} +S &= f(x_1) \Delta_1 + f(x_2) \Delta_2 + \cdots + f(x_n) \Delta_n\\ + &= w_1 f(x_1) + w_2 f(x_2) + \cdots + w_n f(x_n)\\ + &= \sum_{i=1}^n w_i f(x_i). +\end{align*} +$$ + + +The $w$s are "weights" and the $x$s are nodes. Restricting to the interval $[-1,1]$ presents no loss in generality. + +A [Gaussian](http://en.wikipedia.org/wiki/Gaussian_quadrature) *quadrature rule* is a set of weights and nodes for $i=1, \dots n$ for which the sum is *exact* for any $f$ which is a polynomial of degree $2n-1$ or less. Such choices then also approximate well the integrals of functions which are not polynomials of degree $2n-1$ or less, provided $f$ can be well approximated by a polynomial over $[-1,1]$. (Which is the case for the "nice" functions we encounter, though not for highly oscillatory functions.) More details are discussed in the section on *orthogonal polynomials* and some examples are given in the questions. + + +### The quadgk function + + +In `Julia` a modification of the Gauss quadrature rule is implemented in the `quadgk` function (from the `QuadGK` package) to give numeric approximations to integrals. The `quadgk` function also has the familiar interface `action(function, arguments...)`. Unlike our `riemann` function, there is no `n` specified, as the number of steps is *adaptively* determined. (There is more partitioning occurring where the function is changing rapidly.) Instead, the algorithm outputs an estimate on the possible error along with the answer. Instead of $n$, some trickier problems require a specification of an error threshold. + + +To use the function to integrate `f` over an interval `[a,b]` we have: + + +```{julia} +#| hold: true +f(x) = x * log(x) +quadgk(f, 0, 2) +``` + +As mentioned, there are two values returned: an approximate answer, and an error estimate. In this example we see that the value of $0.3862943610307017$ is accurate to within $10^{-9}$. (The actual answer is $-1 + 2\cdot \log(2)$ and the error is only $10^{-11}$. The reported error is an estimated upper bound, and may be conservative, as with this problem.) Our previous answer using $50,000$ right-Riemann sums was $0.38632208884775737$ and is only accurate to $10^{-5}$. By contrast, this method uses just $256$ function evaluations in the above problem. + + +The method should be exact for polynomial functions: + + +```{julia} +#| hold: true +f(x) = x^5 - x + 1 +quadgk(f, -2, 2) +``` + +The error term is $0$, the answer is $4$ up to the last unit of precision (1 ulp), so any error is only in floating point approximations. + + +For the numeric approximation of a definite integral, the `quadgk` function should be preferred over the other methods previously discussed. + + +Here are some sample integrals computed with `quadgk`: + +---- + +$$ +\int_0^\pi \sin(x) dx +$$ + +```{julia} +quadgk(sin, 0, pi) +``` + +(Again, the actual answer is off only in the last digit, the error estimate is an upper bound.) + +---- + +$$ +\int_0^5 e^x dx +$$ + +```{julia} +quadgk(exp, 0, 5) +``` + +---- + +$$ +\int_0^2 x^x dx +$$ + +```{julia} +u(x) = x^x +quadgk(u, 0, 2) +``` + +The function $x^x$ is not continuous at $0$, but can be defined to be so. In this case, the numeric definition of `0^0` matches the limit, so no discussion of redefining the function is necessary, as was done earlier with the function $x\cdot \log(x)$. + +In fact, the specified endpoints to `quadgk` are *never* evaluated, so such concerns are not needed. (Which can be exploited when integrals involving functions with vertical asymptotes are discussed.) This is why the first example---which integrated `x*log(x)`---did not return `NaN` but rather an estimate for the integral. + +#### Dropping the error term + +When composing the answer with other functions it may be desirable to drop the error in the answer, we discuss three styles that can be used for this. The first is to just name the two returned values: + + +```{julia} +#| hold: true +A, err = quadgk(cos, 0, pi/4) +A +``` + +The second is to ask for just the first component of the returned value: + + +```{julia} +#| hold: true +A = first(quadgk(tan, 0, pi/4)) +``` + +Finally, direct indexing can be applied, as with + +```{julia} +quadgk(tan, 0, pi/4)[1] +``` + +Though we try to avoid this style in favor of being more explicit when that is convenient. + +##### Example + +In probability theory, a *univariate density* is a function, $f(x)$ such that $f(x) \geq 0$ and $\int_a^b f(x) dx = 1$, where $a$ and $b$ are the range of the distribution. + +The [Von Mises](http://en.wikipedia.org/wiki/Von_Mises_distribution) distribution, takes the form + + +$$ +k(x) = C \cdot \exp(\cos(x)), \quad -\pi \leq x \leq \pi. +$$ + +Compute $C$ (numerically). + + +The fact that $1 = \int_{-\pi}^\pi C \cdot \exp(\cos(x)) dx = C \int_{-\pi}^\pi \exp(\cos(x)) dx$ implies that $C$ is the reciprocal of the definite integral: + + +```{julia} +k(x) = exp(cos(x)) +A, err = quadgk(k, -pi, pi) +``` + +So + + +```{julia} +C = 1/A +k₁(x) = C * exp(cos(x)) +``` + +The *cumulative distribution function* for $k(x)$ is $K(x) = \int_{-\pi}^x k(u) du$, $-\pi \leq x \leq \pi$. We just showed that $K(\pi) = 1$ and it is trivial that $K(-\pi) = 0$. The quantiles of the distribution are the values $q_1$, $q_2$, and $q_3$ for which $K(q_i) = i/4$. Can we find these? + + +First we define a function, that computes $K(x)$. We only need the first of the two answers given by `quadgk`. + + +```{julia} +K(x) = first(quadgk(k₁, -pi, x)) +``` + + +The question asks us to solve $K(x) = 0.25$, $K(x) = 0.5$ and $K(x) = 0.75$. The `Roots` package can be used for such work, in particular `find_zero`. We will use a bracketing method, as clearly $K(x)$ is increasing, as $k(u)$ is positive, so we can just bracket our answer with $-\pi$ and $\pi$. (We solve $K(x) - p = 0$, so $K(\pi) - p > 0$ and $K(-\pi)-p < 0$.). We could do this with a comprehension, but for variety use broadcasting with `solve` below. + + +```{julia} +#| hold: true +Z = ZeroProblem((x,p) -> K(x) - p, (-pi, pi)) +solve.(Z, (1/4, 1/2, 3/4)) +``` + +The middle one is clearly $0$. This distribution is symmetric about $0$, so half the area is to the right of $0$ and half to the left, so clearly when $p=0.5$, $x$ is $0$. The other two show that the area to the left of $-0.809767$ is equal to the area to the right of $0.809767$ and equal to $0.25$. + + +#### Visualizing the nodes chosen by `quadgk` + +To visualize the choice of nodes by the algorithm, In @fig-visualize-node-choice-quadgk-algorithm-sin-x the nodes chosen are shown for $f(x)=\sin(x)$ over $[0,\pi]$. Relatively few nodes used to get a high-precision estimate. + +::: {#fig-visualize-node-choice-quadgk-algorithm-sin-x} +```{julia} +#| echo: false +function FnWrapper(f) + xs=Any[] + ys=Any[] + x -> begin + fx = f(x) + push!(xs, x) + push!(ys, fx) + fx + end +end +nothing +``` + +```{julia} +#| hold: true +#| echo: false +let + a, b= 0, pi + f(x) = sin(x) + F = FnWrapper(f) + ans,err = quadgk(F, a, b) + plot(f, a, b, legend=false, title="Error ≈ $(round(err,sigdigits=2))") + scatter!(F.xs, F.ys) +end +``` +The nodes chosen by `quadgk` for $f(x) = \sin(x)$ over $[0, \pi]$ +::: + + +For a more oscillatory function, more nodes are chosen, as seen in @fig-visualize-quadgk-nodes-more-osciallations. + +::: {#fig-visualize-quadgk-nodes-more-osciallations} +```{julia} +#| hold: true +#| echo: false +let + a, b= 0, pi + f(x) = exp(-x)*sinpi(x) + F = FnWrapper(f) + ans,err = quadgk(F, a, b) + plot(f, a, b, legend=false, title="Error ≈ $(round(err,sigdigits=2))") + scatter!(F.xs, F.ys) +end +``` + +Visualization nodes chosen by `quadgk` for $f(x) = e^{x} \sin(\pi x)$ over $[0, \pi]$. There are more nodes chosen then for the function $\sin(x)$, as seen in @fig-visualize-node-choice-quadgk-algorithm-sin-x +::: + +In both @fig-visualize-node-choice-quadgk-algorithm-sin-x and @fig-visualize-quadgk-nodes-more-osciallations it can be verified that no node is one of the endpoints. + +##### Example: Gauss nodes + +The `QuadGK.gauss(n)` function returns a pair of $n$ quadrature points and weights to integrate a function over the interval $(-1,1)$, with an option to use a different interval $(a,b)$. For a given $n$, these values exactly integrate any polynomial of degree $2n-1$ or less. In this example, these $5$ points produce an answer accurate already to the $5$th decimal point. + + +```{julia} +xs, ws = QuadGK.gauss(5) +``` + +```{julia} +f(x) = exp(cos(x)) +sum(w * f(x) for (x, w) in zip(xs, ws)) +``` + +The pattern to integrate can be expressed in other ways, but using the `zip` function to iterate over the `xs` and `ws` as pairs of values is pretty direct. + +## Questions + + +###### Question + + +For the function $f(x) = \sin(\pi x)$, estimate the integral for $-1$ to $1$ using a left-Riemann sum with the partition $-1 < -1/2 < 0 < 1/2 < 1$. + + +```{julia} +#| hold: true +#| echo: false +f(x) = sin(pi*x) +xs = -1:1/2:1 +deltas = diff(xs) +val = sum(map(f, xs[1:end-1]) .* deltas) +numericq(val) +``` + + +###### Question + + +For the right Riemann sum approximating $\int_0^{10} e^x dx$ with $n=100$ subintervals, what would be a good estimate for the error? + + +```{julia} +#| hold: true +#| echo: false +choices = [ +"``(10 - 0)/100 \\cdot (e^{10} - e^{0})``", +"``10/100``", +"``(10 - 0) \\cdot e^{10} / 100^4``" +] +answ = 1 +radioq(choices, answ) +``` + +###### Question + + +Use `quadgk` to find the following definite integral: + + +$$ +\int_1^4 x^x dx . +$$ + +```{julia} +#| hold: true +#| echo: false +f(x) = x^x +a, b = 1, 4 +val, _ = quadgk(f, a, b) +numericq(val) +``` + +###### Question + + +Use `quadgk` to find the following definite integral: + + +$$ +\int_0^3 e^{-x^2} dx . +$$ + +```{julia} +#| hold: true +#| echo: false +f(x) = exp(-x^2) +a, b = 0, 3 +val, _ = quadgk(f, a, b) +numericq(val) +``` + +###### Question + + +Use `quadgk` to find the following definite integral: + + +$$ +\int_0^{9/10} \tan(u \frac{\pi}{2}) du. +$$ + +```{julia} +#| hold: true +#| echo: false +f(x) = tan(x*pi/2) +a, b = 0, 9/10 +val, _ = quadgk(f, a, b) +numericq(val) +``` + +###### Question + + +Use `quadgk` to find the following definite integral: + + +$$ +\int_{-1/2}^{1/2} \frac{1}{\sqrt{1 - x^2}} dx +$$ + +```{julia} +#| hold: true +#| echo: false +f(x) = 1/sqrt(1 - x^2) +a, b =-1/2, 1/2 +val, _ = quadgk(f, a, b) +numericq(val) +``` + +###### Question + +Let $A=1.98$ and $B=1.135$ and + +$$ +f(x) = \frac{1 - e^{-Ax}}{B\sqrt{\pi}x} e^{-x^2}. +$$ + +Find $\int_0^1 f(x) dx$ + +```{julia} +#| echo: false +let + A,B = 1.98, 1.135 + f(x) = (1 - exp(-A*x))*exp(-x^2)/(B*sqrt(pi)*x) + val,_ = quadgk(f, 0, 1) + numericq(val) +end +``` + +###### Question + +A bound for the complementary error function ( positive function) is + +$$ +\text{erfc}(x) \leq \frac{1}{2}e^{-2x^2} + \frac{1}{2}e^{-x^2} \leq e^{-x^2} +\quad x \geq 0. +$$ + +Let $f(x)$ be the first bound, $g(x)$ the second. +Assuming this is true, confirm numerically using `quadgk` that + +$$ +\int_0^3 f(x) dx \leq \int_0^3 g(x) dx +$$ + + +The value of $\int_0^3 f(x) dx$ is + +```{julia} +#| echo: false +let + f(x) = 1/2 * exp(-2x^2) + 1/2 * exp(-x^2) + val,_ = quadgk(f, 0, 3) + numericq(val) +end +``` + +The value of $\int_0^3 g(x) dx$ is + +```{julia} +#| echo: false +let + g(x) = exp(-x^2) + val,_ = quadgk(g, 0, 3) + numericq(val) +end +``` + + + +###### Question + +::: {#fig-jsxgraph-riemann-sum-illustration} +```{=html} +
+``` + +```{ojs} +//| echo: false +//| output: false +JXG = require("jsxgraph"); + +b = JXG.JSXGraph.initBoard('jsxgraph', { + boundingbox: [-0.5,0.3,1.5,-1/4], axis:true +}); + +g = function(x) { return x*x*x*x + 10*x*x - 60* x + 100} +f = function(x) {return 1/Math.sqrt(g(x))}; + +type = "right"; +l = 0; +r = 1; +rsum = function() { + return JXG.Math.Numerics.riemannsum(f,n.Value(), type, l, r); +}; +n = b.create('slider', [[0.1, -0.05],[0.75,-0.05], [2,1,50]],{name:'n',snapWidth:1}); + +graph = b.create('functiongraph', [f, l, r]); +os = b.create('riemannsum', + [f, + function(){ return n.Value();}, + type, l, r + ], + {fillColor:'#ffff00', fillOpacity:0.3}); + +b.create('text', [0.1,0.25, function(){ + return 'Riemann sum='+(rsum().toFixed(4)); +}]); +``` + +Interactive graphic showing the area of a right-Riemann sum for different partitions. +::: + + +The function in the interactive graph of @fig-jsxgraph-riemann-sum-illustration is + +$$ +f(x) = \frac{1}{\sqrt{ x^4 + 10x^2 - 60x + 100}}. +$$ + +When $n=5$ what is the area of the Riemann sum? + + +```{julia} +#| hold: true +#| echo: false +numericq(0.1224) +``` + +When $n=50$ what is the area of the Riemann sum? + + +```{julia} +#| hold: true +#| echo: false +numericq(0.1187) +``` + +Using `quadgk` what is the area under the curve? + + +```{julia} +#| hold: true +#| echo: false +g(x) = 1/sqrt(x^4 + 10x^2 - 60x + 100) +val, tmp = quadgk(g, 0, 1) +numericq(val) +``` + +###### Question + + +Gauss nodes for approximating the integral $\int_{-1}^1 f(x) dx$ for $n=4$ are: + + +```{julia} +ns = [-0.861136, -0.339981, 0.339981, 0.861136] +``` + +The corresponding weights are + + +```{julia} +wts = [0.347855, 0.652145, 0.652145, 0.347855] +``` + +Use these to estimate the integral $\int_{-1}^1 \cos(\pi/2 \cdot x)dx$ with $w_1f(x_1) + w_2 f(x_2) + w_3 f(x_3) + w_4 f(x_4)$. + + +```{julia} +#| hold: true +#| echo: false +f(x) = cos(pi/2*x) +val = sum([f(ni)*wi for (wi, ni) in zip(wts, ns)]) +numericq(val) +``` + +The actual answer is $4/\pi$. How far off is the approximation based on 4 points? + + +```{julia} +#| hold: true +#| echo: false +choices = [ +L"around $10^{-1}$", +L"around $10^{-2}$", +L"around $10^{-4}$", +L"around $10^{-6}$", +L"around $10^{-8}$"] +answ = 4 +radioq(choices, answ, keep_order=true) +``` + +###### Question + + +Using the Gauss nodes and weights from the previous question, estimate the integral of $f(x) = e^x$ over $[-1, 1]$. The value is: + + +```{julia} +#| hold: true +#| echo: false +f(x) = exp(x) +val = sum([f(ni)*wi for (wi, ni) in zip(wts, ns)]) +numericq(val) +``` diff --git a/quarto/integrals/orthogonal_polynomials.qmd b/quarto/integrals/orthogonal_polynomials.qmd index 4bdd49d..bdf3907 100644 --- a/quarto/integrals/orthogonal_polynomials.qmd +++ b/quarto/integrals/orthogonal_polynomials.qmd @@ -12,7 +12,7 @@ using Roots using ForwardDiff: derivative ``` -This section takes a detour to give some background on why the underlying method of `quadgk` is more efficient than those of Riemann sums. Orthogonal polynomials play a key role. There are many families of such polynomials. We highlight two. +This section takes an unnecessary detour to give some background on why the underlying method of `quadgk` is more efficient than those of Riemann sums. Orthogonal polynomials play a key role. There are many families of such polynomials. We highlight two. ## Inner product @@ -50,17 +50,15 @@ $$ \angle(f,g) = \cos^{-1}\left(\frac{\langle f, g\rangle}{\lVert f \rVert \lVert g \rVert}\right). $$ -This says, the angle between two orthogonal elements is $90$ degrees (in some orientation) - The Cauchy-Schwarz inequality, $|\langle f, g \rangle| \leq \lVert f \rVert \lVert g\rVert$, for an inner product space, ensures the argument to $\cos^{-1}$ is between $-1$ and $1$. -These properties generalize two-dimensional vectors, with components $\langle x, y\rangle$. Recall, these can be visualized by placing a tail at the origin and a tip at the point $(x,y)$. Such vectors can be added by placing the tail of one at the tip of the other and using the vector from the other tail to the other tip. +These properties generalize two-dimensional vectors, with components $\langle x, y\rangle$. Recall, vectors can be visualized by placing a tail at the origin and a tip at the point $(x,y)$. Such vectors can be added by placing the tail of one at the tip of the other and using the vector from the other tail to the other tip. With this, we have a vector anchored at the origin can be viewed as a line segment with slope $y/x$ (rise over run). A perpendicular line segment would have slope $-x/y$ (the negative reciprocal) which would be associated with the vector $\langle y, -x \rangle$. The dot product is just the sum of the multiplied components, or for these two vectors $x\cdot y + y\cdot (-x)$, which is $0$, as the line segments are perpendicular (orthogonal). Consider now two vectors, say $f$, $g$. We can make a new vector that is orthogonal to $f$ by combining $g$ with a piece of $f$. But what piece? -Consider this +Consider this calculation (using the known answer): $$ \begin{align*} @@ -75,7 +73,7 @@ Define $$ proj_f(g) = \frac{\langle f,g\rangle }{\langle f, f\rangle} f, $$ -then we have $u_1 = f$ and $u_2 = g-proj_f(g)$, $u_1$ and $u_2$ are orthogonal. +then we have with $u_1 = f$ and $u_2 = g-proj_f(g)$ that $u_1$ and $u_2$ are orthogonal. A similar calculation shows if $h$ is added to the set of elements, then $u_3 = h - proj_{u_1}(h) - proj_{u_2}(h)$ will be orthogonal to $u_1$ and $u_2$. etc. @@ -94,7 +92,7 @@ $$ x^2 \mid_{-1}^1 = 1^2 - (-1)^2 = 0. $$ -Now consider a quadratic polynomial, $u_2(x) = ax^2 + bx + c$, we want a polynomial which is orthogonal to $u_0$ and $u_1$ with the extra condition that $u_2(1) = c =1$ (or $c=1$.). We can do this using Gram-Schmidt as above, or as here through a system of two equations: +Now consider a quadratic polynomial, $u_2(x) = ax^2 + bx + c$, we want a polynomial which is orthogonal to $u_0$ and $u_1$ with the extra condition that $u_2(1) = c =1$ (or $c=1$.). We can do this using Gram-Schmidt as above, or, as here, through a system of two equations: ```{julia} @syms a b c d x @@ -117,7 +115,7 @@ u3 = a*x^3 + b*x^2 + c*x + d eqs = (integrate(u0 * u3, (x, -1, 1)) ~ 0, integrate(u1 * u3, (x, -1, 1)) ~ 0, integrate(u2 * u3, (x, -1, 1)) ~ 0) -sols = solve(eqs, (a, b, c, d)) # a => -5c/3, b=>0, d=>0 +sols = solve(eqs, (a, b, c, d)) # a => -5c/3, b => 0, d => 0 u3 = u3(sols...) u3 = simplify(u3/u3(x=>1)) # make u3(1) = 1 ``` @@ -152,8 +150,6 @@ $$ Unique elements can be defined by specifying some additional property. For Legendre, it was $p_n(1)=1$, for other orthogonal families this may be specified by having leading coefficient of $1$ (monic), or a norm of $1$ (orthonormal), etc. -The above is the *absolutely continuous* case, generalizations of the integral allow this to be more general. - Orthogonality can be extended: If $q(x)$ is any polynomial of degree $m < n$, then $\langle q, p_n \rangle = \int_I q(x) p_n(x) w(x) dx = 0$. (See the questions for more detail.) @@ -186,9 +182,9 @@ As this has degree $n$ or less, it can be expressed in terms of $p_0, p_1, \dots $$ \begin{align*} \int_I p_m(x) u(x) w(x) dx &= -\int_I p_m(x) \sum_{j=0}^n p_j(x) w(x) dx \\ -&= \int_I p_m(x) \left(p_m(x) + \textcolor{red}{\sum_{j=0, j\neq m}^{n} p_j(x)}\right) w(x) dx \\ -&= \int_I p_m(x) p_m(x) w(x) dx = h_m +\int_I p_m(x) \left(\sum_{j=0}^n d_j p_j(x)\right) w(x) dx \\ +&= \int_I p_m(x) \left(d_m p_m(x) + \textcolor{red}{\sum_{j=0, j\neq m}^{n} d_j p_j(x)}\right) w(x) dx \\ +&= d_m \int_I p_m(x) p_m(x) w(x) dx = d_m \cdot h_m \end{align*} $$ @@ -205,8 +201,9 @@ $$ $$ The last integral being $0$ as $xp_m(x)$ has degree $n-1$ or less and hence is orthogonal to $p_n$. +If $m < n-1$, then $0 = d_m h_m$, with $h_m > 0$, so $d_m=0$ -That is $p_{n+1} - A_n x p_n(x) = d_n p_n(x) + d_{n-1} p_{n-1}(x)$. Setting $B_n=d_n$ and $C_{n-1} = -d_{n-1}$ shows the three-term recurrence applies. +That is $p_{n+1} - A_n x p_n(x) = u(x) = d_n p_n(x) + d_{n-1} p_{n-1}(x)$. Setting $B_n=d_n$ and $C_{n-1} = -d_{n-1}$ shows the three-term recurrence applies. #### Example: Legendre polynomials @@ -221,7 +218,7 @@ I &= [-1,1]\\ A_n &= \frac{2n+1}{n+1}\\ B_n &= 0\\ C_n & = \frac{n}{n+1}\\ -k_{n+1} &= \frac{2n+1}{n+1}k_n - \frac{n}{n-1}k_{n-1}, k_1=k_0=1\\ +k_{n+1} &= \frac{2n+1}{n+1}k_n - \frac{n}{n-1}k_{n-1}, \quad k_1=k_0=1\\ h_n &= \frac{2}{2n+1} \end{align*} $$ @@ -385,7 +382,7 @@ As $u$ is continuous, this means there are at least $n$ sign changes, hence $n$ -### Integration +## Integration Recall, a Riemann sum can be thought of in terms of weights, $w_i$ and nodes $x_i$ for which $\int_I f(x) dx \approx \sum_{i=0}^{n-1} w_i f(x_i)$. For a right-Riemann sum with partition given by $a_0 < a_1 < \cdots < a_n$ the nodes are $x_i = a_i$ and the weights are $w_i = (a_i - a_{i-1})$ (or in the evenly spaced case, $w_i = (a_n - a_0)/n$. @@ -428,7 +425,7 @@ $$ h(x) = q(x) p_n(x) + r(x) $$ -and the degree of $r(x)$ is less than $n-1$, the degree of $p_n(x)$. Further, the degree of $q(x)$ is also less than $n-1$, as were it more, then the degree of $q(x)p_n(x)$ would be more than $n-1+n$ or $2n-1$. Let's note that if $x_i$ is a zero of $p_n(x)$ that $h(x_i)= r(x_i)$. +and the degree of $r(x)$ is no more than $n-1$, the degree of $p_n(x)$ being $n$. Further, the degree of $q(x)$ is also no more than $n-1$, as were it more, then the degree of $q(x)p_n(x)$ would be more than $n-1+n = 2n-1$. Let's note that if $x_i$ is a zero of $p_n(x)$ that $h(x_i)= r(x_i)$. So @@ -557,18 +554,22 @@ $$ The `QuadGK` package uses a modification to Gauss quadrature to estimate numeric integrals. Let's see how. Behind the scenes, `quadgk` calls `kronrod` to compute nodes and weights. -We have from earlier that +We have from earlier these Legendre polynomials: ```{julia} u₃(x) = x*(5x^2 - 3)/2 u₄(x) = 35x^4 / 8 - 15x^2 / 4 + 3/8 ``` +The zeros can readily be found numerically: + + ```{julia} xs = find_zeros(u₄, -1, 1) ``` -From this we can compute the weights from the derived general formula: + +From this we can compute the weights from the derived general formula when $n=4$: ```{julia} k₃, k₄ = 5/2, 35/8 @@ -578,10 +579,10 @@ ws = [k₄/k₃ * 1/(derivative(u₄,xᵢ) * u₃(xᵢ)) * I for xᵢ ∈ xs] (xs, ws) ``` -We compare now to the values returned by `kronrod` in `QuadGK` +We compare now to the values returned by `kronrod` in `QuadGK` to see they agree after alignment: ```{julia} -kxs, kwts, wts = kronrod(4, -1, 1) +kxs, kwts, wts = QuadGK.kronrod(4, -1, 1) [ws wts xs kxs[2:2:end]] ``` @@ -601,6 +602,54 @@ QL, esterror = quadgk(u, -1, 1) The first two are expected to not be as accurate, as they utilize a fixed number of nodes. +---- + +The computation in `QuadGK` compares two values to estimate the error and to dexide if a subdivision is needed. The "common case" is based on Gauss nodes for degree $7$ and related nodes for *Kronrod*. The values below are copied from the source code, which caches them for re-use. + +```{julia} +# precomputed n=7 rule in double precision (computed in 100-bit arithmetic), +# since this is the common case. +const xd7 = [-9.9145537112081263920685469752598e-01, + -9.4910791234275852452618968404809e-01, + -8.6486442335976907278971278864098e-01, + -7.415311855993944398638647732811e-01, + -5.8608723546769113029414483825842e-01, + -4.0584515137739716690660641207707e-01, + -2.0778495500789846760068940377309e-01, + 0.0] +const wd7 = [2.2935322010529224963732008059913e-02, + 6.3092092629978553290700663189093e-02, + 1.0479001032225018383987632254189e-01, + 1.4065325971552591874518959051021e-01, + 1.6900472663926790282658342659795e-01, + 1.9035057806478540991325640242055e-01, + 2.0443294007529889241416199923466e-01, + 2.0948214108472782801299917489173e-01] +const wgd7 = [1.2948496616886969327061143267787e-01, + 2.797053914892766679014677714229e-01, + 3.8183005050511894495036977548818e-01, + 4.1795918367346938775510204081658e-01] +``` + +These weights can be used directly to integrate a function over $[-1,1]$ in the following manner. We know $\int_{-1}^1 \cos(\pi \cdot x/2) dx$ can be integrated and its value if $4/\pi$ or `1.2732395447351628...` With the weights above, we have this value is approximated by $\sum w_i (f(x_i) + f(-x_i))$. + +The Kronrod weights are `wd7` and they yield:^[The awkward `!iszero` avoids double counting the node at `0.0`.] + +```{julia} +f(x) = cos(pi/2 * x) +kr = sum(w * (f(x) + !iszero(x)*f(-x)) for (w,x) ∈ zip(wd7, xd7)) +kr, kr - 4/pi +``` + +The Gauss weights are a bit different, as only 7 are used: + +```{julia} +g = sum(w * (f(x) + !iszero(x)*f(-x)) for (w,x) ∈ zip(wgd7, xd7[2:2:8])) +g, g - 4/pi +``` + +The algorithm compares the value of `kr` to that of `g` and if it is "large" the interval is split making the algorithm adaptive. (Some detail is available at this [Wikipedia](https://en.wikipedia.org/wiki/Gauss%E2%80%93Kronrod_quadrature_formula) page.) + ## Questions ###### Question diff --git a/quarto/integrals/partial_fractions.qmd b/quarto/integrals/partial_fractions.qmd index 5a871c5..0f9b02c 100644 --- a/quarto/integrals/partial_fractions.qmd +++ b/quarto/integrals/partial_fractions.qmd @@ -11,12 +11,13 @@ using SymPy --- + Integration is facilitated when an antiderivative for $f$ can be found, as then definite integrals can be evaluated through the fundamental theorem of calculus. However, despite differentiation being an algorithmic procedure, integration is not. There are "tricks" to try, such as substitution and integration by parts. These work in some cases---but not all! -However, there are classes of functions for which algorithms exist. For example, the `SymPy` `integrate` function mostly implements an algorithm that decides if an elementary function has an antiderivative. The [elementary](http://en.wikipedia.org/wiki/Elementary_function) functions include exponentials, their inverses (logarithms), trigonometric functions, their inverses, and powers, including $n$th roots. Not every elementary function will have an antiderivative comprised of (finite) combinations of elementary functions. The typical example is $e^{x^2}$, which has no simple antiderivative, despite its ubiquitousness. +Yet, there are classes of functions for which algorithms exist. For example, the `SymPy` `integrate` function mostly implements an algorithm that decides if an elementary function has an antiderivative. The [elementary](http://en.wikipedia.org/wiki/Elementary_function) functions include exponentials, their inverses (logarithms), trigonometric functions, their inverses, and powers, including $n$th roots. Not every elementary function will have an antiderivative comprised of (finite) combinations of elementary functions. The typical example is $e^{x^2}$, which has no simple antiderivative, despite its ubiquitousness. There are classes of functions where an (elementary) antiderivative can always be found. Polynomials provide a case. More surprisingly, so do their ratios, *rational functions*. @@ -30,8 +31,7 @@ Let $f(x) = p(x)/q(x)$, where $p$ and $q$ are polynomial functions with real co The function $q(x)$ will factor over the real numbers. The fundamental theorem of algebra can be applied to say that $q(x)=q_1(x)^{n_1} \cdots q_k(x)^{n_k}$ where $q_i(x)$ is a linear or quadratic polynomial and $n_k$ a positive integer. -::: {.callout-note icon=false} -## Partial Fraction Decomposition +::: {.definition title="Partial Fraction Decomposition"} There are unique polynomials $a_{ij}$ with degree $a_{ij} <$ degree $q_i$ such that @@ -60,7 +60,7 @@ The value of this decomposition is that the terms $a_{ij}(x)/q_i(x)^j$ each have :::{.callout-note} ## Note -Many calculus texts will give some examples for finding a partial fraction decomposition. We push that work off to `SymPy`, as for all but the easiest cases - a few are in the problems - it can be a bit tedious. +Many calculus texts will give some examples for finding a partial fraction decomposition. We push that work off to `SymPy`, as for all but the easiest cases---a few are in the problems---it can be a bit tedious. ::: @@ -211,14 +211,15 @@ integrate(B/((a*x)^2 - 1)^4, x) --- -In [Bronstein](http://www-sop.inria.fr/cafe/Manuel.Bronstein/publications/issac98.pdf) this characterization can be found - "This method, which dates back to Newton, Leibniz and Bernoulli, should not be used in practice, yet it remains the method found in most calculus texts and is often taught. Its major drawback is the factorization of the denominator of the integrand over the real or complex numbers." We can also find the following formulas which formalize the above exploratory calculations ($j>1$ and $b^2 - 4c < 0$ below): +In [Bronstein](http://www-sop.inria.fr/cafe/Manuel.Bronstein/publications/issac98.pdf) this characterization can be found---"This method, which dates back to Newton, Leibniz and Bernoulli, should not be used in practice, yet it remains the method found in most calculus texts and is often taught. Its major drawback is the factorization of the denominator of the integrand over the real or complex numbers." We can also find the following formulas which formalize the above exploratory calculations ($j>1$ and $b^2 - 4c < 0$ below): $$ \begin{align*} \int \frac{A}{(x-a)^j} &= \frac{A}{1-j}\frac{1}{(x-a)^{j-1}}\\ \int \frac{A}{x-a} &= A\log(x-a)\\ -\int \frac{Bx+C}{x^2 + bx + c} &= \frac{B}{2} \log(x^2 + bx + c) + \frac{2C-bB}{\sqrt{4c-b^2}}\cdot \arctan\left(\frac{2x+b}{\sqrt{4c-b^2}}\right)\\ +\int \frac{Bx+C}{x^2 + bx + c} &= \frac{B}{2} \log(x^2 + bx + c)\\ +&\quad + \frac{2C-bB}{\sqrt{4c-b^2}}\cdot \arctan\left(\frac{2x+b}{\sqrt{4c-b^2}}\right)\\ \int \frac{Bx+C}{(x^2 + bx + c)^j} &= \frac{B' x + C'}{(x^2 + bx + c)^{j-1}} + \int \frac{C''}{(x^2 + bx + c)^{j-1}} \end{align*} $$ @@ -231,10 +232,13 @@ That is integrating $f(x)/g(x)$, a rational function, will yield an output that $$ -\int f(x)/g(x) = P(x) + \frac{C(x)}{D{x}} + \sum v_i \log(V_i(x)) + \sum w_j \arctan(W_j(x)) +\begin{align*} +\int f(x)/g(x) &= P(x) + \frac{C(x)}{D{x}} + \sum v_i \log(V_i(x))\\ +&\quad + \sum w_j \arctan(W_j(x)). +\end{align*} $$ -(Bronstein also sketches the modern method which is to use a Hermite reduction to express $\int (f/g) dx = p/q + \int (g/h) dx$, where $h$ is square free (the "`j`" are all $1$). The latter can be written over the complex numbers as logarithmic terms of the form $\log(x-a)$, the "`a`s"found following a method due to Trager and Lazard, and Rioboo, which is mentioned in the SymPy documentation as the method used.) +(Bronstein also sketched the modern method which is to use a Hermite reduction to express $\int (f/g) dx = p/q + \int (g/h) dx$, where $h$ is square free (the "`j`" are all $1$). The latter can be written over the complex numbers as logarithmic terms of the form $\log(x-a)$, the "`a`s"found following a method due to Trager and Lazard, and Rioboo, which is mentioned in the SymPy documentation as the method used.) #### Examples @@ -447,8 +451,8 @@ choices = [ L"The value $c$ is a removable singularity, so the integral will be identical.", L"The resulting function has an identical domain and is equivalent for all $x$." ] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` If $m = n$, then why can we cancel out the $(x-c)^n$ and not have a concern? @@ -462,8 +466,8 @@ choices = [ L"The value $c$ is a removable singularity, so the integral will be identical.", L"The resulting function has an identical domain and is equivalent for all $x$." ] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` If $m < n$, then why can we cancel out the $(x-c)^m$ and not have a concern? @@ -477,8 +481,8 @@ choices = [ L"The value $c$ is a removable singularity, so the integral will be identical.", L"The resulting function has an identical domain and is equivalent for all $x$." ] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` ###### Question diff --git a/quarto/integrals/substitution.qmd b/quarto/integrals/substitution.qmd index 04f967e..7e97f85 100644 --- a/quarto/integrals/substitution.qmd +++ b/quarto/integrals/substitution.qmd @@ -3,35 +3,37 @@ {{< include ../_common_code.qmd >}} +We discuss the integration techniques of $u$-substitution and trigonometric substitution. + This section uses these add-on packages: ```{julia} using CalculusWithJulia -using Plots -plotly() +using Plots; plotly() using SymPy ``` ---- +## Integration by *u*-substitution + The technique of $u$-[substitution](https://en.wikipedia.org/wiki/Integration_by_substitution) is derived from reversing the chain rule: $[f(g(x))]' = f'(g(x)) g'(x)$. +::: {.definition title="Subsitution"} -Suppose that $g$ is continuous and $u(x)$ is differentiable with $u'(x)$ being Riemann integrable. Then both these integrals are defined: +Suppose that $g$ is continuous and $u(x)$ is differentiable with $u'(x)$ being Riemann integrable. Then both these integrals are defined and are equal: $$ -\int_a^b g(u(t)) \cdot u'(t) dt, \quad \text{and}\quad \int_{u(a)}^{u(b)} g(x) dx. +\int_a^b g(u(t)) \cdot u'(t) dt = \quad \int_{u(a)}^{u(b)} g(x) dx $$ - -We wish to show they are equal. +::: -Let $G$ be an antiderivative of $g$, which exists as $g$ is assumed to be continuous. (By the Fundamental Theorem part I.) Consider the composition $G \circ u$. The chain rule gives: +Let $G$ be an antiderivative of $g$, which exists and is differentiable as $g$ is assumed to be continuous. (By the Fundamental Theorem part I.) Consider the composition $G \circ u$. The chain rule gives: $$ @@ -50,28 +52,15 @@ $$ \end{align*} $$ - -That is, this substitution formula applies: - - -> $\int_a^b g(u(x)) u'(x) dx = \int_{u(a)}^{u(b)} g(x) dx.$ - - - -Further, for indefinite integrals, - - -> $\int f(g(x)) g'(x) dx = \int f(u) du.$ - - +--- We have seen a special case of substitution where $u(x) = x-c$ in the formula $\int_{a-c}^{b-c} g(x) dx= \int_a^b g(x-c)dx$. -The main use of this is to take complicated things inside of the function $g$ out of the function (the $u(x)$) by renaming them, then accounting for the change of name. +The main use of substitution is to take complicated things inside of the function $g$ out of the function (the $u(x)$) by renaming them, then accounting for the change of name. -Some examples are in order. +An example is in order. Consider: @@ -85,15 +74,13 @@ Clearly the $\sin(x)$ inside the exponential is an issue. If we let $u(x) = \sin $$ -\int_0^{\pi/2} u\prime(x) e^{u(x)} dx = -\int_{u(0)}^{u(\pi/2)} e^x dx = e^x \big|_{\sin(0)}^{\sin(\pi/2)} = e^1 - e^0. +\int_0^{\pi/2} u'(x) e^{u(x)} dx = +\int_{u(0)}^{u(\pi/2)} e^x dx = e^x \Big|_{\sin(0)}^{\sin(\pi/2)} = e^1 - e^0. $$ This all worked, as the problem was such that it was more or less obvious what to choose for $u$ and $G$. -### Integration by substitution - The process of identifying the result of the chain rule in the function to integrate is not automatic, but rather a bit of an art. The basic step is to try some values and hope one works. Typically, this is taught by "substituting" in some value for part of the expression (basically the $u(x)$) and seeing what happens. @@ -115,8 +102,13 @@ Again, we see that the $x^2$ inside the exponential is a complication. Letting $ $$ -\int_0^2 4x e^{x^2} dx = 2\int_0^2 e^{x^2} \cdot 2x dx = 2\int_{u(0)}^{u(2)} e^u du = 2 \int_0^4 e^u du = -2 e^u\big|_{u=0}^4 = 2(e^4 - 1). +\begin{align*} +\int_0^2 4x e^{x^2} dx = 2\int_0^2 e^{x^2} \cdot 2x dx\\ +& = 2\int_{u(0)}^{u(2)} e^u du\\ +&= 2 \int_0^4 e^u du \\ +&= 2 e^u\Big|_{u=0}^4 \\ +&= 2(e^4 - 1). +\end{align*} $$ --- @@ -126,8 +118,11 @@ Consider now $\int_0^1 2x^2 \sqrt{1 + x^3} dx$. Here we see that the $1 + x^3$ m $$ -\int_0^1 2x^2 \sqrt{1 + x^3} dx = \int_{u(0)}^{u(1)} 2 \sqrt{u} (1/3) du = 2/3 \cdot \frac{u^{3/2}}{3/2} \big|_1^2 = -\frac{4}{9} \cdot(2^{3/2} - 1). +\begin{align*} +\int_0^1 2x^2 \sqrt{1 + x^3} dx +&= \int_{u(0)}^{u(1)} 2 \sqrt{u} (1/3) du = 2/3 \cdot \frac{u^{3/2}}{3/2} \Big|_1^2\\ +&= \frac{4}{9} \cdot(2^{3/2} - 1). +\end{align*} $$ --- @@ -137,26 +132,29 @@ Consider $\int_0^{\pi} \cos(x)^3 \sin(x) dx$. The $\cos(x)$ function inside the $$ -\int_0^{\pi} \cos(x)^3 \sin(x) dx = \int_{u(0)}^{u(\pi)} -u^3 du= -\frac{u^4}{4}\big|_1^{-1} = 0. +\int_0^{\pi} \cos(x)^3 \sin(x) dx = \int_{u(0)}^{u(\pi)} -u^3 du= -\frac{u^4}{4}\Big|_1^{-1} = 0. $$ -Changing limits leaves the two antiderivative values of endpoints the same, which means the total area after substitution is $0$. A graph of this function shows that about $\pi/2$ the function has odd-like symmetry, so the answer of $0$ is supported by the plot: - +Changing limits leaves the two antiderivative values of endpoints the same, which means the total area after substitution is $0$. A graph of this function in @fig-plot-cos-cubed-times-sin-over-0-pi shows that about $\pi/2$ the function has odd-like symmetry, so the answer of $0$ is supported by the plot. +::: {#fig-plot-cos-cubed-times-sin-over-0-pi} ```{julia} -#| hold: true +#| echo: false f(x) = cos(x)^3 * sin(x) -plot(f, 0, 1pi) +plot(f, 0, pi; legend=false) ``` +Plot of $f(x) = \cos(x)^3 \cdot \sin(x)$ over $[0, \pi]$ shows odd symmetry about $x=\pi/2$ +::: + --- -Consider $\int_1^e \log(x)/x dx$. There isn't really an "inside" function here, but instead just a tricky $\log(x)$. If we let $u=\log(x)$, what happens? We get $du = 1/x \cdot dx$, which we see present in the original. So with this, we have: +Consider $\int_1^e (\log(x)/x) dx$. There isn't really an "inside" function here, but instead just a tricky $\log(x)$. If we let $u=\log(x)$, what happens? We get $du = 1/x \cdot dx$, which we see present in the original. So with this, we have: $$ -\int_1^e \frac{\log(x)}{x} dx = \int_{u(1)}^{u(e)} u du = \frac{u^2}{2}\big|_0^1 = \frac{1}{2}. +\int_1^e \frac{\log(x)}{x} dx = \int_{u(1)}^{u(e)} u du = \frac{u^2}{2}\Big|_0^1 = \frac{1}{2}. $$ ##### Example: Transformations @@ -165,7 +163,7 @@ $$ We say that the area intrinsically discussed in the definite integral $A=\int_a^b f(x-c) dx$ is unaffected by shifts, in that $A = \int_{a-c}^{b-c} f(x) dx$. What about more general transformations? For example: if $g(x) = (1/h) \cdot f((x-c)/h)$ for values $c$ and $h$ what is the integral over $a$ to $b$ in terms of the function $f(x)$? -If $A = \int_a^b (1/h) \cdot f((x-c)/h) dx$ then we let $u = (x-c)/h$. With this, $du = 1/h \cdot dx$. This allows a straight substitution: +If $A = \int_a^b (1/h) \cdot f((x-c)/h) dx$ then we let $u = (x-c)/h$. With this, $du = 1/h \cdot dx$ allowing a straight substitution: $$ @@ -176,7 +174,7 @@ So the answer is: the area under the transformed function over $a$ to $b$ is the For example, consider the "hat" function $f(x) = 1 - \lvert x \rvert$ -when $-1 \leq x \leq 1$ and $0$ otherwise. The area under $f$ is just $1$ - the graph forms a triangle with base of length $2$ and height $1$. If we take any values of $c$ and $h$, what do we find for the area under the curve of the transformed function? +when $-1 \leq x \leq 1$ and $0$ otherwise. The area under $f$ is just $1$---the graph forms a triangle with base of length $2$ and height $1$. If we take any values of $c$ and $h$, what do we find for the area under the curve of the transformed function? Let $u(x) = (x-c)/h$ and $g(x) = (1/h) \cdot f(u(x))$. Then, as $du = 1/h dx$ @@ -225,7 +223,7 @@ $$ Gives the *total distance* traveled. -To illustrate with a simple example, if a car drives East for one hour at 60 miles per hour, then heads back West for an hour at 60 miles per hour, the car's position after one hour is $x(2) = x(0)$, with a change in position $x(2) - x(0) = 0$. Whereas, the total distance traveled is $120$ miles. (Gas is paid on total distance, not change in position!). What are the formulas for speed and velocity? Clearly $s(t) = 60$, a constant, whereas here $v(t) = 60$ for $0 \leq t \leq 1$ and $-60$ for $1 < t \leq 2$. +To illustrate with a simple example, if a car drives East for one hour at 60 miles per hour, then heads back West for an hour at 60 miles per hour, the car's position after one hour is $x(2) = x(0)$, with a change in position $x(2) - x(0) = 0$. Whereas, the total distance traveled is $120$ miles. (Gas is paid on total distance, not change in position!) What are the formulas for speed and velocity? Clearly $s(t) = 60$, a constant, whereas here $v(t) = 60$ for $0 \leq t \leq 1$ and $-60$ for $1 < t \leq 2$. Suppose $v(t)$ is given by $v(t) = (t-2)^3/3 - 4(t-2)/3$. If $x(0)=0$ Find the position after 3 time units and the total distance traveled. @@ -235,16 +233,22 @@ We let $u(t) = t - 2$ so $du=dt$. The position is given by $$ -\int_0^3 ((t-2)^3/3 - 4(t-2)/3) dt = \int_{u(0)}^{u(3)} (u^3/3 - 4/3 u) du = -(\frac{u^4}{12} - \frac{4}{3}\frac{u^2}{2}) \big|_{-2}^1 = \frac{3}{4}. +\begin{align*} +\int_0^3 \left(\frac{(t-2)^3}{3} - \frac{4(t-2)}{3}\right) dt +&= \int_{u(0)}^{u(3)} \left(\frac{u^3}{3} - \frac{4u}{3} \right) du\\ +&= \left(\frac{u^4}{12} - \frac{4}{3}\cdot\frac{u^2}{2}\right) \Big|_{-2}^1\\ +&= \frac{3}{4}. +\end{align*} $$ The speed is similar, but we have to work harder: $$ -\int_0^3 \lvert v(t) \rvert dt = \int_0^3 \lvert ((t-2)^3/3 - 4(t-2)/3) \rvert dt = -\int_{-2}^1 \lvert u^3/3 - 4u/3 \rvert du. +\begin{align*} +\int_0^3 \lvert v(t) \rvert dt &= \int_0^3 \lvert \left(\frac{(t-2)^3}{3} - \frac{4(t-2)}{3}\right) \rvert dt\\ +&= \int_{-2}^1 \lvert \frac{u^3}{3} - \frac{4u}{3} \rvert du. +\end{align*} $$ But $u^3/3 - 4u/3 = (1/3) \cdot u(u-2)(u+2)$, so between $-2$ and $0$ it is positive and between $0$ and $1$ negative, so this integral is: @@ -252,9 +256,9 @@ But $u^3/3 - 4u/3 = (1/3) \cdot u(u-2)(u+2)$, so between $-2$ and $0$ it is posi $$ \begin{align*} -\int_{-2}^0 (u^3/3 - 4u/3 ) du + \int_{0}^1 -(u^3/3 - 4u/3) du -&= (\frac{u^4}{12} - \frac{4}{3}\frac{u^2}{2}) \big|_{-2}^0 - (\frac{u^4}{12} - \frac{4}{3}\frac{u^2}{2}) \big|_{0}^1\\ -&= \frac{4}{3} - -\frac{7}{12}\\ +\int_{-2}^0 \left(\frac{u^3}{3} - \frac{4u}{3} \right) du &+ \int_{0}^1 -\left(\frac{u^3}{3} - \frac{4u}{3}\right) du\\ +&= \left(\frac{u^4}{12} - \frac{4}{3}\frac{u^2}{2}\right) \Big|_{-2}^0 - \left(\frac{u^4}{12} - \frac{4}{3}\frac{u^2}{2}\right) \Big|_{0}^1\\ +&= \frac{4}{3} - \left(-\frac{7}{12}\right)\\ &= \frac{23}{12}. \end{align*} $$ @@ -337,7 +341,7 @@ The above calculation is for illustration purposes. The add-on package `Distribu ::: -## SymPy and substitution +### SymPy and *u*-substitution The `integrate` function in `SymPy` can handle most problems which involve substitution. Here are a few examples: @@ -356,11 +360,11 @@ integrate(4x / sqrt(x^2 + 1), (x, 0, 2)) ```{julia} #| hold: true -f(x) = 1/(x*log(x)) -integrate(f(x), (x, sympy.E, sympy.E^2)) +E = sympy.E +integrate(1/(x*log(x)), (x, E, E^2)) ``` -(We used `sympy.E` - and not `e` - to avoid any conversion to floating point, which could yield an inexact answer.) +(We used `sympy.E`---and not `e` or even constant `ℯ`---to avoid any conversion to floating point with $e^2$, which could yield an inexact answer.) The antiderivative is interesting here; it being an *iterated* logarithm. @@ -370,7 +374,7 @@ The antiderivative is interesting here; it being an *iterated* logarithm. integrate(1/(x*log(x)), x) ``` -### Failures... +#### Failures... Not every integral problem lends itself to solution by substitution. For example, we can use substitution to evaluate the integral of $xe^{-x^2}$, but for $e^{-x^2}$ or $x^2e^{-x^2}$. The first has no familiar antiderivative, the second is done by a different technique. @@ -379,7 +383,7 @@ Not every integral problem lends itself to solution by substitution. For example Even when substitution can be used, `SymPy` may not be able to algorithmically identify it. The main algorithm used can determine if expressions involving rational functions, radicals, logarithms, and exponential functions is integrable. Missing from this list are absolute values. -For some such problems, we can help `SymPy` out - by breaking the integral into pieces where we know the sign of the expression. +For some such problems, we can help `SymPy` out---by breaking the integral into pieces where we know the sign of the expression. For substitution problems, we can also help out. For example, to find an antiderivative for @@ -393,8 +397,8 @@ A quick attempt with `SymPy` turns up nothing: ```{julia} -𝒇(x) = (1 + log(x)) * sqrt(1 + (x*log(x))^2 ) -integrate(𝒇(x), x) +f(x) = (1 + log(x)) * sqrt(1 + (x*log(x))^2 ) +integrate(f(x), x) ``` But were we to try $u=x\log(x)$, we'd see that this simplifies to $\int \sqrt{1 + u^2} du$, which has some hope of having an antiderivative. @@ -406,7 +410,7 @@ We can help `SymPy` out by substitution: ```{julia} u(x) = x * log(x) @syms w dw -ex = 𝒇(x) +ex = f(x) ex₁ = ex(u(x) => w, diff(u(x),x) => dw) ``` @@ -441,7 +445,7 @@ This can be found using *trigonometric* substitution. In this example, we know t $$ -\int \sqrt{1 + x^2} dx = \int \sec(u)^2 \lvert \sec(u) \rvert du = \int \sec(u)^3 du, +\int \sqrt{1 + x^2} dx = \int \lvert \sec(u) \rvert \sec(u)^2 du = \int \sec(u)^3 du, $$ if we know $\sec(u) \geq 0$. @@ -491,7 +495,7 @@ integrate(1 / (a^2 + (b*x)^2), x) ##### Example -The expression $1-x^2$ can be attacked by the substitution $\sin(u) =x$ as then $1-x^2 = 1-\sin(u)^2 = \cos(u)^2$. Here we see this substitution being used successfully: +The expression $1-x^2$ can be attacked by the substitution $\sin(u) =x$ as then $1-x^2 = 1-\sin(u)^2 = \cos(u)^2$. Here we see the substitution $3\sin(u) = x$ with $3\cos(u)du = dx$ being used successfully: $$ @@ -500,7 +504,7 @@ $$ &=\int \frac{1}{3\sqrt{1 - \sin(u)^2}}\cdot3\cos(u) du \\ &= \int du \\ &= u \\ -&= \sin^{-1}(x/3). +&= \sin^{-1}(\frac{x}{3}). \end{align*} $$ @@ -514,6 +518,8 @@ Further substitution allows the following integral to be solved for an antideriv integrate(1 / sqrt(a^2 - b^2*x^2), x) ``` +In the above, we implicitly assumed the argument to square root function was non-negative which is the second of these two cases found in general. + ##### Example @@ -541,7 +547,7 @@ $$ $$ -SymPy gives a different representation using the arccosine: +SymPy readily handles the more general case: ```{julia} @@ -560,16 +566,18 @@ We need to compute: $$ -2\int_{-a}^a b \sqrt{1 - x^2/a^2} dx = -4 b \int_0^a\sqrt{1 - x^2/a^2} dx. +2\int_{-a}^a b \sqrt{1 - \frqc{x^2}{a^2}} dx = +4 b \int_0^a\sqrt{1 - \frac{x^2}{a^2}} dx. $$ Letting $\sin(u) = x/a$ gives $a\cos(u)du = dx$ and an antiderivative is found with: $$ -4 b \int_0^a \sqrt{1 - x^2/a^2} dx = 4b \int_0^{\pi/2} \sqrt{1-\sin(u)^2} a \cos(u) du -= 4ab \int_0^{\pi/2} \cos(u)^2 du +\begin{align*} +4 b \int_0^a \sqrt{1 - \frac{x^2}{a^2}} dx &= 4b \int_0^{\pi/2} \sqrt{1-\sin(u)^2} a \cos(u) du\\ +&= 4ab \int_0^{\pi/2} \cos(u)^2 du. +\end{align*} $$ The identify $\cos(u)^2 = (1 + \cos(2u))/2$ makes this tractable: @@ -579,7 +587,7 @@ $$ \begin{align*} 4ab \int \cos(u)^2 du &= 4ab\int_0^{\pi/2}(\frac{1}{2} + \frac{\cos(2u)}{2}) du\\ -&= 4ab(\frac{1}{2}u + \frac{\sin(2u)}{4})\big|_0^{\pi/2}\\ +&= 4ab(\frac{1}{2}u + \frac{\sin(2u)}{4})\Big|_0^{\pi/2}\\ &= 4ab (\pi/4 + 0) = \pi ab. \end{align*} $$ @@ -605,8 +613,8 @@ choices = [ "``\\int u (1 - u^2) du``", "``\\int u \\cos(x) du``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -624,8 +632,8 @@ choices = [ "``u=\\sec(x)``", "``u=\\sec(x)^2``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -643,8 +651,8 @@ choices = [ "``u=\\sqrt{x^2 - 1}``", "``u=x``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -671,7 +679,7 @@ yesnoq("yes") ###### Question -For $\int (\log(x))^3/x dx$ the substitution $u=\log(x)$ reduces this to what? +For $\int (\log(x))^3/x \cdot dx$ the substitution $u=\log(x)$ reduces this to what? ```{julia} @@ -682,8 +690,8 @@ choices = [ "``\\int u du``", "``\\int u^3/x du``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -700,8 +708,8 @@ choices = [ "``u=\\sin(x)``", "``u=\\tan(x)``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -726,10 +734,100 @@ choices = [ "``a=0,~ b=0``", "``a=1,~ b=1``" ] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` +###### Question + +Some integration problems require trigonometric identities, not trigonometric substitution. + +We consider first the integral + +$$ +\int_0^\pi \cos(\theta) \sin(\theta) d\theta +$$ + +Compute this using substitution or by noting the resemblance to the double angle formula $\sin(2\theta) = 2\sin(\theta)\cos(\theta)$. + +```{julia} +#| echo: false +let + @syms x + val = integrate(cos(x)*sin(x), (x, 0, PI)) + numericq(float(val)) +end +``` + +Now let $k > 1$ be an integer. We wish to find the integral of + +$$ +I = \int_0^\pi \cos(k\theta) \cdot \sin(\theta) d\theta +$$ + +First, we combine the two formulas + +$$ +\begin{align*} +\sin(a + b) &= \sin(a) \cos(b) + \cos(a) \sin(b)\\ +\sin(a - b) &= \sin(a) \cos(b) - \cos(a) \sin(b) +\end{align*} +$$ + +to get + +$$ +\sin(a + b) + \sin(a-b) = 2 \sin(a) \cos(b) +$$ + +Taking $a=\theta$ and $b = k\theta$, what is a re-expression for the integrand $\cos(k\theta) \sin(\theta)$? + +```{julia} +#| echo: false +choices = [ +L"1/2 \cdot (\sin((1 + k)\theta) + \sin((1-k)\theta))", +L"1/2 \cdot (\cos((1 + k) \theta) + \cos((1-k)\theta))", +L"\cos(\theta) \sin(k \theta)" +] +answer = 1 +buttonq(choices, answer) +``` + +Using substitution (e.g. $u=a\theta$) or some other means find the value of + +$$ +\int_0^\pi \sin(a \theta) d\theta. +$$ + + +```{julia} +#| echo: false +choices = [L"1/a - \cos(a\cdot \pi)/a", + L"1/a"] +answer = 1 +buttonq(choices, answer) +``` + +When $a=k+1$ or $k-1$ we have to evaluate $\cos((k+1)\pi$ or $\cos((k-1)\pi)$. These have the same value, which can be seen by writing, say, $\cos((k +1)\pi) = \cos(k\pi + \pi)$. What is the value? + +```{julia} +#| echo: false +choices = [L"-\cos(\pi)", + L"-\cos(k\pi)"] +answer = 2 +explanation = L"the extra $\pi$ just rotates the angle half way around the unit circle so changes the sign but not the magnitude of the cosine of $k\pi$" +buttonq(choices, answer; explanation) +``` + +Combining, this gives a value of + +$$ +I = \frac{1 + \cos(k\pi)}{1 - k^2}. +$$ + + + + ###### Question @@ -745,14 +843,14 @@ choices = [ "``\\sec(u) = x``", "``u = 1 - x^2``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question -The integral $\int x/(1+x^2) dx$ lends itself to what substitution? +The integral $\int x/(1+x^2) \cdot dx$ lends itself to what substitution? ```{julia} @@ -764,8 +862,8 @@ choices = [ "``\\tan(u) = x``", "``\\sec(u) = x``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -783,8 +881,8 @@ choices = [ "``\\sec(u) = x``", "``u = 1 - x^2``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -801,8 +899,8 @@ choices = [ "``\\sec(u) = x``", "``4\\sin(u) = x``", "``\\sin(u) = x``"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -819,8 +917,8 @@ choices = [ "``\\tan(u) = x``", "``a\\sec(u) = x``", "``\\sec(u) = x``"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -845,8 +943,8 @@ choices =[ "``a=\\pi/3,~ b=\\pi/2``", "``a=1/2,~ b= 1``" ] -answ =1 -radioq(choices, answ) +answer =1 +radioq(choices, answer) ``` ###### Question @@ -861,6 +959,6 @@ How would we verify that $\log\lvert (\sec(u) + \tan(u))\rvert$ is an antideriva choices = [ L"We could differentiate $\sec(u)$.", L"We could differentiate $\log\lvert (\sec(u) + \tan(u))\rvert$ "] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` diff --git a/quarto/integrals/surface_area.qmd b/quarto/integrals/surface_area.qmd index e6b8f8c..321fd7d 100644 --- a/quarto/integrals/surface_area.qmd +++ b/quarto/integrals/surface_area.qmd @@ -9,8 +9,7 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia -using Plots -plotly() +using Plots; plotly() using SymPy using QuadGK ``` @@ -21,38 +20,22 @@ using QuadGK ## Surfaces of revolution +::: {#fig-gehry-hendrix-museum} -```{julia} -#| hold: true -#| echo: false -imgfile = "figures/gehry-hendrix.jpg" -caption = """ +![](./figures/gehry-hendrix.jpg) The exterior of the Jimi Hendrix Museum in Seattle has the signature style of its architect Frank Gehry. The surface is comprised of patches. A general method to find the amount of material to cover the -surface - the surface area - might be to add up the area of *each* of the +surface---the surface area---might be to add up the area of *each* of the patches. However, in this section we will see for surfaces of revolution, there is an easier way. (Photo credit to [firepanjewellery](http://firepanjewellery.com/).) -""" +::: -# ImageFile(:integrals, imgfile, caption) -nothing -``` +In this section we see how to find the surface area of volumes generated by revolution. -![The exterior of the Jimi Hendrix Museum in Seattle has the signature -style of its architect Frank Gehry. The surface is comprised of -patches. A general method to find the amount of material to cover the -surface - the surface area - might be to add up the area of *each* of the -patches. However, in this section we will see for surfaces of -revolution, there is an easier way. (Photo credit to -[firepanjewellery](http://firepanjewellery.com/).) -](./figures/gehry-hendrix.jpg) - - -::: {.callout-note icon=false} -## Surface area of a rotated curve +::: {.definition title="Surface area of a rotated curve"} The surface area generated by rotating the graph of $f(x)$ between $a$ and $b$ about the $x$-axis is given by the integral @@ -70,7 +53,7 @@ These formulas do not add in the surface area of either of the ends. ::: - +::: {#fig-surface-revolution-cone} ```{julia} #| hold: true #| echo: false @@ -82,6 +65,9 @@ surface(ws..., legend=false) plot!([-0.5,1.5], [0,0],[0,0]) ``` +Surface of revolution forming a cone +::: + The above figure shows a cone (the line $y=x$) presented as a surface of revolution about the $x$-axis. @@ -234,7 +220,7 @@ Illustration of function $(g(t), f(t))$ rotated about the $x$ axis with a secti -Consider a right-circular cone parameterized by an angle $\theta$ which at a given height has radius $r$ and slant height $l$ (so that the height satisfies $r/l=\sin(\theta)$). If this cone were made of paper, cut up a side, and laid out flat, it would form a sector of a circle, as illustrated below: +Consider a right-circular cone parameterized by an angle $\theta$ which at a given height has radius $r$ and slant height $l$ (so that the height satisfies $r/l=\sin(\theta)$). If this cone were made of paper, cut up a side, and laid out flat, it would form a sector of a circle, as illustrated in @fig-frustum-cone-area. ::: {#fig-frustum-cone-area} @@ -433,8 +419,10 @@ Putting this altogether we get that the surface area generarated by rotating the $$ -\text{sa}_i = \pi (f(t_i)^2 - f(t_{i-1})^2) \cdot \sqrt{(\Delta g)^2 + (\Delta f)^2} / \Delta f = -2\pi \frac{f(t_i) + f(t_{i-1})}{2} \cdot \sqrt{(\Delta g)^2 + (\Delta f)^2}. +\begin{align*} +\text{sa}_i &= \pi \left(f(t_i)^2 - f(t_{i-1})^2\right) \cdot \sqrt{(\Delta g)^2 + (\Delta f)^2} / \Delta f\\ +&= 2\pi \frac{f(t_i) + f(t_{i-1})}{2} \cdot \sqrt{(\Delta g)^2 + (\Delta f)^2}. +\end{align*} $$ (This is $2 \pi$ times the average radius times the slant height.) @@ -444,10 +432,10 @@ As was done in the derivation of the formula for arc length, these pieces are mu $$ -\text{sa}_i = \pi (f(t_i) + f(t_{i-1})) \cdot \sqrt{(g'(\xi))^2 + (f'(\psi))^2} \cdot (t_i - t_{i-1}). +\text{sa}_i = \pi \left(f(t_i) + f(t_{i-1})\right) \cdot \sqrt{(g'(\xi))^2 + (f'(\psi))^2} \cdot (t_i - t_{i-1}). $$ -Adding these up, $\text{sa}_1 + \text{sa}_2 + \cdots + \text{sa}_n$, we get a Riemann sum approximation to the integral +Adding these up, $\text{sa}_1 + \text{sa}_2 + \cdots + \text{sa}_n$, we get a Riemann sum approximation to the integral: $$ @@ -472,7 +460,7 @@ $$ \begin{align*} \int_0^h 2\pi f(x) \sqrt{1 + f'(x)^2}dx &= \int_0^h 2\pi x \tan(\theta) \sqrt{1 + \tan(\theta)^2}dx \\ -&= (2\pi\tan(\theta)\sqrt{1 + \tan(\theta)^2}) x^2/2 \big|_0^h \\ +&= (2\pi\tan(\theta)\sqrt{1 + \tan(\theta)^2}) \frac{x^2}{2} \Big|_0^h \\ &= \pi \tan(\theta) \sec(\theta) h^2 \\ &= \pi r^2 / \sin(\theta). \end{align*} @@ -522,47 +510,65 @@ f(u) = 2cos(u) a, b = 0, 2pi ``` -The plot of this curve is: - +The plot of this curve is shown in @fig-plot-of-some-circle-begin-rotated. +::: {#fig-plot-of-some-circle-begin-rotated} ```{julia} #| hold: true us = range(a, b, length=100) plot(g.(us), f.(us), xlims=(-0.5, 9), aspect_ratio=:equal, legend=false) -plot!([(0, -3), (0, 3)], line=(:red, 5)) # z axis emphasis -plot!([(3, 0), (9, 0)], line=(:green, 5)) # x axis emphasis +plot!([(0, -3), (0, 3)], line=(5, :red)) # z axis emphasis +plot!([(3, 0), (9, 0)], line=(5, :green)) # x axis emphasis ``` +Plot of curve to be rotated to form a torus +::: -Though parametric plots have a convenience constructor, `plot(g, f, a, b)`, we constructed the points with `Julia`'s broadcasting notation, as we will need to do for a surface of revolution. The `xlims` are adjusted to show the $y$ axis, which is emphasized with a layered line. The line is drawn by specifying two points, $(x_0, y_0)$ and $(x_1, y_1)$ using tuples and wrapping in a vector. +Though parametric plots have a convenience constructor, `plot(g, f, a, b)`, we constructed the points with `Julia`'s broadcasting notation, as we will need to do for a surface of revolution. The `xlims` are adjusted to show the $y$ axis, which is emphasized with a layered line. (The line is drawn by specifying two points, $(x_0, y_0)$ and $(x_1, y_1)$, using tuples and wrapping in a vector.) -Now, to rotate this about the $z$ axis, creating a surface plot, we have the following pattern: +Now, to rotate this about the $z$ axis, creating a surface plot, we have the following pattern. First we form a function $S$ of two variables in terms of $g$ and $f$: ```{julia} S(u,v) = [g(u)*cos(v), g(u)*sin(v), f(u)] +``` + +The steps to plot the surface are then always similar, save for possibly adjustments to the viewing window, as is done with `zlims` in forming @fig-torus-plotted-as-rotated-parameterized-circle-of-radius-2 + +::: {#fig-torus-plotted-as-rotated-parameterized-circle-of-radius-2} +```{julia} us = range(a, b, length=100) vs = range(0, 2pi, length=100) -ws = unzip(S.(us, vs')) # reorganize data -surface(ws..., zlims=(-6,6), legend=false) -plot!([(0,0,-3), (0,0,3)], line=(:red, 5)) # z axis emphasis +ws = unzip(S.(us, vs')) # reorganize data into 3 vectors + +surface(ws...; zlims=(-10,10), legend=false) +plot!([(0, 0, -10), (0, 0, 10)]; line=(5, :red, 0.25)) # add axis of rotation ``` + +A circle of radius $2$ rotated about the $z$ axis forms a torus +::: + The `unzip` function is not part of base `Julia`, rather part of `CalculusWithJulia` (it is really `SplitApplyCombine`'s `invert` function). This function rearranges data into a form consumable by the plotting methods like `surface`. In this case, the result of `S.(us,vs')` is a grid (matrix) of points, the result of `unzip` is three grids of values, one for the $x$ values, one for the $y$ values, and one for the $z$ values. A manual adjustment to the `zlims` is used, as `aspect_ratio` does not have an effect with the `plotly()` backend. -To rotate this about the $x$ axis, we have this pattern: - +To rotate this region about the $x$ axis, we have the pattern forming @fig-surface-of-rotation-formed-by-rotating-about-x-axis. +::: {#fig-surface-of-rotation-formed-by-rotating-about-x-axis} ```{julia} S(u,v) = [g(u), f(u)*cos(v), f(u)*sin(v)] + us = range(a, b, length=100) vs = range(0, 2pi, length=100) ws = unzip(S.(us,vs')) -plot([(3,0,0), (9,0,0)], line=(:green,5)) # x axis emphasis -surface!(ws..., legend=false) + +surface(ws...; zlims=(-3,3), legend=false) +plot!([(3,0,0), (9,0,0)], line=(5, :green)) # emphasize axis of rotation ``` -The above pattern covers the case of rotating the graph of a function $f(x)$ of $a,b$ by taking $g(t)=t$. +Figure showing rotation of circle parameterized by $(g, f)$ being rotated around the $x$ axis +::: + +The above pattern covers the case of rotating the graph of a function $f(x)$ over $[a,b]$ by taking $g(t)=t$. ##### Example @@ -584,17 +590,19 @@ val (The function is not defined at $x=0$ mathematically, but is on the computer to be $1$, the limiting value. Even were this not the case, the `quadgk` function doesn't evaluate the function at the points `a` and `b` that are specified.) - +::: {#fig-rotate-x-to-x-about-x-axis} ```{julia} #| hold: true g(u) = u f(u) = u^u S(u,v) = [g(u), f(u)*cos(v), f(u)*sin(v)] us = range(0, 3/2, length=100) -vs = range(0, pi, length=100) # not 2pi (to see inside) +vs = range(0, pi, length=100) # not 2pi (to see inside) ws = unzip(S.(us,vs')) surface(ws..., alpha=0.75) ``` +Partial rotation of $x^x$ about the $x$ axis +::: We compare this answer to that of the frustum of a cone with radii $1$ and $(3/2)^2$, formed by rotating the line segment connecting $(0,f(0))$ with $(3/2,f(3/2))$. From looking at the graph of the surface, these values should be comparable. The surface area of the cone part is $\pi (r_1^2 - r_0^2) / \sin(\theta) = \pi (r_1 + r_0) \cdot \sqrt{(\Delta h)^2 + (r_1-r_0)^2}$. @@ -613,8 +621,10 @@ What is the surface area generated by Gabriel's Horn, the solid formed by rotati $$ -\text{SA} = \int_a^b 2\pi f(x) \sqrt{1 + f'(x)^2}dx = -\lim_{M \rightarrow \infty} \int_1^M 2\pi \frac{1}{x} \sqrt{1 + (-1/x^2)^2} dx. +\begin{align*} +\text{SA} &= \int_1^\infty 2\pi f(x) \sqrt{1 + f'(x)^2}dx \\ +&= \lim_{M \rightarrow \infty} \int_1^M 2\pi \frac{1}{x} \sqrt{1 + (-1/x^2)^2} dx. +\end{align*} $$ We do this with `SymPy`: @@ -632,7 +642,7 @@ The limit as $M$ gets large is of interest. The only term that might get out of limit(asinh(M), M => oo) ``` -So indeed it does. There is nothing to balance this out, so the integral will be infinite, as this shows: +So indeed that term gets out of hand. There is nothing to balance this out, so the integral will be infinite, as this shows: ```{julia} @@ -648,9 +658,9 @@ This figure would have infinite surface, were it possible to actually construct The curve described parametrically by $g(t) = 2(1 + \cos(t))\cos(t)$ and $f(t) = 2(1 + \cos(t))\sin(t)$ from $0$ to $\pi$ is rotated about the $x$ axis. Find the resulting surface area. -The graph shows half a heart, the resulting area will resemble an apple. - +@fig-rotate-heart-to-get-apple shows half a heart, the resulting rotated surface area will resemble an apple. +::: {#fig-rotate-heart-to-get-apple} ```{julia} #| hold: true g(t) = 2(1 + cos(t)) * cos(t) @@ -658,6 +668,9 @@ f(t) = 2(1 + cos(t)) * sin(t) plot(g, f, 0, 1pi) ``` +Paremeterized curve to rotate about $x$ axis +::: + The integrand simplifies to $8\sqrt{2}\pi \sin(t) (1 + \cos(t))^{3/2}$. This lends itself to $u$-substitution with $u=\cos(t)$. @@ -665,7 +678,7 @@ $$ \begin{align*} \int_0^\pi 8\sqrt{2}\pi \sin(t) (1 + \cos(t))^{3/2} &= 8\sqrt{2}\pi \int_1^{-1} (1 + u)^{3/2} (-1) du\\ -&= 8\sqrt{2}\pi (2/5) (1+u)^{5/2} \big|_{-1}^1\\ +&= 8\sqrt{2}\pi (2/5) (1+u)^{5/2} \Big|_{-1}^1\\ &= 8\sqrt{2}\pi (2/5) 2^{5/2} = \frac{2^7 \pi}{5}. \end{align*} $$ @@ -681,13 +694,13 @@ $$ \text{SA} = 2 \pi \rho L $$ -That is, the surface area is simply the circumference of the circle traced out by the centroid of the curve times the length of the curve - the distances rotated are collapsed to that of just the centroid. +That is, the surface area is simply the circumference of the circle traced out by the centroid of the curve times the length of the curve---the distances rotated are collapsed to that of just the centroid. ##### Example -The surface area of an open cone can be computed, as the arc length is $\sqrt{h^2 + r^2}$ and the centroid of the line is a distance $r/2$ from the axis. This gives SA$=2\pi (r/2) \sqrt{h^2 + r^2} = \pi r \sqrt{h^2 + r^2}$. +The surface area of an open cone can be computed, as the arc length is $\sqrt{h^2 + r^2}$ and the centroid *of the line* is a distance $r/2$ from the axis. This gives $\text{SA} = 2\pi (r/2) \sqrt{h^2 + r^2} = \pi r \sqrt{h^2 + r^2}$. ##### Example @@ -696,22 +709,9 @@ The surface area of an open cone can be computed, as the arc length is $\sqrt{h^ We can get the surface area of a torus from this formula. -The torus is found by rotating the curve $(x-b)^2 + y^2 = a^2$ about the $y$ axis. The centroid is $b$, the arc length $2\pi a$, so the surface area is $2\pi (b) (2\pi a) = 4\pi^2 a b$. +The torus is found by rotating the curve $(x-b)^2 + y^2 = a^2$ about the $y$ axis. The centroid is $b$, the arc length $2\pi a$, so the surface area is $2\pi (b) (2\pi a) = 4\pi^2 a b$. A torus with $a=2$ and $b=6$ was plotted for @fig-torus-plotted-as-rotated-parameterized-circle-of-radius-2. -A torus with $a=2$ and $b=6$ - - -```{julia} -#| hold: true -#| echo: false -a,b = 2, 6 -F₀(u,v) = [a*(cos(u) + b)*cos(v), a*(cos(u) + b)*sin(v), a*sin(u)] -us = vs = range(0, 2pi, length=35) -ws = unzip(F₀.(us, vs')) -surface(ws..., legend=false, zlims=(-12,12)) -``` - ##### Example @@ -720,8 +720,8 @@ The surface area of sphere will be SA$=2\pi \rho (\pi r) = 2 \pi^2 r \cdot \rho$ $$ \begin{align*} -\text{cm}_x &= \frac{1}{L} \int_a^b g(t) \sqrt{g'(t)^2 + f'(t)^2} dt\\ -\text{cm}_y &= \frac{1}{L} \int_a^b f(t) \sqrt{g'(t)^2 + f'(t)^2} dt. +\overline{\text{cm}}_x &= \frac{1}{L} \int_a^b g(t) \sqrt{g'(t)^2 + f'(t)^2} dt\\ +\overline{\text{cm}}_y &= \frac{1}{L} \int_a^b f(t) \sqrt{g'(t)^2 + f'(t)^2} dt. \end{align*} $$ @@ -733,11 +733,14 @@ For the sphere parameterized by $g(t) = r \cos(t)$, $f(t) = r\sin(t)$, we get th $$ -\text{cm}_x = \frac{1}{L}\int_0^\pi r\cos(t) \sqrt{r^2(\sin(t)^2 + \cos(t)^2)} dt = \frac{1}{L}r^2 \int_0^\pi \cos(t) = 0. -$$ - -$$ -\text{cm}_y = \frac{1}{L}\int_0^\pi r\sin(t) \sqrt{r^2(\sin(t)^2 + \cos(t)^2)} dt = \frac{1}{L}r^2 \int_0^\pi \sin(t) = \frac{1}{\pi r} r^2 \cdot 2 = \frac{2r}{\pi}. +\begin{align*} +\overline{\text{cm}}_x &= \frac{1}{L}\int_0^\pi r\cos(t) \sqrt{r^2(\sin(t)^2 + \cos(t)^2)} dt\\ +&= \frac{1}{L}r^2 \int_0^\pi \cos(t)\\ +&= 0\\ +\overline{\text{cm}}_y &= \frac{1}{L}\int_0^\pi r\sin(t) \sqrt{r^2(\sin(t)^2 + \cos(t)^2)} dt\\ +&= \frac{1}{L}r^2 \int_0^\pi \sin(t) \\ +&= \frac{1}{\pi r} r^2 \cdot 2 = \frac{2r}{\pi}. +\end{align*} $$ Combining this, we see that the surface area of a sphere is $2 \pi^2 r (2r/\pi) = 4\pi r^2$, by Pappus' Theorem. @@ -760,8 +763,8 @@ choices = [ "``-\\int_1^{_1} 2\\pi u \\sqrt{1 + u^2} du``", "``-\\int_1^{_1} 2\\pi u^2 \\sqrt{1 + u} du``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` Though the integral can be computed by hand, give a numeric value. @@ -830,8 +833,8 @@ choices = [ "``\\int_u^{u_h} 2\\pi y dx``", "``\\int_u^{u_h} 2\\pi x dx``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ##### Questions diff --git a/quarto/integrals/twelve-qs.qmd b/quarto/integrals/twelve-qs.qmd index 947e794..7dac73e 100644 --- a/quarto/integrals/twelve-qs.qmd +++ b/quarto/integrals/twelve-qs.qmd @@ -16,8 +16,11 @@ gr(); --- -In the March 2003 issue of the College Mathematics Journal, Leon M Hall posed 12 questions related to the following figure: +In the March 2003 issue of the College Mathematics Journal, Leon M Hall posed 12 questions related to @fig-parabola-tangent-normal-CMJ. The figure shows $f(x) = x^2$, the tangent line at $P = (a, f(a))$ (for $a > 0$), and the *normal* line at $(a, f(a))$. The questions all involve finding the value $a$ which minimizes a related quantity that has been previously discussed. + + +::: {#fig-parabola-tangent-normal-CMJ} ```{julia} #| echo: false a₀ = 7/8 @@ -63,10 +66,11 @@ end make_plot() ``` -The figure shows $f(x) = x^2$, the tangent line at $(a, f(a))$ (for $a > 0$), and the *normal* line at $(a, f(a))$. The questions all involve finding the value $a$ which minimizes a related quantity. +A parabola with tangent and normal line at point $P$ +::: -We set up some variables to work symbolically: +The solutions are a bit of fun. To approach them, we set up some variables to work symbolically: ```{julia} @syms a::positive x::real @@ -86,6 +90,7 @@ The first question is simply: > 1a. The $y$ coordinate of $Q$ +::: {#fig-parabola-tangent-normal-CMJ-y-coordinate} ```{julia} #| echo: false let @@ -94,6 +99,9 @@ let end ``` +Emphasis of $y$ coordinate of $Q$ +::: + The value is $f(q)$ ```{julia} @@ -111,7 +119,7 @@ The lone critical point must be at a minimum. (Given the geometry of the problem ::: {.callout-note} ## We hide the code -In the remaining examples we don't show the code by default. +In the remaining examples we don't immediately show the code to describe the value to optimize; it is hidden in a collapsed block. ::: @@ -119,12 +127,15 @@ In the remaining examples we don't show the code by default. > 1b. The length of the line segment $PQ$ +::: {#fig-parabola-tangent-normal-CMJ-PQ-line-segment} ```{julia} #| echo: false p = make_plot() plot!([q₀, a₀], [f(q₀), f(a₀)], linewidth=5) ``` +Emphasis of line segment connecting $P$ and $Q$ +::: ```{julia} #| code-fold: true @@ -138,12 +149,15 @@ lseg = sqrt((f(a) - f(q))^2 + (a - q)^2); > 2a. The horizontal distance between $P$ and $Q$ +::: {#fig-parabola-tangent-normal-CMJ-PQ-horizontal-distance-PQ} ```{julia} #| echo: false p = make_plot() plot!([q₀, a₀], [f(a₀), f(a₀)], linewidth=5) ``` +Emphasis of horizontal distance between $P$ and $Q$ +::: ```{julia} #| code-fold: true @@ -155,7 +169,7 @@ hd = a - q; > 2b. The area of the parabolic segment - +::: {#fig-parabola-tangent-normal-CMJ-area-parabolic-segment} ```{julia} #| echo: false p = make_plot() @@ -165,6 +179,8 @@ ys = vcat(f.(xs′), normal.(reverse(xs′)), f(first(xs′))) plot!(xs, ys, fill=(:green, 0.25, 0)) ``` +Emphasis of parabolic segment formed by the normal line +::: ```{julia} #| code-fold: true @@ -189,14 +205,15 @@ V = simplify(integrate(2PI*(nl-f(x))*(a - x + k),(x, q, a))); > 3. The $y$ coordinate of the centroid of the parabolic segment - +::: {#fig-parabola-tangent-normal-CMJ-centroid-parabolic-segment} ```{julia} #| echo: false p = make_plot() scatter!(p, [-1/(4a₀)], [1], marker=(10, :diamond)) p ``` - +Emphasis of centrood of the parabolic segment formed by the normal line +::: We warm up with the $x$ coordinate, given by: @@ -218,7 +235,7 @@ yₘ = simplify(yₘ); > 4. The length of the arc of the parabola between $P$ and $Q$ - +::: {#fig-parabola-tangent-normal-CMJ-arc-length-parabola} ```{julia} #| echo: false p = make_plot() @@ -227,6 +244,9 @@ plot!(xs, f.(xs), linewidth=5) p ``` +Emphasis of arc-length along parabola +::: + ```{julia} #| code-fold: true #| code-summary: "Show the code" @@ -237,7 +257,7 @@ L = integrate(sqrt(1 + fp(x)^2), (x, q, a)); > 5. The $y$ coordinate of the midpoint of the line segment $PQ$ - +::: {#fig-parabola-tangent-normal-CMJ-y-coordinate-of-PQ} ```{julia} #| echo: false p = make_plot() @@ -247,6 +267,8 @@ scatter!([mid], [normal(mid)], markersize=5) p ``` +Emphasis of $y$ intercept for line $PQ$ +::: ```{julia} #| code-fold: true @@ -258,7 +280,7 @@ mp = nl(x => (a + q)/2); > 6. The area of the trapezoid bound by the normal line, the $x$-axis, and the vertical lines through $P$ and $Q$. - +::: {#fig-parabola-tangent-normal-CMJ-trapezoid-normal-segment} ```{julia} #| echo: false p = make_plot() @@ -267,6 +289,10 @@ plot!([q₀, a₀, a₀, q₀, q₀], p ``` +Emphasis of traapezoid bounded by the normal line +::: + + ```{julia} #| code-fold: true #| code-summary: "Show the code" @@ -277,6 +303,7 @@ trap = 1//2 * (f(q) + f(a)) * (a - q); > 7. The area bounded by the parabola and the $x$ axis and the vertical lines through $P$ and $Q$ +::: {#fig-parabola-tangent-normal-CMJ-area-under-parabola} ```{julia} #| echo: false p = make_plot() @@ -287,6 +314,8 @@ plot!(xs, ys, fill=(:green, 0.25, 0)) p ``` +Area under the parabola between $P$ and $Q$ +::: ```{julia} @@ -299,6 +328,7 @@ pa = integrate(x^2, (x, q, a)); > 8. The area of the surface formed by revolving the arc of the parabola between $P$ and $Q$ around the vertical line through $P$ +::: {#fig-parabola-tangent-normal-CMJ-surface-area-rotation} ```{julia} #| echo: false let @@ -310,6 +340,8 @@ let end ``` +Surface area formed by revolving arc through $P$ +::: ```{julia} @@ -325,6 +357,7 @@ SA = 2PI * integrate(uu(x) * sqrt(diff(uu(x),x)^2 + diff(vv(x),x)^2), (x, q, a)) > 9. The height of the parabolic segment (i.e. the distance between the normal line and the tangent line to the parabola that is parallel to the normal line) +::: {#fig-parabola-tangent-normal-CMJ-height-parabolic-segment} ```{julia} #| echo: false # distance point to a line @@ -339,6 +372,9 @@ plot!(x -> f(b₀) + (-1/fp(a₀))*(x - b₀), -1, 1/2) plot!([b₀,x₀], [f(b₀), normal(x₀)]; linewidth=5) ``` +Emphasis of the height of the parabolic segment +::: + ```{julia} #| code-fold: true #| code-summary: "Show the code" @@ -354,6 +390,7 @@ segment_height = sqrt((b-b′)^2 + (f(b) - nl(x=>b′))^2); > 10. The volume of the solid formed by revolving the parabolic segment around the $x$-axis +::: {#fig-parabola-tangent-normal-CMJ-volume-formed-on-revolving-parabolic-segment} ```{julia} #| echo: false let @@ -365,6 +402,8 @@ let end ``` +Volume formed by revolving parabolic segment around $x$ axis +::: ```{julia} #| code-fold: true #| code-summary: "Show the code" @@ -375,6 +414,7 @@ Vₓ = integrate(pi * (nl^2 - f(x)^2), (x, q, a)); > 11. The area of the triangle bound by the normal line, the vertical line through $Q$ and the $x$-axis +::: {#fig-parabola-tangent-normal-CMJ-area-triangle-normal-line-x-axis} ```{julia} #| echo: false make_plot() @@ -384,7 +424,8 @@ xlims!((-2, p₀ + 0.2)) plot!([p₀,q₀,q₀,p₀], [0,f(q₀),0,0]; fill=(:green, 0.25,0)) ``` - +Emphasis of triangle formed along normal line +::: ```{julia} #| code-fold: true #| code-summary: "Show the code" @@ -396,27 +437,27 @@ triangle = 1/2 * f(q) * (a - f(a)/(-1/fp(a)) - q); > 12. The area of the quadrilateral bound by the normal line, the tangent line, the vertical line through $Q$ and the $x$-axis +::: {#fig-parabola-tangent-normal-CMJ-quadrilateral-normal-line} ```{julia} #| echo: false make_plot() plot!([a₀,q₀,q₀,a₀-f(a₀)/fp(a₀),a₀], [f(a₀), f(q₀), 0, 0,f(a₀)], fill=(:green, 0.25, 0)) ``` - +Emphasis of quadrilateral bounded by normal line and tangent line +::: ```{julia} #| code-fold: true #| code-summary: "Show the code" -# @syms x[1:4], y[1:4] -# v1, v2, v3 = [[x[i]-x[1],y[i]-y[1], 0] for i in 2:4] -# area = 1//2 * last(cross(v3,v2) + cross(v2, v1)) # 1/2 area of parallelogram -# print(simplify(area)) -# (x₁ - x₂)*(y₁ - y₃)/2 - (x₁ - x₃)*(y₁ - y₂)/2 + (x₁ - x₃)*(y₁ - y₄)/2 - (x₁ - x₄)*(y₁ - y₃)/2 +# use shoelace formula +# (1/2) * (x₁⋅y₂-y₁⋅x₂ + x₂⋅y₃-y₂⋅x₃ + x₃⋅y₄-y₃⋅x₄ + x₄⋅y₁-y₄⋅x₁) tl₀ = a - f(a) / fp(a) -x₁,x₂,x₃,x₄ = (a,q,q,tl₀) +x₁, x₂, x₃, x₄ = (a, q, q, tl₀) y₁, y₂, y₃, y₄ = (f(a), f(q), 0, 0) -quadrilateral = (x₁ - x₂)*(y₁ - y₃)/2 - (x₁ - x₃)*(y₁ - y₂)/2 + (x₁ - x₃)*(y₁ - y₄)/2 - (x₁ - x₄)*(y₁ - y₃)/2; +quadrilateral = (1/2) * (x₁*y₂ - y₁*x₂ + x₂*y₃ - y₂*x₃ + x₃*y₄ - y₃*x₄ + x₄*y₁ - y₄*x₁); ``` + --- The answers appear here in sorted order, some given as approximate floating point values: diff --git a/quarto/integrals/volumes_slice.qmd b/quarto/integrals/volumes_slice.qmd index 1993aac..0eca5c4 100644 --- a/quarto/integrals/volumes_slice.qmd +++ b/quarto/integrals/volumes_slice.qmd @@ -8,8 +8,7 @@ This section uses these add-on packages: ```{julia} using CalculusWithJulia -using Plots -plotly() +using Plots; plotly() using QuadGK using Unitful, UnitfulUS using Roots @@ -75,8 +74,11 @@ Hey Michelin Man, how much does that costume weigh? # ImageFile(:integrals, imgfile, caption) nothing ``` +::: {#fig-michelin-man-as-volume-slice} +![](./figures/michelin-man.jpg) -![Hey Michelin Man, how much does that costume weigh?](./figures/michelin-man.jpg) +Hey Michelin Man, how much does that costume weigh? +::: An ad for a summer job says work as the Michelin Man! Sounds promising, but how much will that costume weigh? A very hot summer may make walking around in a heavy costume quite uncomfortable. @@ -84,10 +86,13 @@ An ad for a summer job says work as the Michelin Man! Sounds promising, but how A back-of-the envelope calculation would start by - * Mentally separating out each "tire" and lining them up one by one. - * Counting the number of "tires" (or rings), say $n$. - * Estimating the radius for each tire, say $r_i$ for $1 \leq i \leq n$. - * Estimating the height for each tire, say $h_i$ for $1 \leq i \leq n$ +* Mentally separating out each "tire" and lining them up one by one. + +* Counting the number of "tires" (or rings), say $n$. + +* Estimating the radius for each tire, say $r_i$ for $1 \leq i \leq n$. + +* Estimating the height for each tire, say $h_i$ for $1 \leq i \leq n$ Then the volume would be found by adding: @@ -105,12 +110,15 @@ Looking at the sum though, we see the makings of an approximate integral. If the In fact, we have in general: +::: {.definition title="Volume of a figure with a known cross section"} -> **Volume of a figure with a known cross section**: The volume of a solid with known cross-sectional area $A_{xc}(x)$ from $x=a$ to $x=b$ is given by -> -> $V = \int_a^b A_{xc}(x) dx.$ -> -> This assumes $A_{xc}(x)$ is integrable. +The volume of a solid with known, integrable, cross-sectional area $A_{xc}(x)$ from $x=a$ to $x=b$ is given by + +$$ +V = \int_a^b A_{xc}(x) dx. +$$ + +::: @@ -120,12 +128,12 @@ This formula is derived by approximating the volume by "slabs" with volume $A_{x ## Solids of revolution -We begin with some examples of a special class of solids - solids of revolution. These have an axis of symmetry from which the slabs are then just circular disks. +We begin with some examples of a special class of solids---solids of revolution. These have an axis of symmetry from which the slabs are then just circular disks. -Consider the volume contained in this glass, it will depend on the radius at different values of $x$: - +Consider the volume contained in the glass of @fig-wine-glass-rotation, the volume will depend on the radius at different values of $x$: +::: {#fig-wine-glass-rotation} ```{julia} #| hold: true #| echo: false @@ -141,12 +149,14 @@ between about $0$ and $6.2$cm. nothing ``` -![A wine glass oriented so that it is seen as generated by revolving a +![](./figures/integration-glass.jpg) + +A wine glass oriented so that it is seen as generated by revolving a curve about the $x$ axis. The radius of revolution varies as a function of $x$ between about $0$ and $6.2$cm. -](./figures/integration-glass.jpg) +::: -If $r(x)$ is the radius as a function of $x$, then the cross sectional area is $\pi r(x)^2$ so the volume is given by: +If $r(x)$ is the radius as a function of $x$, then the cross-sectional area is $\pi r(x)^2$ so the volume is given by: $$ @@ -251,7 +261,7 @@ nothing ``` -Illustration of a figure being rotated around the $x$-axis. The discs have approximate volume given by the area of the base times the height or $\pi r(x)^2 \Delta x$. (Figure ported from @Angenent.) +Illustration^[This illustration and others were directly inspired by @Angenent.] of a figure being rotated around the $x$-axis. The discs have approximate volume given by the area of the base times the height or $\pi r(x)^2 \Delta x$. ::: @@ -302,9 +312,9 @@ If you are poor with units, `Julia` can provide some help through the `Unitful` vol * u"inch"^3 |> us"floz" ``` -Before Solo "squared" the cup, the Solo cup had markings that - [some thought](http://www.snopes.com/food/prepare/solocups.asp) - indicated certain volume amounts. - +Before Solo "squared" the cup, the Solo cup had markings that--[some thought](http://www.snopes.com/food/prepare/solocups.asp)---indicated certain volume amounts. +::: {#fig-red-solo-cup-with-markings} ```{julia} #| hold: true #| echo: false @@ -314,7 +324,10 @@ caption = "Markings on the red Solo cup indicated various volumes" nothing ``` -![Markings on the red Solo cup indicated various volumes.](./figures/red-solo-cup.jpg) +![](./figures/red-solo-cup.jpg) + +Markings on the red Solo cup indicated various volumes +::: What is the height for $5$ ounces (for a glass of wine)? $12$ ounces (for a beer unit)? @@ -322,14 +335,14 @@ What is the height for $5$ ounces (for a glass of wine)? $12$ ounces (for a beer Here the volume is fixed, but the height is not. For $v$ ounces, we need to convert to cubic inches. The conversion is $1$ ounce is $231/128 \text{in}^3$. -So we need to solve $v \cdot (231/128) = \int_0^h\pi r(x)^2 dx$ for $h$ when $v=5$ and $v=12$. +So we need to solve $v \cdot (231/128) =\int_0^h\pi r(x)^2 dx$ for $h$ when $v=5$ and $v=12$. Let's express volume as a function of $h$: ```{julia} -Vol(h) = quadgk(x -> pi * rad(x)^2, 0, h)[1] +Vol(h) = first(quadgk(x -> pi * rad(x)^2, 0, h)) ``` Then to solve we have: @@ -395,12 +408,12 @@ frustum - cone * ( 3h0/h - 3(h0/h)^2 + (h0/h)^3) |> simplify ##### Example -[Gabriel's](http://tinyurl.com/8a6ygv) horn is a geometric figure of mathematics - but not the real world - which has infinite height, but not volume! The figure is found by rotating the curve $y=1/x$ around the $x$ axis from $1$ to $\infty$. If the volume formula holds, what is the volume of this "horn?" +[Gabriel's](https://en.wikipedia.org/wiki/Gabriel%27s_horn) horn is a geometric figure of mathematics---but not the real world---which has infinite height, but not volume! The figure is found by rotating the curve $y=1/x$ around the $x$ axis from $1$ to $\infty$. If the volume formula holds, what is the volume of this "horn?" ```{julia} radius(x) = 1/x -quadgk(x -> pi*radius(x)^2, 1, Inf)[1] +first(quadgk(x -> pi*radius(x)^2, 1, Inf)) ``` That is a value very reminiscent of $\pi$, which it is as $\int_1^\infty 1/x^2 dx = -1/x\big|_1^\infty=1$. @@ -408,7 +421,7 @@ That is a value very reminiscent of $\pi$, which it is as $\int_1^\infty 1/x^2 d :::{.callout-note} ## Note -The interest in this figure is that soon we will be able to show that it has **infinite** surface area, leading to the [paradox](http://tinyurl.com/osawwqm) that it seems possible to fill it with paint, but not paint the outside. +The interest in this figure is that soon we will be able to show that it has **infinite** surface area, leading to the paradox that it seems possible to fill it with paint, but not paint the outside. ::: @@ -479,17 +492,20 @@ Rather than use $\pi r(x)^2$ for a cross section, we would use $\pi (R(x)^2 - r( In general we call a shape like the tire a "washer" and use this formula for a washer's cross section $A_{xc}(x) = \pi(R(x)^2 - r(x)^2)$. -Then the volume for the solid of revolution whose cross sections are washers would be: +::: {.definition title="Volume by the washer method"} +The volume for the solid of revolution whose cross sections are washers with outer and inner radius given by $R(x)$ and $r(x)$ is: $$ V = \int_a^b \pi \cdot (R(x)^2 - r(x)^2) dx. $$ +::: + ::: {#fig-washer-illustration} ```{julia} #| echo: false -plt = let +let gr() # Follow lead of # https://github.com/SigurdAngenent/WisconsinCalculus/blob/master/figures/221/09surf_of_rotation2.py # plot surface of revolution around x axis between [0, 3] @@ -519,7 +535,7 @@ plt = let α = 1.0 line_style = (; line=(:black, 1)) - plot(; empty_style..., aspect_ratio=:equal) + plt = plot(; empty_style..., aspect_ratio=:equal) # by layering, we get x-axis as desired plot!(pline(viewp, [-1,0,0], [0,0,0]); line_style...) @@ -572,20 +588,12 @@ plt = let plot!(curve; line=(:black, 1)) end - current() - - + plotly() + plt end -plt ``` -```{julia} -#| echo: false -plotly() -nothing -``` - -Modification of earlier figure to show washer method. The interior volume would be given by $\int_a^b \pi r(x)^2 dx$, the entire volume by $\int_a^b \pi R(x)^2 dx$. The difference then is the volume computed by the washer method. +Modification of @fig-solid-of-revolution to show the washer method. The interior volume would be given by $\int_a^b \pi r(x)^2 dx$, the entire volume by $\int_a^b \pi R(x)^2 dx$. The difference then is the volume computed by the washer method. ::: @@ -629,10 +637,10 @@ vol, _ = quadgk(x -> d(x)^2, 0, h) vol / 231 * 128 ``` -This shape would have more volume - the cross sections are bigger. Presumably the dimensions have changed. Without going out and buying a cup, let's assume the cross-sectional diameter remained the same, not the diameter. This means the largest dimension is the same. The cross section diameter is $\sqrt{2}$ larger. What would this do to the area? +This shape would have more volume---the cross sections are bigger. Presumably the dimensions have changed. Without going out and buying a cup, let's assume the cross-sectional diameter remained the same, not the diameter. This means the largest dimension is the same. The cross-sectional diameter is $\sqrt{2}$ larger. What would this do to the area? -We could do this two ways: divide $d_0$ and $d_1$ by $\sqrt{2}$ and recompute. However, each cross section of this narrower cup would simply be $\sqrt{2}^2$ smaller, so the total volume would change by $2$, or be 13 ounces. We have $26.04$ is too big, and $13.02$ is too small, so some other overall dimensions are used. +We could do this two ways: divide $d_0$ and $d_1$ by $\sqrt{2}$ and recompute. However, each cross section of this narrower cup would simply be $\sqrt{2}^2$ smaller, so the total volume would change by $2$, or be $13$ ounces. We have $26.04$ is too big, and $13.02$ is too small, so some other overall dimensions are used. ##### Example @@ -645,7 +653,7 @@ For a general cone, we use this [definition](http://en.wikipedia.org/wiki/Cone): -Let $h$ be the distance from the apex to the base. Consider cones with the property that all planes parallel to the base intersect the cone with the same shape, though perhaps a different scale. This figure shows an example, with the rays coming from the apex defining the volume. +Let $h$ be the distance from the apex to the base. Consider cones with the property that all planes parallel to the base intersect the cone with the same shape, though perhaps a different scale. @fig-generic-cone shows an example, with the rays coming from the apex defining the volume. ::: {#fig-generic-cone} ```{julia} @@ -736,18 +744,21 @@ $$ V = \int_0^h A_{xc}(u) du. $$ -The cross sectional area $A_{xc}(u)$ satisfies a formula in terms of $A_{xc}(0)$, the area of the base: +The cross-sectional area $A_{xc}(u)$ satisfies a formula in terms of $A_{xc}(0)$, the area of the base: $$ -A_{xc}(u) = A_{xc}(0) \cdot (1 - \frac{u}{h})^2 +A_{xc}(u) = A_{xc}(0) \cdot \left(1 - \frac{u}{h}\right)^2 $$ So the integral becomes: $$ -V = \int_0^h A_{xc}(u) du = A_{xc}(0) \int_0^h (1 - \frac{u}{h})^2 du = A_{xc}(0) \int_0^1 v^2 h dv = A_{xc}(0) \frac{h}{3}. +\begin{align*} +V &= \int_0^h A_{xc}(u) du = A_{xc}(0) \int_0^h \left(1 - \frac{u}{h}\right)^2 du\\ +&= A_{xc}(0) \int_0^1 v^2 h dv = A_{xc}(0) \frac{h}{3}. +\end{align*} $$ This gives a general formula for the volume of such cones. @@ -814,9 +825,8 @@ end plt ``` -This figure shows the volume of a figure being comprised of slices. A discrete approximation would be found by estimating the volume of each slice by the cross sectional area times a small $\Delta h$. This leads to a formula -$V = \int_a^b A(h)dh$, where $A$ computes the cross sectional area. -(This figure was ported from @Angenent.) +This figure shows the volume of a figure being comprised of slices. A discrete approximation would be found by estimating the volume of each slice by the cross-sectional area times a small $\Delta h$. This leads to a formula +$V = \int_a^b A(h)dh$, where $A$ computes the cross-sectional area. ::: ```{julia} @@ -828,7 +838,9 @@ nothing ### Cavalieri's method -[Cavalieri's](http://tinyurl.com/oda9xd9) Principle is "Suppose two regions in three-space (solids) are included between two parallel planes. If every plane parallel to these two planes intersects both regions in cross-sections of equal area, then the two regions have equal volumes." (Wikipedia). +[Cavalieri's](https://en.wikipedia.org/wiki/Cavalieri%27s_principle) Principle is: + +> Suppose two regions in three-space (solids) are included between two parallel planes. If every plane parallel to these two planes intersects both regions in cross-sections of equal area, then the two regions have equal volumes." (Wikipedia). ::: {#fig-Cavalieris-first} @@ -868,19 +880,19 @@ plotly() nothing ``` -Illustration of Cavalieri's first principle. The discs from the left are moved around to form the left volume, but as the volumes of each cross-sectional disc remains the same, the two volumes are equally approximated. (This figure ported from @Angenent.) - +Illustration of Cavalieri's first principle. The discs from the left are moved around to form the left volume, but as the volumes of each cross-sectional disc remains the same, the two volumes are equally approximated. ::: With the formula for the volume of solids based on cross sections, this is a trivial observation, as the functions giving the cross-sectional area are identical. Still, it can be surprising. -Consider a sphere with an interior cylinder bored out of it. (The [Napkin](http://tinyurl.com/o237v83) ring problem.) The bore has height $h$ - for larger radius spheres this means very wide bores. +Consider a sphere with an interior cylinder bored out of it. (The [Napkin](https://en.wikipedia.org/wiki/Cavalieri%27s_principle#The_napkin_ring_problem) ring problem.) The bore has height $h$---for larger radius spheres this means very wide bores. ::: {#fig-napkin-ring-1} ```{julia} #| echo: false -plt = let +let + gr() # Follow lead of # https://github.com/SigurdAngenent/WisconsinCalculus/blob/master/figures/221/09surf_of_rotation2.py # plot surface of revolution around x axis between [0, 3] # best if r(t) decreases @@ -937,7 +949,7 @@ plt = let α = 1.0 line_style = (; line=(:black, 1)) - plot(; empty_style..., aspect_ratio=:equal) + plt = plot(; empty_style..., aspect_ratio=:equal) # washer t0 = sqrt(3/4) @@ -997,9 +1009,9 @@ plt = let end =# - current() + plotly() + plt end -plt ``` Figure showing sphere with interior cylinder bored out. @@ -1040,7 +1052,7 @@ plt = let x₀ = sqrt(R^2 - (h/2)^2) annotate!( [ - (x₀/2, 0, text(L"\sqrt{R^2- (\frac{h}{2})^2}",10, :top)), + (x₀/2, 0, text(L"\sqrt{R^2- (\frac{h}{2})^2}", :top)), (x₀, h/4, text(L"\frac{h}{2}",:right)), (R/2*cos(θ),R/2*sin(θ), text(L"R", :bottom; rotation=rad2deg(θ))) ]) @@ -1067,13 +1079,15 @@ The small orange line is rotated, so using the washer method we get the cross se The outer radii has points $(x,y)$ satisfying $x^2 + y^2 = R^2$, so is $\sqrt{R^2 - y^2}$. The inner radii has a constant value, and as indicated in the figure, is $\sqrt{R^2 - (h/2)^2}$, by the Pythagorean theorem. -Thus the cross sectional area is +Thus the cross-sectional area is $$ -\pi( (\sqrt{R^2 - y^2})^2 - (\sqrt{R^2 - (h/2)^2})^2 ) -= \pi ((R^2 - y^2) - (R^2 - (h/2)^2)) -= \pi ((\frac{h}{2})^2 - y^2) +\begin{align*} +\pi\left( (\sqrt{R^2 - y^2})^2 - (\sqrt{R^2 - (h/2)^2})^2 \right) +&= \pi \left((R^2 - y^2) - (R^2 - (h/2)^2)\right)\\ +&= \pi \left((\frac{h}{2})^2 - y^2\right) +\end{align*} $$ As this does not depend on $R$, and the limits of integration would always be $-h/2$ to $h/2$ by Cavalieri's principle, the volume of the solid will be independent of $R$ too. @@ -1085,7 +1099,9 @@ To actually compute this volume, we take $R=h/2$, so that the bore hole is just ## The second theorem of Pappus -The second theorem of [Pappus](http://tinyurl.com/l43vw4) says that if a plane figure $F$ is rotated around an axis to form a solid of revolution, the total volume can be written as $2\pi r A(F)$, where $r$ is the distance the centroid is from the axis of revolution, and $A(F)$ is the area of the plane figure. In short, the distance traveled by the centroid times the area. +The second theorem of [Pappus](https://en.wikipedia.org/wiki/Pappus%27s_centroid_theorem) says + +> If a plane figure $F$ is rotated around an axis to form a solid of revolution, the total volume can be written as $2\pi r A(F)$, where $r$ is the distance the centroid is from the axis of revolution, and $A(F)$ is the area of the plane figure. In short, the distance traveled by the centroid times the area. (Wikipedia) This can make some computations trivial. For example, we can make a torus (or donut) by rotating the circle $(x-2)^2 + y^2 = 1$ about the $y$ axis. As the centroid is clearly $(2, 0)$, with $r=2$ in the above formula, and the area of the circle is $\pi 1^2$, the volume of the donut is $2\pi(2)(\pi) = 4\pi^2$. @@ -1103,9 +1119,9 @@ Above, we found the volume of a cone, as it is a solid of revolution, through th ###### Question -Consider this big Solo cup: - +@fig-big-solo-cup shows a big Solo cup. +::: {#fig-big-solo-cup} ```{julia} #| hold: true #| echo: false @@ -1115,16 +1131,19 @@ caption = " Big solo cup. " nothing ``` -![Big solo cup.](./figures/big-solo-cup.jpg) +![](./figures/big-solo-cup.jpg) + +Big solo cup +::: It has approximate dimensions: smaller radius 5 feet, upper radius 8 feet and height 15 feet. How many gallons is it? At $8$ pounds a gallon this would be pretty heavy! Two facts are useful: +* a cubic foot is 7.48052 gallons - * a cubic foot is 7.48052 gallons - * the radius as a function of height is $r(h) = 5 + (3/15)\cdot h$ +* the radius as a function of height is $r(h) = 5 + (3/15)\cdot h$ ```{julia} @@ -1143,9 +1162,9 @@ numericq(val, 1e1) In *Glass Shape Influences Consumption Rate* for Alcoholic [Beverages](http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0043007) the authors demonstrate that the shape of the glass can have an effect on the rate of consumption, presumably people drink faster when they aren't sure how much they have left. In particular, they comment that people have difficulty judging the half-finished-by-volume mark. -This figure shows some of the wide variety of beer-serving glasses: - +@fig-beer-glasses shows some of a wide variety of beer-serving glasses: +::: {#fig-beer-glasses} ```{julia} #| hold: true #| echo: false @@ -1155,7 +1174,10 @@ caption = "A variety of different serving glasses for beer." nothing ``` -![A variety of different serving glasses for beer.](./figures/beer_glasses.jpg) +![](./figures/beer_glasses.jpg) + +A variety of different serving glasses for beer +::: We work with metric units, as there is a natural relation between volume in cm$^3$ and liquid measure ($1$ liter = $1000$ cm$^3$, so a $16$-oz pint glass is roughly $450$ cm$^3$.) @@ -1180,11 +1202,11 @@ The following functions find the volume as a function of height, $h$: ```{julia} r1(h) = 3 + h/5 s1(h) = 2 + log(1 + h) -r_vol(h) = quadgk(x -> pi*r1(x)^2, 0, h)[1] -s_vol(h) = quadgk(x -> pi*s1(x)^2, 0, h)[1] +r_vol(h) = first(quadgk(x -> pi*r1(x)^2, 0, h)) +s_vol(h) = first(quadgk(x -> pi*s1(x)^2, 0, h)) ``` - * For the straight-sided glass find $h$ so that the volume is $450$. +* For the straight-sided glass find $h$ so that the volume is $450$. ```{julia} @@ -1193,7 +1215,7 @@ h450 = find_zero(h -> r_vol(h) - 450, 10) numericq(h450) ``` - * For the straight-sided glass find $h$ so that the volume is $225$ (half full). +* For the straight-sided glass find $h$ so that the volume is $225$ (half full). ```{julia} @@ -1202,7 +1224,7 @@ h225 = find_zero(h -> r_vol(h) - 225, 10) numericq(h225) ``` - * For the straight-sided glass, what is the percentage of the total height when the glass is half full. (For a cylinder it would just be 50.) +* For the straight-sided glass, what is the percentage of the total height when the glass is half full. (For a cylinder it would just be 50.) ```{julia} @@ -1210,7 +1232,7 @@ numericq(h225) numericq(h225/h450 * 100, 2, units="percent") ``` - * People often confuse the half-way by height amount for the half way by volume, as it is for the cylinder. Take the height for the straight-sided glass filled with $450$ mm, divide it by $2$, then compute the percentage of volume at the half way height to the original. +* People often confuse the half-way by height amount for the half way by volume, as it is for the cylinder. Take the height for the straight-sided glass filled with $450$ mm, divide it by $2$, then compute the percentage of volume at the half way height to the original. ```{julia} @@ -1221,7 +1243,7 @@ numericq(r_vol(h450/2)/450*100, 2, units="percent") --- - * For the curved-sided glass find $h$ so that the volume is $450$. +* For the curved-sided glass find $h$ so that the volume is $450$. ```{julia} @@ -1230,7 +1252,7 @@ h_450 = find_zero(h -> s_vol(h) - 450, 10) numericq(h_450) ``` - * For the curved-sided glass find $h$ so that the volume is $225$ (half full). +* For the curved-sided glass find $h$ so that the volume is $225$ (half full). ```{julia} @@ -1239,7 +1261,7 @@ h_225 = find_zero(h -> s_vol(h) - 225, 10) numericq(h_225) ``` - * For the curved-sided glass, what is the percentage of the total height when the glass is half full. (For a cylinder it would just be 50.) +* For the curved-sided glass, what is the percentage of the total height when the glass is half full. (For a cylinder it would just be 50.) ```{julia} @@ -1247,7 +1269,7 @@ numericq(h_225) numericq(h_225/h_450 * 100, 2, units="percent") ``` - * People often confuse the half-way by height amount for the half way by volume, as it is for the cylinder. Take the height for the curved-sided glass filled with $450$ mm, divide it by $2$, then compute the percentage of volume at the half way height to the original. +* People often confuse the half-way by height amount for the half way by volume, as it is for the cylinder. Take the height for the curved-sided glass filled with $450$ mm, divide it by $2$, then compute the percentage of volume at the half way height to the original. ```{julia} @@ -1343,13 +1365,6 @@ numericq(val) The region enclosed by the graphs of $y=x^3 - 1$ and $y=x-1$ are rotated around the $y$ axis. What is the volume of the solid? -```{julia} -#| hold: true -@syms x -plot(x^3 - 1, 0, 1, legend=false) -plot!(x-1) -``` - ```{julia} #| hold: true #| echo: false @@ -1363,11 +1378,13 @@ numericq(val) ###### Question -Rotate the region bounded by $y=e^x$, the line $x=\log(2)$ and the first quadrant ($x,y \geq 0$) about the line $x=\log(2)$. +Rotate the region bounded by $y=e^x$, the line $x=\ln(2)$ and the first quadrant ($x,y \geq 0$) about the line $x=\log(2)$, as illustrated in @fig-rotate-e-to-x-around-log-2 +::: {#fig-rotate-e-to-x-around-log-2} ```{julia} #| echo: false let + gr() p = plot(exp, -0.1, log(2.2); legend=false, ylim = (-0.3, 2.3)) hline!(p, [0], color=:black) vline!(p, [0], color=:black) @@ -1381,12 +1398,17 @@ let bs(t) = (exp(log(2)) - 4b) + b*sin(t) plot!(p, as.(ts), bs.(ts), linewidth=1, color=:black, arrow=(:closed, 2.0)) + plotly() p end ``` -(Be careful, the radius in the formula $V=\int_a^b \pi r(u)^2 du$ is from the line $x=\log(2)$, further, the constraint $x \geq 0$ needs attention.) +Rotate a region around the line $x=\ln(2)$ +::: + + +(Be careful, the radius in the formula $V=\int_a^b \pi r(u)^2 du$ is from the line $x=\ln(2)$, further, the constraint $x \geq 0$ needs attention.) ```{julia} @@ -1428,11 +1450,12 @@ You can integrate in the length along the line $y=x$ ($u$ from $0$ to $\sqrt{2}$ theta = pi/4 ## we write y=x as y = x * tan(pi/4) for more generality, as this allows other slants. f(x) = x^2 -𝒙(u) = find_zero(x -> u*sin(theta) - 1/tan(theta) * (x - u*cos(theta)) - f(x), (u*cos(theta), 1)) -𝒓(u) = sqrt((u*cos(theta) - 𝒙(u))^2 + (u*sin(theta) - f(𝒙(u)))^2) +𝑥(u) = find_zero(x -> u*sin(theta) - 1/tan(theta) * (x - u*cos(theta)) - f(x), (u*cos(theta), 1)) +𝑟(u) = sqrt((u*cos(theta) - 𝑥(u))^2 + (u*sin(theta) - f(𝑥(u)))^2) ``` -(Though in this case you can also find `r(u)` using the quadratic formula.) +(Though in this case you can also find `𝑟(u)` using the quadratic formula.) + With this, find the volume. @@ -1442,7 +1465,7 @@ With this, find the volume. #| hold: true #| echo: false a, b = 0, sqrt(2) -val, _ = quadgk(u -> pi*𝒓(u)^2, a, b) +val, _ = quadgk(u -> pi*𝑟(u)^2, a, b) numericq(val) ``` @@ -1455,10 +1478,13 @@ Repeat (find the volume) only this time with the function $f(x) = x^{20}$. ```{julia} #| hold: true #| echo: false -a, b = 0, sqrt(2) -f(x) = x^20 -xval(u) = find_zero(x -> u*sin(theta) - 1/tan(theta) * (x - u*cos(theta)) - f(x), (0,sqrt(2))) -rad(u) = sqrt((u*cos(theta) - xval(u))^2 + (u*sin(theta) - f(xval(u)))^2) -val, _ = quadgk(u -> pi*rad(u)^2, a, b) -numericq(val) +let + theta = pi/4 + a, b = 0, sqrt(2) + f(x) = x^20 + x(u) = find_zero(x -> u*sin(theta) - 1/tan(theta) * (x - u*cos(theta)) - f(x), (0,sqrt(2))) + r(u) = sqrt((u*cos(theta) - x(u))^2 + (u*sin(theta) - f(x(u)))^2) + val, _ = quadgk(u -> pi*r(u)^2, a, b) + numericq(val) +end ``` diff --git a/quarto/limits.qmd b/quarto/limits.qmd index af555fe..a737de3 100644 --- a/quarto/limits.qmd +++ b/quarto/limits.qmd @@ -1,6 +1,7 @@ -# Limits +# Limits and continuity The concept of a limit is behind most all the concepts in Calculus. A limit in mathematics is the value some function or sequence approaches as an input approaches some value. It will be seen that there are many -different ways to define "approaches." +different ways to define "approaches". Limits are used to formally define +the important notion of continuity. diff --git a/quarto/limits/Project.toml b/quarto/limits/Project.toml index 86b9abe..db54f93 100644 --- a/quarto/limits/Project.toml +++ b/quarto/limits/Project.toml @@ -1,9 +1,12 @@ [deps] +AbbreviatedStackTraces = "ac637c84-cc71-43bf-9c33-c1b4316be3d4" CalculusWithJulia = "a2e0e22d-7d4c-5312-9169-8b992201a882" DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" IJulia = "7073ff75-c697-5162-941a-fcdaad2a7d2a" LaTeXStrings = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f" Mustache = "ffc61752-8dc7-55ee-8c37-f3e9cdd09e70" +PlotlyBase = "a03496cd-edff-5a9b-9e67-9cda94a718b5" +PlotlyKaleido = "f2990250-8cf9-495f-b13a-cce12b45703c" Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80" QuizQuestions = "612c44de-1021-4a21-84fb-7261cf5eb2d4" Richardson = "708f8203-808e-40c0-ba2d-98a6953ed40d" diff --git a/quarto/limits/continuity.qmd b/quarto/limits/continuity.qmd index 26822f2..418f62a 100644 --- a/quarto/limits/continuity.qmd +++ b/quarto/limits/continuity.qmd @@ -16,7 +16,11 @@ using SymPy --- -![A Möbius strip by Koo Jeong A](figures/korean-mobius.jpg){width=40%} +::: {#fig-mobius-strip-biennale} +![](figures/korean-mobius.jpg){width=40%} + +A Möbius strip by Koo Jeong A +::: The definition Google finds for *continuous* is *forming an unbroken whole; without interruption*. @@ -25,30 +29,29 @@ The definition Google finds for *continuous* is *forming an unbroken whole; with The concept in calculus, as transferred to functions, is similar. Roughly speaking, a continuous function is one whose graph could be drawn without having to lift (or interrupt) the pencil drawing it. -Consider these two graphs: - +Consider the two graphs show in @fig-two-plots-one-continuous-one-not-almost-always-equal. +::: {#fig-two-plots-one-continuous-one-not-almost-always-equal} ```{julia} #| hold: true #| echo: false -plt = plot([-1,0], [-1,-1], color=:black, legend=false, linewidth=5) -plot!(plt, [0, 1], [ 1, 1], color=:black, linewidth=5) -plt -``` - -and - - -```{julia} +plt1 = plot([-1,0], [-1,-1], color=:black, legend=false, linewidth=5) +plot!(plt1, [0, 1], [ 1, 1], color=:black, linewidth=5) +plt1 #| hold: true #| echo: false -plot([-1,-.1, .1, 1], [-1,-1, 1, 1], color=:black, legend=false, linewidth=5) +plt2 = plot([-1,-.1, .1, 1], [-1,-1, 1, 1], color=:black, legend=false, linewidth=5) + +plot(plt1, plt2) ``` -Though similar at some level - they agree at nearly every value of $x$ - the first has a "jump" from $-1$ to $1$ instead of the transition in the second one. The first is not continuous at $0$ - a break is needed to draw it - where as the second is continuous. +Plot of two functions that are equal at most of their points +::: + +Though similar at some level---they agree at nearly every value of $x$--- the first has a "jump" from $-1$ to $1$ instead of the transition in the second one. The first is not continuous at $0$---a break is needed to draw it---where as the second is continuous. -A formal definition of continuity was a bit harder to come about. At [first](http://en.wikipedia.org/wiki/Intermediate_value_theorem) the concept was that for any $y$ between any two values in the range for $f(x)$, the function should take on the value $y$ for some $x$. Clearly this could distinguish the two graphs above, as one takes no values in $(-1,1)$, whereas the other - the continuous one - takes on all values in that range. +A formal definition of continuity was historically a bit harder to develop. At [first](http://en.wikipedia.org/wiki/Intermediate_value_theorem) the concept was that for any $y$ between any two values in the range for $f(x)$, the function should take on the value $y$ for some $x$. Clearly this could distinguish the two graphs above, as one takes no values in $(-1,1)$, whereas the other---the continuous one---takes on all values in that range. However, [Cauchy](http://en.wikipedia.org/wiki/Cours_d%27Analyse) defined continuity by $f(x + \alpha) - f(x)$ being small whenever $\alpha$ was small. This basically rules out "jumps" and proves more useful as a tool to describe continuity. @@ -56,26 +59,25 @@ However, [Cauchy](http://en.wikipedia.org/wiki/Cours_d%27Analyse) defined contin The [modern](http://en.wikipedia.org/wiki/Continuous_function#History) definition simply pushes the details to the definition of the limit: -::: {.callout-note icon=false} -## Definition of continuity at a point +::: {.definition title="Definition of continuity at a point"} A function $f(x)$ is continuous at $x=c$ if $\lim_{x \rightarrow c}f(x) = f(c)$. ::: -The definition says three things +The definition says three things: +* the limit exists at $c$; - * The limit exists at $c$. - * The function is defined at $c$ ($c$ is in the domain). - * The value of the limit is the same as $f(c)$. +* the function is defined at $c$ ($c$ is in the domain of $f$); and + +* the value of the limit is the same as $f(c)$. The definition speaks to continuity at a point, we can extend it to continuity over an interval $(a,b)$ by saying: -::: {.callout-note icon=false} -## Definition of continuity over an open interval +::: {.definition title="Definition of continuity over an open interval"} A function $f(x)$ is continuous over $(a,b)$ if at each point $c$ with $a < c < b$, $f(x)$ is continuous at $c$. @@ -84,7 +86,13 @@ A function $f(x)$ is continuous over $(a,b)$ if at each point $c$ with $a < c < Finally, as with limits, it can be convenient to speak of *right* continuity and *left* continuity at a point, where the limit in the definition is replaced by a right or left limit, as appropriate. -In particular, a function is *continuous* over $[a,b]$ if it is continuous on $(a,b)$, left continuous at $b$ and right continuous at $a$. +In particular: + +::: {.definition title="Definition of continuity over a close interval"} + +A function is *continuous* over $[a,b]$ if it is continuous on $(a,b)$, left continuous at $b$ and right continuous at $a$. + +::: :::{.callout-warning} @@ -102,7 +110,7 @@ Most familiar functions are continuous everywhere. * For example, a monomial function $f(x) = ax^n$ for non-negative, integer $n$ will be continuous. This is because the limit exists everywhere, the domain of $f$ is all $x$ and there are no jumps. * Similarly, the building-block trigonometric functions $\sin(x)$, $\cos(x)$ are continuous everywhere. * So are the exponential functions $f(x) = a^x, a > 0$. - * The hyperbolic sine ($(e^x - e^{-x})/2$) and cosine ($(e^x + e^{-x})/2$) are, as $e^x$ is. + * The hyperbolic sine ($(e^x - e^{-x})/2$) and cosine ($(e^x + e^{-x})/2$) are continuous everywhere, as $e^x$ is. * The hyperbolic tangent is, as $\cosh(x) > 0$ for all $x$. @@ -112,7 +120,7 @@ Some familiar functions are *mostly* continuous but not everywhere. * For example, $f(x) = \sqrt{x}$ is continuous on $(0,\infty)$ and right continuous at $0$, but it is not defined for negative $x$, so can't possibly be continuous there. * Similarly, $f(x) = \log(x)$ is continuous on $(0,\infty)$, but it is not defined at $x=0$, so is not right continuous at $0$. * The tangent function $\tan(x) = \sin(x)/\cos(x)$ is continuous everywhere *except* the points $x$ with $\cos(x) = 0$ ($\pi/2 + k\pi, k$ an integer). - * The hyperbolic co-tangent is not continuous at $x=0$ – when $\sinh$ is $0$, + * The hyperbolic co-tangent is not continuous at $x=0$---when $\sinh$ is $0$, * The semicircle $f(x) = \sqrt{1 - x^2}$ is *continuous* on $(-1, 1)$. It is not continuous at $-1$ and $1$, though it is right continuous at $-1$ and left continuous at $1$. (It is continuous on $[-1,1]$.) @@ -122,10 +130,13 @@ Some familiar functions are *mostly* continuous but not everywhere. There are various reasons why a function may not be continuous. - * The function $f(x) = \sin(x)/x$ has a limit at $0$ but is not defined at $0$, so is not continuous at $0$. The function can be redefined to make it continuous. - * The function $f(x) = 1/x$ is continuous everywhere *except* $x=0$ where *no* limit exists. - * A rational function $f(x) = p(x)/q(x)$ will be continuous everywhere except where $q(x)=0$. (The function $f$ may still have a limit where $q$ is $0$, should factors cancel, but $f$ won't be defined at such values.) - * The function +* The function $f(x) = \sin(x)/x$ has a limit at $0$ but is not defined at $0$, so is not continuous at $0$. The function can be redefined to make it continuous. + +* The function $f(x) = 1/x$ is continuous everywhere *except* $x=0$ where *no* limit exists. + +* A rational function $f(x) = p(x)/q(x)$ will be continuous everywhere except where $q(x)=0$. (The function $f$ may still have a limit where $q$ is $0$, should factors cancel, but $f$ won't be defined at such values.) + +* The function $$ @@ -139,9 +150,10 @@ $$ is implemented by `Julia`'s `sign` function. It has a value at $0$, but no limit at $0$, so is not continuous at $0$. Furthermore, the left and right limits exist at $0$ but are not equal to $f(0)$ so the function is not left or right continuous at $0$. It is continuous everywhere except at $x=0$. - * Similarly, the function defined by this graph +* Similarly, the function defined by the graph in @fig-line-with-removable-discontinity-at-0 is not continuous at $x=0$. It has a limit of $0$ at $0$, a function value $f(0) =1/2$, but the limit and the function value are not equal. +::: {#fig-line-with-removable-discontinity-at-0} ```{julia} #| hold: true #| echo: false @@ -153,11 +165,13 @@ C = Shape(0.02 * sin.(ts), 0.03 * cos.(ts)) plot!(C, fill=(:white,1), line=(:black, 1)) ``` -is not continuous at $x=0$. It has a limit of $0$ at $0$, a function value $f(0) =1/2$, but the limit and the function value are not equal. +Function with a removable discontinuity +::: - * The `floor` function, which rounds down to the nearest integer, is also not continuous at the integers, but is right continuous at the integers, as, for example, $\lim_{x \rightarrow 0+} f(x) = f(0)$. This graph emphasizes the right continuity by placing a filled marker for the value of the function when there is a jump and an open marker where the function is not that value. +* The `floor` function, which rounds down to the nearest integer, is also not continuous at the integers, but is right continuous at the integers, as, for example, $\lim_{x \rightarrow 0+} f(x) = f(0)$. The graph in @fig-floor-function-with-left-right-limits-indicated emphasizes the right continuity by placing a filled marker for the value of the function when there is a jump and an open marker where the function is not that value. +::: {#fig-floor-function-with-left-right-limits-indicated} ```{julia} #| echo: false plt = let @@ -200,9 +214,13 @@ plt plotly() nothing ``` +The `floor` function +::: - * The function $f(x) = 1/x^2$ is not continuous at $x=0$: $f(x)$ is not defined at $x=0$ and $f(x)$ has no limit at $x=0$ (in the usual sense). - * On the Wikipedia page for [continuity](https://en.wikipedia.org/wiki/Continuous_function) the example of Dirichlet's function is given: +* The function $f(x) = 1/x^2$ is not continuous at $x=0$: $f(x)$ is not defined at $x=0$ and $f(x)$ has no limit at $x= +0$ (in the usual sense). + +* On the Wikipedia page for [continuity](https://en.wikipedia.org/wiki/Continuous_function) the example of Dirichlet's function is given: $$ @@ -232,7 +250,7 @@ $$ What value of $c$ will make $f(x)$ a continuous function? -We note that for $x < 0$ and for $x > 0$ the function is defined by a simple polynomial, so is continuous. At $x=0$ to be continuous we need a limit to exists and be equal to $f(0)$, which is $c$. A limit exists if the left and right limits are equal. This means we need to solve for $c$ to make the left and right limits equal. We do this next with a bit of overkill in this case: +We note that for $x < 0$ and for $x > 0$ the function is defined by a simple polynomial, so is continuous. At $x=0$ to be continuous we need a limit to exist *and* be equal to $f(0)$, which is $c$. A limit exists if the left and right limits are equal. This means we need to solve for $c$ to make the left and right limits equal. We do this next (with a bit of overkill in this case): ```{julia} @@ -260,15 +278,18 @@ solve(ex1(x=>0) ~ ex2(x=>0), c) ##### Example -Identifying from its graph that a function is discontinuous or not can be complicated by the graphing algorithm which simply connects adjacent points with a line segment allowing the eye to fill in the dot-to-dot graphic as a curve. The default plot of the `floor` function shows a potential issue: +Identifying from its graph that a function is discontinuous or not can be complicated by the graphing algorithm which simply connects adjacent points with a line segment allowing the eye to fill in the dot-to-dot graphic as a curve. The default plot in @fig-plot-of-floor-function-minus5-halves-to-5-halves-shows-artifacts-of-plotting of the `floor` function shows a potential issue. +::: {#fig-plot-of-floor-function-minus5-halves-to-5-halves-shows-artifacts-of-plotting} ```{julia} -plot(floor, -5/2, 5/2; label=false) +plot(floor, -5/2, 5/2; legend=false) ``` +The `floor` function plotted without taking care of the discontinuties may appear "continuous" +::: -The "risers" on the steps are an artifact of the basic dot-to-dot algorithm, which assumes continuity between adjacent points (we were more careful in our earlier plot of this function). +The "risers" on the steps are an artifact of the basic dot-to-dot algorithm, which assumes continuity between adjacent points (we were more careful in our earlier plot of this function in @fig-floor-function-with-left-right-limits-indicated). -The following simple function just plots a bunch of points, leaving the eye to fill in the line, though so many points are chosen this doesn't require much effort for simple cases. This function also plots a point on the $x$- and $y$-axes (emphasized by the argument `framestyle=:origin`) for each point graphed to emphasize the range of $y$ values for the specified $x$ values. +The following simple function just plots a bunch of points, leaving the eye to fill in the line, though so many points are chosen this doesn't require much effort for simple cases. This function also plots a point on the $x$- and $y$-axes (emphasized by the argument `framestyle=:origin`) for each point graphed to emphasize the range of $y$ values for the specified $x$ values. @fig-pixel-plot-function-kinda-silly-but-just-points-loads shows the `floor` function plotted this way. ```{julia} function pixel_plot(f, a, b; kwargs...) @@ -280,25 +301,33 @@ function pixel_plot(f, a, b; kwargs...) p = plot(;framestyle=:origin, legend=false, kwargs...) scatter!(p, xs, ys; marker=(:square, :black, 1)) # f(x) - scatter!(p, xs, zs; marker=(:square, :blue, 2, 0.03)) # domain, [a,b] - scatter!(p, zs, ys; marker=(:square, :red, 3, 0.25)) # range + scatter!(p, xs, zs; marker=(:square, :blue, 2, 0.03)) # domain, [a,b] + scatter!(p, zs, ys; marker=(:square, :red, 3, 0.25)) # range p end - -pixel_plot(floor, -5/2, 5/2) ``` -The broken up range suggests a fundamentally discontinuous function. In the next section we will see this differently---that a continuous function will have an unbroken range when restricted to some interval $[a,b]$. +::: {#fig-pixel-plot-function-kinda-silly-but-just-points-loads} +```{julia} +pixel_plot(floor, -5/2, 5/2) +``` +Plot of `floor` function over $[-5/2, 5/2]$ generated by plotting many points rather than using the dot-to-dot method +::: -For one more example, here we see the difference between `sin` and `sign`, as functions: +The broken up range suggests a fundamentally discontinuous function. In the next section we will state this differently---that a continuous function *will* have an unbroken range when restricted to some interval $[a,b]$. +For one more example, @fig-pixel-plot-sin-sign-big-differences shows the difference between `sin` and `sign`, as functions: + +::: {#fig-pixel-plot-sin-sign-big-differences} ```{julia} p1 = pixel_plot(sin, -pi, pi; title="sin") p2 = pixel_plot(sign, -pi, pi; title="sign") plot(p1, p2) ``` +Plot of both `sin` and `sign` functions +::: The continuous `sin` function has an unbroken range, $[-1,1]$; the discountinous `sign` function has a broken range consisting of ${-1, 0, 1}$. ## Rules for continuity @@ -306,27 +335,33 @@ The continuous `sin` function has an unbroken range, $[-1,1]$; the discountinous As we've seen, functions can be combined in several ways. How do these relate with continuity? - +::: {.relationship title="Rules of continuity"} Suppose $f(x)$ and $g(x)$ are both continuous on $I$. Then: +* The linear combination $h(x) = a f(x) + b g(x)$ is continuous on $I$ for any real numbers $a$ and $b$; - * The linear combination $h(x) = a f(x) + b g(x)$ is continuous on $I$ for any real numbers $a$ and $b$; - * The product $h(x) = f(x) \cdot g(x)$ is continuous on $I$; and - * The quotient $h(x) = f(x) / g(x)$ is continuous at all points $c$ in $I$ **where** $g(c) \neq 0$. - * The composition $h(x) = f(g(x))$ is continuous at $x=c$ *if* $g(x)$ is continuous at $c$ *and* $f(x)$ is continuous at $g(c)$. +* The product $h(x) = f(x) \cdot g(x)$ is continuous on $I$; and +* The quotient $h(x) = f(x) / g(x)$ is continuous at all points $c$ in $I$ **where** $g(c) \neq 0$. -So, continuity is preserved for all of the basic operations except when dividing by $0$. +* The composition $h(x) = f(g(x))$ is continuous at $x=c$ *if* $g(x)$ is continuous at $c$ *and* $f(x)$ is continuous at $g(c)$. + +Continuity is preserved for all of the basic operations except when dividing by $0$. +::: ##### Examples - * Since a monomial $f(x) = ax^n$ ($n$ a non-negative integer) is continuous, by the first rule, any polynomial will be continuous. - * Since both $f(x) = e^x$ and $g(x)=\sin(x)$ are continuous everywhere, so will be $h(x) = e^x \cdot \sin(x)$. - * Since $f(x) = e^x$ is continuous everywhere and $g(x) = -x$ is continuous everywhere, the composition $h(x) = e^{-x}$ will be continuous everywhere. - * Since $f(x) = x$ is continuous everywhere, the function $h(x) = 1/x$ - a ratio of continuous functions - will be continuous everywhere *except* possibly at $x=0$ (where it is not continuous). - * The function $h(x) = e^{x\ln(x)}$ will be continuous on $(0,\infty)$, the same domain that $g(x) = x\ln(x)$ is continuous. This function (which simplifies to $x^x$ when $x>0$) has a right limit at $0$ (of $1$), but is not right continuous, as $h(0)$ is not defined. (The function `h(x) = exp(x*log(x))` is not defined at `0` **but** the function `h(x) = x^x` is defined at `0.0` to be `1.0`.) +* Since a monomial $f(x) = ax^n$ ($n$ a non-negative integer) is continuous, by the first rule, any polynomial will be continuous. + +* Since both $f(x) = e^x$ and $g(x)=\sin(x)$ are continuous everywhere, so will be $h(x) = e^x \cdot \sin(x)$. + +* Since $f(x) = e^x$ is continuous everywhere and $g(x) = -x$ is continuous everywhere, the composition $h(x) = e^{-x}$ will be continuous everywhere. + +* Since $f(x) = x$ is continuous everywhere, the function $h(x) = 1/x$---a ratio of continuous functions---will be continuous everywhere *except* possibly at $x=0$ (where it is not continuous). + +* The function $h(x) = e^{x\ln(x)}$ will be continuous on $(0,\infty)$, the same domain that $g(x) = x\ln(x)$ is continuous. This function (which simplifies to $x^x$ when $x>0$) has a right limit at $0$ (of $1$), but is not right continuous, as $h(0)$ is not defined. (In `Julia`, the function `h(x) = exp(x*log(x))` is not defined at `0` **but** the function `h(x) = x^x` is defined at `0.0` to be `1.0`.) ## Questions diff --git a/quarto/limits/intermediate_value_theorem.qmd b/quarto/limits/intermediate_value_theorem.qmd index c80eb02..c865cec 100644 --- a/quarto/limits/intermediate_value_theorem.qmd +++ b/quarto/limits/intermediate_value_theorem.qmd @@ -10,22 +10,24 @@ This section uses these add-on packages: using CalculusWithJulia using Plots plotly() -using Roots using SymPy +using Roots # zero-finding algorithms ``` --- +::: {#fig-lhospitals-plot-of-ivt} +![](figures/ivt.jpg){width=40%} -![Between points M and M lies an F for a continuous curve. [L'Hospitals](https://ia801601.us.archive.org/26/items/infinimentpetits1716lhos00uoft/infinimentpetits1716lhos00uoft.pdf) figure 55.](figures/ivt.jpg){width=40%} +Between points M and M lies an F for a continuous curve. [L'Hospitals](https://ia801601.us.archive.org/26/items/infinimentpetits1716lhos00uoft/infinimentpetits1716lhos00uoft.pdf) figure 55. +::: Continuity for functions is a valued property which carries implications. In this section we discuss two: the intermediate value theorem and the extreme value theorem. These two theorems speak to some fundamental applications of calculus: finding zeros of a function and finding extrema of a function. ## Intermediate Value Theorem -::: {.callout-note icon=false} -## The intermediate value theorem +::: {.theorem title="The intermediate value theorem"} If $f$ is continuous on $[a,b]$ with, say, $f(a) < f(b)$, then for any $y$ with $f(a) \leq y \leq f(b)$ there exists a $c$ in $[a,b]$ with $f(c) = y$. @@ -100,7 +102,7 @@ plotly() nothing ``` -Illustration of the intermediate value theorem. The theorem implies that any randomly chosen $y$ value between $f(a)$ and $f(b)$ will have at least one $c$ in $[a,b]$ with $f(c)=y$. This graphic shows one of several possible values for the given choice of $y$. +Illustration of the intermediate value theorem. The theorem implies that any arbitrarily chosen $y$ value between $f(a)$ and $f(b)$ will have at least one $c$ in $[a,b]$ with $f(c)=y$. This graphic shows one of several possible values for the given choice of $y$. ::: @@ -116,20 +118,20 @@ The basic proof starts with a set of points in $[a,b]$: $C = \{x \text{ in } [a, Suppose we have a continuous function $f(x)$ on $[a,b]$ with $f(a) < 0$ and $f(b) > 0$. Then as $f(a) < 0 < f(b)$, the intermediate value theorem guarantees the existence of a $c$ in $[a,b]$ with $f(c) = 0$. This was a special case of the intermediate value theorem proved by Bolzano first. Such $c$ are called *zeros* of the function $f$. -Here, we use the Bolzano theorem to give an algorithm - the *bisection method* - to locate a value $c$ in $[a,b]$ with $f(c) = 0$ under the assumptions: +Here, we use the Bolzano theorem to give an algorithm---the *bisection method*---to locate a value $c$ in $[a,b]$ with $f(c) = 0$ under the assumptions: -* $f$ is continuous on $[a,b]$ +* $f$ is continuous on $[a,b]$; -* $f$ changes sign between $a$ and $b$. (In particular, when $f(a)$ and $f(b)$ have different signs.) +* $f(a)$ and $f(b)$ have different signs. ::: {.callout-note} -#### Between +## Between The bisection method is used to find a zero, $c$, of $f(x)$ *between* two values, $a$ and $b$. The method is guaranteed to work under the assumption of a continuous function having different signs at $a$ and $b$. ::: - +::: {#fig-bisection-method-animation} ```{julia} #| hold: true #| echo: false @@ -172,17 +174,16 @@ imgfile = tempname() * ".gif" gif(anim, imgfile, fps = 1) -caption = L""" +caption = "" +plotly() +ImageFile(imgfile, caption) +``` Illustration of the bisection method to find a zero of a function. At each step the interval has $f(a)$ and $f(b)$ having opposite signs so that the intermediate value theorem guarantees a zero. -""" - -plotly() -ImageFile(imgfile, caption) -``` +::: Call $[a,b]$ a *bracketing* interval if $f(a)$ and $f(b)$ have different signs. We remark that having different signs can be expressed mathematically as $f(a) \cdot f(b) < 0$. @@ -190,12 +191,14 @@ Call $[a,b]$ a *bracketing* interval if $f(a)$ and $f(b)$ have different signs. We can narrow down where a zero is in $[a,b]$ by following this recipe: - * Pick a midpoint of the interval, for concreteness $c = (a+b)/2$. - * If $f(c) = 0$ we are done, having found a zero in $[a,b]$. - * Otherwise it must be that either $f(a)\cdot f(c) < 0$ or $f(c) \cdot f(b) < 0$. If $f(a) \cdot f(c) < 0$, then let $b=c$ and repeat the above. Otherwise, let $a=c$ and repeat the above. +* Pick a midpoint of the interval, for concreteness $c = (a+b)/2$. + +* If $f(c) = 0$ we are done, having found a zero in $[a,b]$. + +* Otherwise it must be that either $f(a)\cdot f(c) < 0$ or $f(c) \cdot f(b) < 0$. If $f(a) \cdot f(c) < 0$, then let $b=c$ and repeat the above. Otherwise, let $a=c$ and repeat the above. -At each step the bracketing interval is narrowed – indeed split in half as defined – or a zero is found. +At each step the bracketing interval is narrowed---indeed split in half as defined---or a zero is found. For the real numbers this algorithm never stops unless a zero is found. A "limiting" process is used to say that if it doesn't stop, it will converge to some value. @@ -209,26 +212,27 @@ We can write a relatively simple program to implement this algorithm: ```{julia} function simple_bisection(f, a, b) - if f(a) == 0 return(a) end - if f(b) == 0 return(b) end - if f(a) * f(b) > 0 error("[a,b] is not a bracketing interval") end + f(a) == 0 && return a + f(b) == 0 && return b + f(a) * f(b) > 0 && error("[a,b] is not a bracketing interval") - tol = 1e-14 # small number (but should depend on size of a, b) - c = a/2 + b/2 - - while abs(b-a) > tol - if f(c) == 0 return(c) end - - if f(a) * f(c) < 0 - a, b = a, c - else - a, b = c, b - end + tol = 1e-14 # small number (but should depend on size of a, b) c = a/2 + b/2 - end - c + while abs(b-a) > tol + f(c) == 0 && return c + + if f(a) * f(c) < 0 + a, b = a, c + else + a, b = c, b + end + + c = a/2 + b/2 + + end + c end ``` @@ -254,8 +258,9 @@ sin(c) ### The `find_zero` function to solve `f(x) = 0` +The `Roots` package has a function `find_zero` that implements the bisection method when called as `find_zero(f, (a,b))` where $[a,b]$ is a bracketing interval for $f$, a continuous function. Its use is similar to `simple_bisection` above. This package is part of the `CalculusWithJulia` package. -The `Roots` package has a function `find_zero` that implements the bisection method when called as `find_zero(f, (a,b))` where $[a,b]$ is a bracket. Its use is similar to `simple_bisection` above. This package is loaded when `CalculusWithJulia` is. We illlustrate the usage of `find_zero` in the following: +We illlustrate the usage of `find_zero` in the following to numerically find a zero of $f(x) = \sin(x)$ in $[3,4]$: ```{julia} @@ -264,15 +269,16 @@ xstar = find_zero(sin, (3, 4)) :::{.callout-note} ## Action template -Notice, the call `find_zero(sin, (3, 4))` again fits the template `action(function, args...)` that we see repeatedly. The `find_zero` function can also be called through `fzero`. The use of `(3, 4)` to specify the interval is not necessary. For example `[3,4]` would work equally as well. (Anything where `extrema` is defined works.) +Notice, the call `find_zero(sin, (3, 4))` again fits the template `action(function, args...)` that we see repeatedly. The use of `(3, 4)` to specify the interval is not necessary. For example `[3,4]` would work equally as well. (Anything where `extrema` is defined works.) ::: -This function utilizes some facts about floating point values to guarantee that the answer will be an *exact* zero or a value where there is a sign change between the next bigger floating point or the next smaller, which means the sign at the next and previous floating point values is different: +This function utilizes some facts about floating point values to guarantee that the answer will be an *exact* zero (this answer isn't) *or& a value where there is a sign change between the next bigger floating point or the next smaller, which means the sign at the next and previous floating point values is different: ```{julia} -sin(xstar), sign(sin(prevfloat(xstar))), sign(sin(nextfloat(xstar))) +prev, next = prevfloat(xstar), nextfloat(xstar) +sign(sin(prev)), sign(sin(next)) ``` ##### Example @@ -297,13 +303,17 @@ p(c₀), sign(p(prevfloat(c₀))), sign(p(nextfloat(c₀))) ##### Example -The function $q(x) = e^x - x^4$ has a zero between $5$ and $10$, as this graph shows: - +The function $q(x) = e^x - x^4$ has a zero between $5$ and $10$, as @fig-plot-exp-x-minus-x-to-4-from-5-to-10 shows. +::: {#fig-plot-exp-x-minus-x-to-4-from-5-to-10} ```{julia} +#| echo: false q(x) = exp(x) - x^4 -plot(q, 5, 10) +plot(q, 5, 10; legend=false) +plot!(zero) ``` +Plot of $e^x - x^4$ over $[5, 10]$ +::: Find the zero numerically. The plot shows $q(5) < 0 < q(10)$, so $[5,10]$ is a bracket. We thus have: @@ -313,7 +323,7 @@ find_zero(q, (5, 10)) ``` ::: {.callout-note} -### Between need not be near +## Between need not be near Later, we will see more efficient algorithms to find a zero *near* a given guess. The bisection method finds a zero *between* two values of a bracketing interval. This interval need not be small. Indeed in many cases it can be infinite. For this particular problem, any interval like `(2,N)` will work as long as `N` is bigger than the zero and small enough that `q(N)` is finite *or* infinite *but* not `NaN`. (Basically, `q` must evaluate to a number with a sign. Here, the value of `q(Inf)` is `NaN` as it evaluates to the indeterminate `Inf - Inf`. But `q` is still not `NaN` for quite large numbers, such as `1e77`, as `x^4` can as big as `1e308`---technically `floatmax(Float64)`---and be finite.) @@ -328,24 +338,28 @@ Find all real zeros of $f(x) = x^3 -x + 1$ using the bisection method. We show next that symbolic values can be used with `find_zero`, should that be useful. -First, we produce a plot to identify a bracketing interval - +First, in @fig-plot-x-cubed-minus-x-plus-1-over-minus3-3 we produce a plot over $[-3,3]$ to identify a bracketing interval. +::: {#fig-plot-x-cubed-minus-x-plus-1-over-minus3-3} ```{julia} +#| echo: false @syms x plot(x^3 - x + 1, -3, 3) ``` +Plot of $x^3 - x + 1$ over $[-3,3]$ +::: It appears (and a plot over $[0,1]$ verifies) that there is one zero between $-2$ and $-1$. It is found with: ```{julia} +@syms x find_zero(x^3 - x + 1, (-2, -1)) ``` -#### The `find_zero` function to solve `f(x) = c` +### The `find_zero` function to solve `f(x) = c` -Solving `f(x) = c` is related to solving `h(x) = 0`. The key is to make a new function using the difference of the two sides: `h(x) = f(x) - c`. +Solving `f(x) = c` is related to solving `h(x) = 0` for a related `h`. The key is to make a new function using the difference of the two sides: `h(x) = f(x) - c`. ##### Example @@ -385,18 +399,20 @@ end The check on `fa < fb` is due to the possibility that $f$ is increasing (in which case `fa < fb`) or decreasing (in which case `fa > fb`). -To see this used, we consider the monotonic function $f(x) = x - \sin(x)$ over $[0, 5\pi]$. To graph, we have: +To see this used, we consider the monotonic function $f(x) = x - \sin(x)$ over $[0, 5\pi]$. We make @fig-plot-of-numerically-found-inverse-to-x-minus-sinx with the following commands. (We plot over the range $[f(a), f(b)]$ here, as we can guess $f(x)$ is *increasing*.) +::: {#fig-plot-of-numerically-found-inverse-to-x-minus-sinx} ```{julia} f(x) = x - sin(x) a, b = 0, 5pi -plot(inverse_function(f, a, b), f(a), f(b); aspect_ratio=:equal) +plot(inverse_function(f, a, b), f(a), f(b); aspect_ratio=:equal, legend=false) ``` - -(We plot over the range $[f(a), f(b)]$ here, as we can guess $f(x)$ is *increasing*.) +Plot of numerically identified inverse function of $f(x) = x - \sin(x)$. +::: -#### The `find_zero` function to solve `f(x) = g(x)` + +### The `find_zero` function to solve `f(x) = g(x)` Solving `f(x) = g(x)` is related to solving `h(x) = 0`. The key is to make a new function using the difference of the two sides: `h(x) = f(x) - g(x)`. @@ -404,20 +420,23 @@ Solving `f(x) = g(x)` is related to solving `h(x) = 0`. The key is to make a new ##### Example -The equation $\cos(x) = x$ has just one solution, as can be seen in this plot: - +The equation $\cos(x) = x$ has just one solution, as can be seen in @fig-plot-cosx-x-over-minus-pi-pi where both $y=\cos(x)$ and $y=x$ are plotted over $[-\pi,\pi]$. Find it. +::: {#fig-plot-cosx-x-over-minus-pi-pi} ```{julia} +#| echo: false f(x) = cos(x) g(x) = x plot(f, -pi, pi) plot!(g) ``` -Find it. +Plot of $f(x)=\cos(x)$ and $g(x) = x$ over $[-\pi, \pi]$. +::: -We see from the graph that it is clearly between $0$ and $2$, so all we need is a function. (We have two.) The trick is to observe that solving $f(x) = g(x)$ is the same problem as solving for $x$ where $f(x) - g(x) = 0$. So we define the difference and use that: + +We see from the graphs that the lone intersection is clearly between $0$ and $2$, so all we need is a function. (We have two.) The trick is to observe that solving $f(x) = g(x)$ is the same problem as solving for $x$ where $f(x) - g(x) = 0$. So we define the difference and use that: ```{julia} @@ -426,7 +445,7 @@ find_zero(h, (0, 2)) ``` ::: {.callout-note} -### Solving `f(x) = g(x)` and `f(x) = c` +## Solving `f(x) = g(x)` and `f(x) = c` The above examples show a means to translate a given problem into one that can be solved with `find_zero`. Basically to solve either when a function is a non-zero constant (`f(x) = c`) or when a function is equal to some other function (`f(x) = g(x)`), the difference between the two sides is formed and turned into a function, called `h` above. @@ -438,16 +457,21 @@ find_zero(cos(x) ~ x, (0, 2)) ``` ::: -[![Intersection of two curves as illustrated by Canadian artist Kapwani Kiwanga.](figures/intersection-biennale.jpg)](https://www.gallery.ca/whats-on/touring-exhibitions-and-loans/around-the-world/canada-pavilion-at-the-venice-biennale/kapwani-kiwanga-trinket){width=40%} +::: {#fig-intersection-of-two-curves-biennale} + +![](figures/intersection-biennale.jpg){width=60%} + +Intersection of two curves as illustrated by Canadian artist [Kapwani Kiwanga](https://www.gallery.ca/whats-on/touring-exhibitions-and-loans/around-the-world/canada-pavilion-at-the-venice-biennale/kapwani-kiwanga-trinket) +::: ##### Example We wish to compare two trash collection plans +* Plan 1: You pay $47.49$ plus $0.77$ per bag. - * Plan 1: You pay $47.49$ plus $0.77$ per bag. - * Plan 2: You pay $30.00$ plus $2.00$ per bag. +* Plan 2: You pay $30.00$ plus $2.00$ per bag. There are some cases where plan 1 is cheaper and some where plan 2 is. Categorize them. @@ -461,13 +485,16 @@ plan1(x) = 47.49 + 0.77x plan2(x) = 30.00 + 2.00x ``` -Assuming this is a realistic problem and an average American household might produce $10$-$20$ bags of trash a month (yes, that seems too much!) we plot in that range: - +Assuming this is a realistic problem and an average American household might produce $10$-$20$ bags of trash a month (yes, that seems too much!) @fig-plot-two-trash-plans shows the two plots in that range. +::: {#fig-plot-two-trash-plans} ```{julia} -plot(plan1, 10, 20) -plot!(plan2) +#| echo: false +plot(plan1, 10, 20; label="Plan 1") +plot!(plan2; label="Plan 2") ``` +Plots of two plans for trash collection +::: We can see the intersection point is around $14$ and that if a family generates between $0$-$14$ bags of trash per month that plan $2$ would be cheaper. @@ -518,14 +545,16 @@ end For each model, we wish to find the value of $x$ after launching where the height is modeled to be $0$. That is how far will the arrow travel before touching the ground? -For the model without wind resistance, we can graph the function easily enough. Let's guess the distance is no more than $500$ feet: - +For the model without wind resistance, we can graph the function easily enough. Let's guess the distance is no more than $500$ feet. +::: {#fig-plot-j-over-0-500-no-peak} ```{julia} plot(j, 0, 500) ``` +Plot of projectile motion modeled by `j` over $[0, 500]$ +::: -Well, we haven't even seen the peak yet. Better to do a little spade work first. This is a quadratic function, so we can use `solve` from `SymPy` to find the roots: +Well, in @fig-plot-j-over-0-500-no-peak we haven't even seen the peak yet. Better to do a little spade work first. This is a quadratic function, so we can use `solve` from `SymPy` to find the roots: ```{julia} @@ -533,21 +562,28 @@ Well, we haven't even seen the peak yet. Better to do a little spade work first. solve(j(x) ~ 0, x) ``` -We see that $1250$ is the largest root. So we plot over this domain to visualize the flight: - +We see that $1250$ is the largest root. In @fig-plot-j-over-0-1250-found-by-solve we use this domain to visualize the flight. +::: {#fig-plot-j-over-0-1250-found-by-solve} ```{julia} +#| echo: false plot(j, 0, 1250) ``` +Plot of `j` over $[0, 1250]$ +::: -As for the model with wind resistance, a quick plot over the same interval, $[0, 1250]$ yields: +As for the model with wind resistance, a quick plot over the same interval, $[0, 1250]$ yields @fig-plot-d-over-0-1250-wait-what. This graph eventually goes negative and then stops. This is due to the asymptote in model when `(a - gamma^2*x)/a` is zero. To plot the trajectory until it returns to $0$, we need to identify the value of the zero. This model is non-linear and we don't have the simplicity of using `roots` to find out the answer, so we solve for when $a-\gamma^2 x$ is $0$: + +::: {#fig-plot-d-over-0-1250-wait-what} ```{julia} +#| echo: false plot(d, 0, 1250) ``` +Plot of model, `d`, with wind resistance over $[0, 1250]$. Once the $x$ values cross over the asymptote value, not plot is drawn. +::: -This graph eventually goes negative and then stops. This is due to the asymptote in model when `(a - gamma^2*x)/a` is zero. To plot the trajectory until it returns to $0$, we need to identify the value of the zero. This model is non-linear and we don't have the simplicity of using `roots` to find out the answer, so we solve for when $a-\gamma^2 x$ is $0$: ```{julia} @@ -556,7 +592,7 @@ a = 200 * cos(pi/4) b = a/gamma^2 ``` -Note that the function is infinite at `b`: +Note that the function is infinite at `b`, as `b` is a vertical asymptote: ```{julia} @@ -576,13 +612,15 @@ The answer is approximately $140.7$ (The bisection method only needs to know the sign of the function. Other bracketing methods would have issues with an endpoint with an infinite function value. To use them, some value between the zero and `b` would needed.) -Finally, we plot both graphs at once to see that it was a very windy day indeed. - +Finally, we plot both graphs at once in @fig-plot-projectile-motion-motion-with-wind to see that it was a very windy day indeed. +::: {#fig-plot-projectile-motion-motion-with-wind} ```{julia} plot(j, 0, 1250, label="no wind") plot!(d, 0, x1, label="windy day") ``` +Plots of both `j` and `d` showing the difference on the trajectory that wind can make +::: ##### Example: bisection and non-continuity @@ -590,13 +628,16 @@ plot!(d, 0, x1, label="windy day") The Bolzano theorem assumes a continuous function $f$, and when applicable, yields an algorithm to find a guaranteed zero. -However, the algorithm itself does not know that the function is continuous or not, only that the function changes sign. As such, it can produce useful answers that are not "zeros" when applied to discontinuous functions. +However, the algorithm itself does not know that the function is continuoust, only that the function changes sign. As such, it can produce useful answers that are not "zeros" when applied to discontinuous functions. In general a function over floating point values could be considered as a large table of mappings: each of the $2^{64}$ floating point values gets assigned a value. This is a discrete mapping, there is nothing the computer sees related to continuity. +::: {.relationship title="Continuity needs verification"} + +For numeric algorithms, the concept of continuity, if needed, must be verified by the user of the algorithm. Floating point ultimately is about discrete functions. +::: -> The concept of continuity, if needed, must be verified by the user of the algorithm. @@ -610,21 +651,21 @@ As an example, let $f(x) = 1/x$. Clearly the interval $[-1,1]$ is a "bracketing" ```{julia} -fᵢ(x) = 1/x -x0 = find_zero(fᵢ, (-1, 1)) +f(x) = 1/x +x0 = find_zero(f, (-1, 1)) ``` The function is not defined at the answer, but we do have the fact that just to the left of the answer (`prevfloat`) and just to the right of the answer (`nextfloat`) the function changes sign: ```{julia} -sign(fᵢ(prevfloat(x0))), sign(fᵢ(nextfloat(x0))) +sign(f(prevfloat(x0))), sign(f(nextfloat(x0))) ``` -So, the "bisection method" applied here finds a point where the function crosses $0$, either by continuity or by jumping over the $0$. (A `jump` discontinuity at $x=c$ is defined by the left and right limits of $f$ at $c$ existing but being unequal. The algorithm can find $c$ when this type of function jumps over $0$.) +So, the "bisection method" applied here finds a point where the function crosses $0$, either by continuity or by jumping over the $0$. (A `jump` discontinuity at $x=c$ is defined by the left and right limits of $f$ at $c$ existing but being unequal.) -#### Using parameterized functions (`f(x,p)`) with `find_zero` +### Using parameterized functions (`f(x,p)`) with `find_zero` Geometry will tell us that $\cos(x) - x/p$ for *one* $x$ in $[0, \pi/2]$ whenever $p>0$. We could set up finding this value for a given $p$ by making $p$ part of the function definition, but as an illustration of passing parameters, we leave `p` as a parameter (in this case, as a second value with default of $1$): @@ -647,6 +688,33 @@ find_zero.(f, Ref(I), 1:5) # solutions for p=1,2,3,4,5 (The use of `Ref` above prevents broadcasting over the specified bracketing interval.) + +### An alternate interface to `find_zero` + + +The `find_zero` function in the `Roots` package is an interface to one of several methods. For now we focus on the *bracketing* methods, later we will see others. Bracketing methods, among others, include `Roots.Bisection()`, the basic bisection method though with a different sense of "middle" than $(a+b)/2$ and used by default above; `Roots.A42()`, which will typically converge much faster than simple bisection; `Roots.Brent()` for the classic method of Brent, and `FalsePosition()` for a family of *regula falsi* methods. These can all be used by specifying the method in a call to `find_zero`. + + +Alternatively, `Roots` implements the `CommonSolve` interface popularized by its use in the `DifferentialEquations.jl` ecosystem, a wildly successful area for `Julia`. The basic setup involves two steps: setup a "problem"; solve the problem. + + +To set up a problem we call `ZeroProblem` with the function and an initial bracketing interval, as in: + + +```{julia} +f(x) = x^5 - x - 1 +prob = ZeroProblem(f, (1,2)) +``` + +Then we can "solve" this problem with `solve`. For example: + + +```{julia} +solve(prob), solve(prob, Roots.Brent()), solve(prob, Roots.A42()) +``` + +Though the answers are identical, the methods employed were not. The first call, with an unspecified method, defaults to bisection. + ### The `find_zeros` function @@ -701,33 +769,6 @@ f.(zs) (For a continuous function this should be the case that the values returned by `find_zeros` are approximate zeros. Bear in mind that if $f$ is not continuous the algorithm might find jumping points that are not zeros and may not even be in the domain of the function.) -### An alternate interface to `find_zero` - - -The `find_zero` function in the `Roots` package is an interface to one of several methods. For now we focus on the *bracketing* methods, later we will see others. Bracketing methods, among others, include `Roots.Bisection()`, the basic bisection method though with a different sense of "middle" than $(a+b)/2$ and used by default above; `Roots.A42()`, which will typically converge much faster than simple bisection; `Roots.Brent()` for the classic method of Brent, and `FalsePosition()` for a family of *regula falsi* methods. These can all be used by specifying the method in a call to `find_zero`. - - -Alternatively, `Roots` implements the `CommonSolve` interface popularized by its use in the `DifferentialEquations.jl` ecosystem, a wildly successful area for `Julia`. The basic setup involves two steps: setup a "problem"; solve the problem. - - -To set up a problem we call `ZeroProblem` with the function and an initial interval, as in: - - -```{julia} -f(x) = x^5 - x - 1 -prob = ZeroProblem(f, (1,2)) -``` - -Then we can "solve" this problem with `solve`. For example: - - -```{julia} -solve(prob), solve(prob, Roots.Brent()), solve(prob, Roots.A42()) -``` - -Though the answers are identical, the methods employed were not. The first call, with an unspecified method, defaults to bisection. - - ## Extreme value theorem @@ -736,8 +777,7 @@ The Extreme Value Theorem is another consequence of continuity. To discuss the extreme value theorem, we define an *absolute maximum*. -::: {.callout-note icon=false} -## Absolute maximum, absolute minimum +::: {.definition title="Absolute maximum, absolute minimum"} The absolute maximum of $f(x)$ over an interval $I$, when it exists, is the value $f(c)$, $c$ in $I$, where $f(x) \leq f(c)$ for any $x$ in $I$. @@ -747,10 +787,11 @@ Similarly, an *absolute minimum* of $f(x)$ over an interval $I$ can be defined, Related but different is the concept of a relative of *local extrema*: -::: {.callout-note icon=false} -## Local maximum, local minimum +::: {.definition title="Local maximum, local minimum, local extrema"} -A local maxima for $f$ is a value $f(c)$ where $c$ is in **some** *open* interval $I=(a,b)$, $I$ in the domain of $f$, and $f(c)$ is an absolute maxima for $f$ over $I$. Similarly, a local minima for $f$ is a value $f(c)$ where $c$ is in **some** *open* interval $I=(a,b)$, $I$ in the domain of $f$, and $f(x)$ is an absolute minima for $f$ over $I$. +A local maxima for $f$ is a value $f(c)$ where $c$ is in **some** *open* interval $I=(a,b),$ $I$ in the domain of $f$, and $f(c)$ is an absolute maxima for $f$ over $I$. + +Similarly, a local minima for $f$ is a value $f(c)$ where $c$ is in **some** *open* interval $I=(a,b)$, $I$ in the domain of $f$, and $f(x)$ is an absolute minima for $f$ over $I$. The term *local extrema* is used to describe either a local maximum or local minimum. @@ -761,24 +802,13 @@ The key point, is the extrema are values in the *range* that are realized by som This chart of the [Hardrock 100](http://hardrock100.com/) illustrates the two concepts. +::: {#fig-hardrock-100-elevation-profile} +![](figures/hardrock-100.jpeg) -```{julia} -#| echo: false -###{{{hardrock_profile}}} -imgfile = "figures/hardrock-100.jpeg" -caption = """ -Elevation profile of the Hardrock 100 ultramarathon. Treating the elevation profile as a function, the absolute maximum is just about 14,000 feet and the absolute minimum about 7600 feet. These are of interest to the runner for different reasons. Also of interest would be each local maxima and local minima - the peaks and valleys of the graph - and the total elevation climbed - the latter so important/unforgettable its value makes it into the chart's title. -""" +Elevation profile of the Hardrock 100 ultramarathon (for one race, it alternates direction). Treating the elevation profile as a function, the absolute maximum is just about 14,000 feet and the absolute minimum about 7600 feet. These are of interest to the runner for different reasons. Also of interest would be each local maxima and local minima - the peaks and valleys of the graph - and the total elevation climbed - the latter so important/unforgettable its value makes it into the chart's title. +::: -# ImageFile(:limits, imgfile, caption) -nothing -``` - -[![Elevation profile of the Hardrock 100 ultramarathon. Treating the elevation profile as a function, the absolute maximum is just about 14,000 feet and the absolute minimum about 7600 feet. These are of interest to the runner for different reasons. Also of interest would be each local maxima and local minima - the peaks and valleys of the graph - and the total elevation climbed - the latter so important/unforgettable its value makes it into the chart's title. -](figures/hardrock-100.jpeg)](https://hardrock100.com){width=50%} - - -This figure shows the two concepts as well. +@fig-absolute-relative also illustrates the two concepts. ::: {#fig-absolute-relative} @@ -821,11 +851,11 @@ plt = let annotate!([ (a, 0, text(L"a", :top)), (b,0, text(L"b", :top)), - (a + κ/5, p(a), text(raw"absolute max", 10, :left)), - (z₁, p(z₁)-κ, text(raw"absolute min", 10, :top)), + (a + κ/5, p(a), text(raw"absolute max, endpoint", 10, :left)), + (z₁, p(z₁)-κ, text(raw"absolute min, relative min", 10, :top)), (z₂, p(z₂) + κ, text(raw"relative max", 10, :bottom)), (z₃, p(z₃) - κ, text(raw"relative min", 10, :top)), - (b, p(b) + κ, text(raw"endpoint", 10, :bottom)) + (b, p(b) + κ, text(raw"endpoint", 10, :bottom, :right)) ]) @@ -847,8 +877,7 @@ Figure illustrating absolute and relative minima for a function $f(x)$ over $I=[ The extreme value theorem discusses an assumption that ensures absolute maximum and absolute minimum values exist. -::: {.callout-note icon=false} -## The extreme value theorem +::: {.theorem title="The extreme value theorem"} If $f(x)$ is continuous over a closed interval $[a,b]$ then $f$ has an absolute maximum and an absolute minimum over $[a,b]$. @@ -883,13 +912,13 @@ plot(x -> x * exp(-x), 0, 5) ##### Example -The tangent function does not have a *guarantee* of an absolute maximum or a minimum over $(-\pi/2, \pi/2),$ as it is not *continuous* at the endpoints. In fact, it doesn't have either extrema - it has vertical asymptotes at each endpoint of this interval. +The tangent function does not have a *guarantee* of an absolute maximum or a minimum over $(-\pi/2, \pi/2),$ as it is not *continuous* at the endpoints. In fact, it doesn't have either extrema---it has vertical asymptotes at each endpoint of this interval. ##### Example -The function $f(x) = x^{2/3}$ over the interval $[-2,2]$ has cusp at $0$. However, it is continuous on this closed interval, so must have an absolute maximum and absolute minimum. They can be seen from the graph to occur at the endpoints and the cusp at $x=0$, respectively: +The function $f(x) = x^{2/3}$ over the interval $[-2,2]$ has a cusp at $0$. However, it is continuous on this closed interval, so must have an absolute maximum and absolute minimum. They can be seen from the graph to occur at the endpoints and the cusp at $x=0$, respectively: ```{julia} @@ -923,30 +952,37 @@ That $f(I)$ is an interval is a consequence of the intermediate value theorem. T On the real line, sets that are closed and bounded are "compact," a term that generalizes to other settings. +::: {.relationship title="Images"} -> Continuity implies that the *image* of a compact set is compact. +Continuity implies that the *image* of a compact set is compact. +::: + + +Now let $(c,d)$ be an *open* interval in the range of $f$. An open interval is an open set. On the real line, an open set is one where each point in the set, $a$, has some $\delta$ such that if $\lvert b-a \rvert < \delta$ then $b$ is also in the set. + +::: {.relationship title="Pre-images"} +Continuity implies that the *preimage* of an open set is an open set. +::: -Now let $(c,d)$ be an *open* interval in the range of $f$. An open interval is an open set. On the real line, an open set is one where each point in the set, $a$, has some $\delta$ such that if $|b-a| < \delta$ then $b$ is also in the set. - - -> Continuity implies that the *preimage* of an open set is an open set. - - - -The *preimage* of an open set, $I$, is $\{a: f(a) \in I\}$. (All $a$ with an image in $I$.) Taking some pair $(a,y)$ with $y$ in $I$ and $a$ in the preimage as $f(a)=y$. Let $\epsilon$ be such that $|x-y| < \epsilon$ implies $x$ is in $I$. Then as $f$ is continuous at $a$, given $\epsilon$ there is a $\delta$ such that $|b-a| <\delta$ implies $|f(b) - f(a)| < \epsilon$ or $|f(b)-y| < \epsilon$ which means that $f(b)$ is in the $I$ so $b$ is in the preimage, implying the preimage is an open set. +The *preimage* of an open set, $I$, is $\{a: f(a) \in I\}$. (All $a$ with an image in $I$.) Taking some pair $(a,y)$ with $y$ in $I$ and $a$ in the preimage as $f(a)=y$. Let $\epsilon$ be such that $\lvert x-y \rvert < \epsilon$ implies $x$ is in $I$. Then as $f$ is continuous at $a$, given $\epsilon$ there is a $\delta$ such that $\lvert b-a \rvert <\delta$ implies $\lvert f(b) - f(a) \rvert < \epsilon$ or $\lvert f(b)-y \rvert < \epsilon$ which means that $f(b)$ is in the $I$ so $b$ is in the preimage, implying the preimage is an open set. ## Questions ###### Question -Consider the following plot +Consider the following plot in @fig-plot-airy-over-minus5-5. +::: {#fig-plot-airy-over-minus5-5} ```{julia} +#| echo: false plot(airy, -5, 5; xticks=-5:5) ``` +Plot of `airy` function over $[-5, 5]$ +::: + There is a guaranteed zero between: ```{julia} @@ -962,12 +998,16 @@ radioq(choices, answer; keep_order=true) ###### Question -Consider the following plot +Consider the plot in @fig-plot-erf-from-minus5-to-5 of the `erf` function. +::: {#fig-plot-erf-from-minus5-to-5} ```{julia} plot(erf, -5, 5; xticks=-5:5) ``` +Plot of `erf` function over $[-5, 5]$ +::: + There is a guaranteed zero to `erf(x) = 0.5` between @@ -1041,13 +1081,16 @@ numericq(val, 1e-3) ###### Question -The `airyai` function has infinitely many negative roots, as the function oscillates when $x < 0$ and *no* positive roots. Find the *second largest root* using the graph to bracket the answer, and then solve. - +The `airyai` function has infinitely many negative roots, as the function oscillates when $x < 0$ and *no* positive roots. @fig-plot-airyai-over-minus10-10-find-second shows the graph over $[-10,10]$. In this interval, *numerically* find the *second largest root* using the graph to bracket the answer, and then solve. +::: {#fig-plot-airyai-over-minus10-10-find-second} ```{julia} plot(airyai, -10, 10) # `airyai` loaded in `SpecialFunctions` by `CalculusWithJulia` ``` +Plot of `airyai` function over $[-10, 10]$. +::: + The second largest root is: @@ -1115,10 +1158,14 @@ Trajectories of potential cannonball fires with air-resistance included. (http:/ nothing ``` -![Trajectories of potential cannonball fires with air-resistance included. (http://ej.iop.org/images/0143-0807/33/1/149/Full/ejp405251f1_online.jpg) -](figures/cannonball.jpg){width=50%} -In 1638, according to Amir D. [Aczel](http://books.google.com/books?id=kvGt2OlUnQ4C&pg=PA28&lpg=PA28&dq=mersenne+cannon+ball+tests&source=bl&ots=wEUd7e0jFk&sig=LpFuPoUvODzJdaoug4CJsIGZZHw&hl=en&sa=X&ei=KUGcU6OAKJCfyASnioCoBA&ved=0CCEQ6AEwAA#v=onepage&q=mersenne%20cannon%20ball%20tests&f=false), an experiment was performed in the French Countryside. A monk, Marin Mersenne, launched a cannonball straight up into the air in an attempt to help Descartes prove facts about the rotation of the earth. Though the experiment was not successful, Mersenne later observed that the time for the cannonball to go up was less than the time to come down. ["Vertical Projection in a Resisting Medium: Reflections on Observations of Mersenne".](http://www.maa.org/publications/periodicals/american-mathematical-monthly/american-mathematical-monthly-contents-junejuly-2014) +::: {#fig-woodcut-trajectories-air-resistance} +![](figures/cannonball.jpg){width=50%} + +Trajectories of potential cannonball fires with air-resistance included +::: + +In 1638, according to Amir D. [Aczel](http://books.google.com/books?id=kvGt2OlUnQ4C&pg=PA28&lpg=PA28&dq=mersenne+cannon+ball+tests&source=bl&ots=wEUd7e0jFk&sig=LpFuPoUvODzJdaoug4CJsIGZZHw&hl=en&sa=X&ei=KUGcU6OAKJCfyASnioCoBA&ved=0CCEQ6AEwAA#v=onepage&q=mersenne%20cannon%20ball%20tests&f=false), an experiment was performed in the French Countryside. A monk, Marin Mersenne, launched a cannonball straight up into the air in an attempt to help Descartes prove facts about the rotation of the earth. Though the experiment was not successful, Mersenne later observed that the time for the cannonball to go up was less than the time to come down. ["Vertical Projection in a Resisting Medium: Reflections on Observations of Mersenne"](https://doi.org/10.4169/amer.math.monthly.121.06.499). This isn't the case for simple ballistic motion where the time to go up is equal to the time to come down. We can "prove" this numerically. For simple ballistic motion: @@ -1140,8 +1187,8 @@ Let $v_0= 390$. The three times in question can be found from the zeros of `f` a choices = ["``(0.0, 12.1875, 24.375)``", "``(-4.9731, 0.0, 4.9731)``", "``(0.0, 625.0, 1250.0)``"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question What goes up must come down... (again) @@ -1176,8 +1223,8 @@ ta = find_zero(D(h), (t0, tf)) choices = ["``(0, 13.187, 30.0)``", "``(0, 32.0, 390.0)``", "``(0, 2.579, 13.187)``"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` ###### Question @@ -1192,14 +1239,14 @@ Part of the proof of the intermediate value theorem rests on knowing what the li choices = [L"It must be that $L > y$ as each $f(x)$ is.", L"It must be that $L \geq y$", L"It can happen that $L < y$, $L=y$, or $L>y$"] -answ = 2 -radioq(choices, 2, keep_order=true) +answer = 2 +buttonq(choices, answer) ``` ###### Question -The extreme value theorem has two assumptions: a continuous function and a *closed* interval. Which of the following examples fails to satisfy the consequence of the extreme value theorem because the interval is not closed? (The consequence - the existence of an absolute maximum and minimum - can happen even if the theorem does not apply.) +The extreme value theorem has two assumptions: a continuous function and a *closed* interval. Which of the following examples fails to satisfy the consequence of the extreme value theorem because the interval is not closed? (The consequence---the existence of an absolute maximum and minimum---can happen even if the theorem does not apply.) ```{julia} @@ -1210,8 +1257,8 @@ choices = [ "``f(x) = \\sin(x),~ I=(-\\pi, \\pi)``", "``f(x) = \\sin(x),~ I=(-\\pi/2, \\pi/2)``", "None of the above"] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +buttonq(choices, answer) ``` ###### Question @@ -1228,8 +1275,8 @@ choices = [ "``f(x) = 1/x,~ I=[-2, -1]``", "``f(x) = 1/x,~ I=[-1, 1]``", "none of the above"] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +buttonq(choices, answer) ``` ###### Question @@ -1246,37 +1293,37 @@ choices = [ "``f(x) = 1/x,~ I=[-4, -1]``", "``f(x) = \\text{floor}(x),~ I=[-1/2, 1/2]``", "none of the above"] -answ = 4 -radioq(choices, answ, keep_order=true) +answer = 4 +buttonq(choices, answer) ``` ###### Question -The extreme value theorem is true when $f$ is a continuous function on an interval $I$ *and* $I=[a,b]$ is a *closed* interval. Which of these illustrates why it doesn't apply as $f$ is not continuous on $I$ but is defined on $I$? - +The extreme value theorem is true when $f$ is a continuous function on an interval $I$ *and* $I=[a,b]$ is a *closed* interval. Which of the graphs in @fig-examples-possibly-of-needed-conditions-in-extreme-value-theorem illustrates why it doesn't apply as $f$ is not continuous on $I$ but is defined everywhere on $I$? +::: {#fig-examples-possibly-of-needed-conditions-in-extreme-value-theorem} ```{julia} #| hold: true #| echo: false -let +several_plots = let gr() - empty_style = (xaxis=([], false), - yaxis=([], false), - framestyle=:origin, - legend=false) - axis_style = (arrow=true, side=:head, line=(:gray, 1)) + empty_style = (xaxis=([], false), + yaxis=([], false), + framestyle=:origin, + legend=false) + axis_style = (arrow=true, side=:head, line=(:gray, 1, :dash)) ts = range(0, 2pi, 100) # defined on I; not continuous on I p1 = plot(;empty_style..., aspect_ratio=:equal) title!(p1, "(a)") - plot!(p1, x -> 1 - abs(2x), -1, 1, color=:black) - plot!(p1, zero; line=(:black, 1), arrow=true, side=:head) - C = Shape(0.03 .* sin.(ts), 1 .+ 0.03 .* cos.(ts)) + plot!(p1, x -> 2.1 - abs(2x), -1, 1, color=:black) + plot!(p1, zero; axis_style...) + C = Shape(0.04 .* sin.(ts), 2.1-0.025 .+ 0.04 .* cos.(ts)) plot!(p1, C, fill=(:white, 1), line=(:black,1)) - C = Shape(0.03 .* sin.(ts), - 0.25 .+ 0.03 .* cos.(ts)) + C = Shape(0.04 .* sin.(ts), 1.25 .+ 0.04 .* cos.(ts)) plot!(p1, C, fill=(:black,1)) annotate!(p1, [ (-1,0,text(L"a", :top)), @@ -1298,7 +1345,7 @@ let p3 = plot(;empty_style...) title!(p3, "(c)") plot!(p3, x -> 1/(1-x), 0, .95, color=:black) - ylims!((-0.25, 1/(1 - 0.96))) + ylims!((-1.2, 1/(1 - 0.96))) plot!(p3, [0,1.05],[0,0]; axis_style...) vline!(p3, [1]; line=(:black, 1, :dash)) annotate!(p3,[ @@ -1323,89 +1370,27 @@ let l = @layout[a b; c d] p = plot(p1, p2, p3, p4, layout=l) - imgfile = tempname() * ".png" - savefig(p, imgfile) - hotspotq(imgfile, (0,1/2), (1/2,1)) end + +imgfile = tempname() * ".png" +savefig(several_plots, imgfile) +hotspotq(imgfile, (0,1/2), (1/2,1)) + ``` +Several plots, select one to match question +::: +The extreme value theorem is true when $f$ is a continuous function on an interval $I$ and $I=[a,b]$ is a *closed* interval. Which of the graphs in @fig-examples-possibly-of-needed-conditions-in-extreme-value-theorem-take-2 illustrates when the theorem's assumptions are true? -The extreme value theorem is true when $f$ is a continuous function on an interval $I$ and $I=[a,b]$ is a *closed* interval. Which of these illustrates when the theorem's assumptions are true? - - +::: {#fig-examples-possibly-of-needed-conditions-in-extreme-value-theorem-take-2} ```{julia} #| hold: true #| echo: false ## come on; save this figure... -let - gr() - empty_style = (xaxis=([], false), - yaxis=([], false), - framestyle=:origin, - legend=false) - axis_style = (arrow=true, side=:head, line=(:gray, 1)) - - ts = range(0, 2pi, 100) - - # defined on I; not continuous on I - p1 = plot(;empty_style..., aspect_ratio=:equal) - title!(p1, "(a)") - plot!(p1, x -> 1 - abs(2x), -1, 1, color=:black) - plot!(p1, zero; line=(:black, 1), arrow=true, side=:head) - C = Shape(0.03 .* sin.(ts), 1 .+ 0.03 .* cos.(ts)) - plot!(p1, C, fill=(:white, 1), line=(:black,1)) - C = Shape(0.03 .* sin.(ts), - 0.25 .+ 0.03 .* cos.(ts)) - plot!(p1, C, fill=(:black,1)) - annotate!(p1, [ - (-1,0,text(L"a", :top)), - (1,0,text(L"b", :top)) - ]) - - # not defined on I - p2 = plot(;empty_style...) - title!(p2, "(b)") - plot!(p2, x -> 1/(1-x), 0, .95, color=:black) - plot!(p2, x-> -1/(1-x), 1.05, 2, color=:black) - plot!(p2, zero; axis_style...) - annotate!(p2,[ - (0,0,text(L"a", :top)), - (2, 0, text(L"b", :top)) - ]) - - # not continuous on I - p3 = plot(;empty_style...) - title!(p3, "(c)") - plot!(p3, x -> 1/(1-x), 0, .95, color=:black) - ylims!((-0.1, 1/(1 - 0.96))) - plot!(p3, [0,1.05],[0,0]; axis_style...) - vline!(p3, [1]; line=(:black, 1, :dash)) - annotate!(p3,[ - (0,0,text(L"a", :top)), - (1, 0, text(L"b", :top)) - ]) - - # continuous - p4 = plot(;empty_style...) - title!(p4, "(d)") - f(x) = x^x - a, b = 0, 2 - ylims!(p4, (-.25, f(b))) - plot!(p4, f, a, b; line=(:black,1)) - plot!(p4, [a-.1, b+.1], [0,0]; axis_style...) - scatter!([0,2],[ f(0),f(2)]; marker=(:circle,:black)) - annotate!([ - (a, 0, text(L"a", :top)), - (b, 0, text(L"b", :top)) - - ]) - - l = @layout[a b; c d] - p = plot(p1, p2, p3, p4, layout=l) - imgfile = tempname() * ".png" - savefig(p, imgfile) - hotspotq(imgfile, (1/2,1), (0,1/2)) -end +imgfile = tempname() * ".png" +savefig(several_plots, imgfile) +hotspotq(imgfile, (1/2,1), (0,1/2)) ``` ```{julia} @@ -1413,6 +1398,8 @@ end plotly(); ``` +Several plots, select one to match question +::: ###### Question @@ -1454,8 +1441,8 @@ L"There is no value $c$ for which $f(c)$ is an absolute maximum over $I$.", L"There is just one value of $c$ for which $f(c)$ is an absolute maximum over $I$.", L"There are many values of $c$ for which $f(c)$ is an absolute maximum over $I$." ] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -1472,8 +1459,8 @@ L"There is no value $M$ for which $M=f(c)$, $c$ in $I$ for which $M$ is an absol L"There is just one value $M$ for which $M=f(c)$, $c$ in $I$ for which $M$ is an absolute maximum over $I$.", L"There are many values $M$ for which $M=f(c)$, $c$ in $I$ for which $M$ is an absolute maximum over $I$." ] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -1489,8 +1476,8 @@ choices = [ "``f(x) = \\sin(x),\\quad I=[-\\pi/2, \\pi/2]``", "``f(x) = \\sin(x),\\quad I=[0, 2\\pi]``", "``f(x) = \\sin(x),\\quad I=[-2\\pi, 2\\pi]``"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` ###### Question @@ -1505,3 +1492,61 @@ The zeros of the equation $\cos(x) \cdot \cosh(x) = 1$ are related to vibrations val = maximum(find_zeros(x -> cos(x) * cosh(x) - 1, (0, 6pi))) numericq(val) ``` + + +##### Question + +The `find_zeros` function only works *most of the time*. Here are three examples. + +Consider the function defined below: + +```{julia} +f(x) = x < -1 ? x + 1 : + x < 1 ? 0 : + x - 1 +``` + +This function is flat between $[-1, 1]$. Does `find_zeros(f, -3, 3)` find *all* the zeros of $f$? + +```{julia} +#| echo: false +choices = ["Yes", + L"No. It finds a lot, but there are infinitely many"] +answer = 2 +buttonq(choices, answer) +``` + +The function defined by: + +```{julia} +f(x) = x^7 - 16129x^2 + 254x - 1 +``` + +has three zeros in $[0,10]$. Does `find_zeros` find them all? + +```{julia} +#| echo: false +choices = ["Yes", + "No. There are two *really* close together near ``0.0078...`` and `find_zeros` finds just one"] +answer = 2 +buttonq(choices, answer) +``` + + +The function defined by + +```{julia} +f(x) = iszero(x) ? 0.0 : x * sin(1/x) +``` + +has infinitely many zeros in $[-1,1]$. How many does `find_zeros` find? + +```{julia} +#| echo: false +choices = ["Between 0-10", + "Between 11 - 50", + "Between 50 - 100", + "More than 100"] +answer = 3 +buttonq(choices, answer) +``` diff --git a/quarto/limits/limits.qmd b/quarto/limits/limits.qmd index 3cd99b6..9dadade 100644 --- a/quarto/limits/limits.qmd +++ b/quarto/limits/limits.qmd @@ -10,7 +10,7 @@ This section uses the following add-on packages: using CalculusWithJulia using Plots plotly() -using Richardson # for extrapolation +using Richardson # for numeric limits using SymPy # for symbolic limits ``` @@ -18,12 +18,12 @@ using SymPy # for symbolic limits --- -An historic problem in the history of math was to find the area under the graph of $f(x)=x^2$ between $[0,1]$. +A historic problem in the history of math was to find the area under the graph of $f(x)=x^2$ between $[0,1]$. There wasn't a ready-made formula for the area of this shape, as was known for a triangle or a square. However, [Archimedes](http://en.wikipedia.org/wiki/The_Quadrature_of_the_Parabola) found a method to compute areas enclosed by a parabola and line segments that cross the parabola. - +::: {#fig-make-archimedes-triangle-graph} ```{julia} #| hold: true #| echo: false @@ -84,33 +84,35 @@ gif(anim, imgfile, fps = 1) caption = L""" -The first triangle has area $1/2$, the second has area $1/8$, then $2$ have area $(1/8)^2$, $4$ have area $(1/8)^3$, ... -With some algebra, the total area then should be $1/2 \cdot (1 + (1/4) + (1/4)^2 + \cdots) = 2/3$. """ plotly() ImageFile(imgfile, caption) ``` -The figure illustrates a means to compute the area bounded by the parabola, the line $y=1$ and the line $x=0$ using triangles. It suggests that this area can be found by adding the following sum +The first triangle has area $1/2$, the second has area $1/8$, then $2$ have area $(1/8)^2$, $4$ have area $(1/8)^3$, ... With some algebra, the total area then should be $1/2 \cdot (1 + (1/4) + (1/4)^2 + \cdots) = 2/3$. +::: + + +@fig-make-archimedes-triangle-graph illustrates a means to compute the area bounded by the parabola, the line $y=1$ and the line $x=0$ using triangles. It suggests that this area can be found by adding the following sum $$ A = 1/2 + 1/8 + 2 \cdot (1/8)^2 + 4 \cdot (1/8)^3 + \cdots $$ -This value is $2/3$, so the area under the curve would be $1/3$. Forget about this specific value - which through more modern machinery becomes uneventful - and focus for a minute on the method: a problem is solved by a suggestion of an infinite process, in this case the creation of more triangles to approximate the unaccounted for area. This is the so-call method of [exhaustion](http://en.wikipedia.org/wiki/Method_of_exhaustion) known since the 5th century BC. +This value is $2/3$, so the area under the curve would be $1/3$. Forget about this specific value---which through more modern machinery becomes uneventful---and focus for a minute on the method: a problem is solved by a suggestion of an infinite process, in this case the creation of more triangles to approximate the unaccounted for area. This is the so-called method of [exhaustion](http://en.wikipedia.org/wiki/Method_of_exhaustion), a method known since the 5th century BC. Archimedes used this method to solve a wide range of area problems related to basic geometric shapes, including a more general statement of what we described above. -The $\cdots$ in the sum expression are the indication that this process continues and that the answer is at the end of an *infinite* process. To make this line of reasoning rigorous requires the concept of a limit. The concept of a limit is then an old one, but it wasn't until the age of calculus that it was formalized. +The $\cdots$ in the sum expression indicate that this process continues and that the answer is at the end of an *infinite* process. To make this line of reasoning rigorous requires the concept of a limit. The concept of a limit is then an old one, but it wasn't until the age of calculus that it was formalized. Next, we illustrate how Archimedes approximated $\pi$ – the ratio of the circumference of a circle to its diameter – using interior and exterior $n$-gons whose perimeters could be computed. - +::: {#fig-archimedes-pi-approximation} ```{julia} #| hold: true #| echo: false @@ -172,13 +174,15 @@ for (x, y, n, col) ∈ zip(xs, ys, ns, (blue, green, purple, red)) end caption = L""" -The ratio of the circumference of a circle to its diameter, $\pi$, can be approximated from above and below by computing the perimeters of the inscribed $n$-gons. Archimedes computed the perimeters for $n$ being $12$, $24$, $48$, and $96$ to determine that $3~10/71 \leq \pi \leq 3~1/7$. """ plotly() ImageFile(p, caption) ``` -Here Archimedes uses *bounds* to constrain an unknown value. Had he been able to compute these bounds for larger and larger $n$ the value of $\pi$ could be more accurately determined. In a "limit" it would be squeezed in to have a specific value, which we now know is an irrational number. +The ratio of the circumference of a circle to its diameter, $\pi$, can be approximated from above and below by computing the perimeters of the inscribed $n$-gons. Archimedes computed the perimeters for $n$ being $12$, $24$, $48$, and $96$ to determine that $3~10/71 \leq \pi \leq 3~1/7$. +::: + +With @fig-archimedes-pi-approximation Archimedes uses *bounds* to constrain an unknown value. Had he been able to compute these bounds for larger and larger $n$ the value of $\pi$ could be more accurately determined. In a "limit" it would be squeezed in to have a specific value, which we now know is an irrational number. Continuing these concepts, [Fermat](http://en.wikipedia.org/wiki/Adequality) in the 1600s essentially took a limit to find the slope of a tangent line to a polynomial curve. Newton in the late 1600s, exploited the idea in his development of calculus (as did Leibniz). Yet it wasn't until the 1800s that [Bolzano](http://en.wikipedia.org/wiki/Limit_of_a_function#History), Cauchy and Weierstrass put the idea on a firm footing. @@ -187,15 +191,20 @@ Continuing these concepts, [Fermat](http://en.wikipedia.org/wiki/Adequality) in To make things more precise, we begin by discussing the limit of a univariate function as $x$ approaches $c$. -Informally, if a limit exists it is the value that $f(x)$ gets close to as $x$ gets close to - but not equal to - $c$. +Informally, if a limit exists it is the value that $f(x)$ gets close to as $x$ gets close to---but not equal to---$c$. The modern formulation is due to Weierstrass: -::: {.callout-note icon=false} -## The $\epsilon-\delta$ Definition of a limit of $f(x)$ +::: {.definition title="Definition: a limit"} +The $\epsilon-\delta$ Definition of a limit of $f(x)$ defines +the limit of $f(x)$ as $x$ approaches $c$ as $L$, written, -The limit of $f(x)$ as $x$ approaches $c$ is $L$ if for every real $\epsilon > 0$, there exists a real $\delta > 0$ such that for all real $x$, $0 < \lvert x − c \rvert < \delta$ implies $\lvert f(x) − L \rvert < \epsilon$. The notation used is $\lim_{x \rightarrow c}f(x) = L$. +$$ +\lim_{x \rightarrow c} f(x) = L, +$$ + +by if for every real $\epsilon > 0$, there exists a real $\delta > 0$ such that for all real $x$, if $0 < \lvert x − c \rvert < \delta$ then $\lvert f(x) − L \rvert < \epsilon$. ::: @@ -210,7 +219,9 @@ $$ \frac{\sin(x)}{x} \quad\text{and}\quad (1 + x)^{1/x}. $$ -These take the indeterminate forms $0/0$ and $1^\infty$, which are found by just putting $0$ in for $x$. An expression does not need to be defined at $c$, as these two aren't at $c=0$, to discuss its limit. Cauchy illustrates two methods to approach the questions above. The first is to pull out an inequality: +These take the indeterminate forms $0/0$ and $1^\infty$, which are found by just putting $0$ in for $x$. To discuss a limit an expression does not need to be defined at $c$, as these two aren't at $c=0$. Cauchy illustrates two methods to approach the questions above. + +The first is to pull out an inequality: $$ @@ -227,7 +238,8 @@ $$ This bounds the expression $\sin(x)/x$ between $1$ and $\cos(x)$ and as $x$ gets close to $0$, the value of $\cos(x)$ "clearly" goes to $1$, hence $L$ must be $1$. This is an application of the squeeze theorem, the same idea Archimedes implied when bounding the value for $\pi$ above and below. -The above bound comes from this figure, for small $x > 0$: +The above bound comes from a picture like that in @fig-sin-cos-bound, for small $\theta > 0$: + ::: {#fig-sin-cos-bound} @@ -283,7 +295,7 @@ plotly() nothing ``` -Triangle $\triangle ABD$ has less area than the shaded wedge, which has less area than triangle $\triangle ACD$. Their respective areas are $(1/2)\sin(\theta)$, $(1/2)\theta$, and $(1/2)\tan(\theta)$. The inequality used to show $\sin(x)/x$ is bounded below by $\cos(x)$ and above by $1$ comes from a division by $(1/2) \sin(x)$ and taking reciprocals. +Triangle $\triangle ABD$ has less area than the shaded wedge, which has less area than triangle $\triangle ACD$. Their respective areas are $(1/2)\sin(\theta)$, $(1/2)\theta$, and $(1/2)\tan(\theta)$. The inequality used to show $\sin(\theta)/\theta$ is bounded below by $\cos(\theta)$ and above by $1$ comes from a division by $(1/2) \sin(\theta)$ and taking reciprocals. ::: @@ -306,7 +318,7 @@ xs = [1/10^i for i in 1:5] [xs f.(xs)] ``` -This progression can be seen to be increasing. Cauchy, in his treatise, can see this through: +This progression can be seen to be increasing. Cauchy, in his treatise, saw this through: $$ @@ -322,7 +334,7 @@ These values are clearly increasing as $m$ increases. Cauchy showed the value wa $$ -e^x = \lim_{n \rightarrow \infty} (1 + \frac{x}{n})^n, +e^x = \lim_{n \rightarrow \infty} \left(1 + \frac{x}{n}\right)^n, $$ with a suitably defined limit. @@ -337,14 +349,14 @@ These two cases illustrate that though the definition of the limit exists, the c First it should be noted that for most of the functions encountered, the concepts of a limit at a typical point $c$ is nothing more than just function evaluation at $c$. This is because, at a typical point, the functions are nicely behaved (what we will soon call "*continuous*"). However, most questions asked about limits find points that are not typical. For these, the result of evaluating the function at $c$ is typically undefined, and the value comes in one of several *indeterminate forms*: $0/0$, $\infty/\infty$, $0 \cdot \infty$, $\infty - \infty$, $0^0$, $1^\infty$, and $\infty^0$. -`Julia` can help - at times - identify these indeterminate forms, as many such operations produce `NaN`. For example: +`Julia` can help---at times---identify these indeterminate forms, as many such operations produce `NaN`. For example: ```{julia} 0/0, Inf/Inf, 0 * Inf, Inf - Inf ``` -However, the values with powers generally do not help, as the IEEE standard has `0^0` evaluating to 1: +However, the values with powers generally do not look indeterminate, as the IEEE standard has `0^0` evaluating to 1: ```{julia} @@ -375,49 +387,60 @@ The above is really just a heuristic. For some functions this is just not true. ## Graphical approaches to limits -Let's return to the function $f(x) = \sin(x)/x$. This function was studied by Euler as part of his solution to the [Basel](http://en.wikipedia.org/wiki/Basel_problem) problem. He knew that near $0$, $\sin(x) \approx x$, so the ratio is close to $1$ if $x$ is near $0$. Hence, the intuition is $\lim_{x \rightarrow 0} \sin(x)/x = 1$, as Cauchy wrote. We can verify this limit graphically two ways. First, a simple graph shows no issue at $0$: +Let's return to the function $f(x) = \sin(x)/x$. This function was studied by Euler as part of his solution to the [Basel](http://en.wikipedia.org/wiki/Basel_problem) problem. He knew that near $0$, $\sin(x) \approx x$, so the ratio is close to $1$ if $x$ is near $0$. Hence, the intuition is $\lim_{x \rightarrow 0} \sin(x)/x = 1$, as Cauchy wrote. We can verify this limit graphically two ways. First, a graph along with the points identified to produce it shows no issue at $0$. The $y$ values of @fig-sinx-over-x-minuspi-over-2-to-pi-over-2 seem to go to $1$ as the $x$ values get close to $0$. (That the graph looks defined at $0$ is due to the fact that the points sampled to graph do not include $0$.) +::: {#fig-sinx-over-x-minuspi-over-2-to-pi-over-2} ```{julia} #| hold: true - +#| echo: false f(x) = sin(x)/x plot(f, -pi/2, pi/2; seriestype=[:scatter, :line], # show points and line segments legend=false) ``` -The $y$ values of the graph seem to go to $1$ as the $x$ values get close to $0$. (That the graph looks defined at $0$ is due to the fact that the points sampled to graph do not include $0$.) +Plot of $f(x) = \sin(x)/x$ over $[-\pi/2, \pi2/]$. The graph made by `plot(f, -pi/2, pi/2`)` does not show the issue at $0$ and so suggests a limit of $1$. +::: -We can also verify Euler's intuition through this graph: +We can also verify Euler's intuition through a graph (@fig-plot-sin-and-x-over-minus-pi-over-2-to-pi-over-2). +::: {#fig-plot-sin-and-x-over-minus-pi-over-2-to-pi-over-2} ```{julia} #| hold: true -plot(sin, -pi/2, pi/2) -plot!(identity) # the function y = x, like how zero is y = 0 +plot(sin, -pi/2, pi/2; label="f(x)=sin(x)") +plot!(identity; label="f(x)=x") # the function y = x, like how zero is y = 0 ``` -That the two are indistinguishable near $0$ makes it easy to see that their ratio should be going towards $1$. +Plot of both $f(x) = \sin(x)$ and the line $y=x$ over $[-\pi/2, \pi/2]$ showing there similar behaviour near $0$ +::: + +That the two graphs are indistinguishable near $0$ makes it easy to see that their ratio should be going towards $1$. -A parametric plot shows the same, we see below the slope at $(0,0)$ is *basically* $1$, because the two functions are varying at the same rate when they are each near $0$ - +A parametric plot shows the same, we see below the slope at $(0,0)$ is *basically* $1$, because the two functions are varying at the same rate when they are each near $0$. In @fig-parametric-plot-of-sin-and-y-equal-x the line $y=x$ is added for easy identification of the slope. +::: {#fig-parametric-plot-of-sin-and-y-equal-x} ```{julia} #| hold: true -plot(sin, identity, -pi/2, pi/2) # parametric plot +plot(sin, identity, -pi/2, pi/2; label="sin") # parametric plot +plot!(identity; line=(:dash, :gray25), label="y=x" ) # add y-x line ``` -The graphical approach to limits - plotting $f(x)$ around $c$ and observing if the $y$ values seem to converge to an $L$ value when $x$ get close to $c$ - allows us to gather quickly if a function seems to have a limit at $c$, though the precise value of $L$ may be hard to identify. +Parametric plot of $(\sin(t), t)$ for $t$ in $[-\pi/2, \pi/2]$ along with the line $y=x$. That the two graphs are similar at $(0,0)$ suggests that the limit of the ratio is $1$. +::: + +The graphical approach to limits---plotting $f(x)$ around $c$ and observing if the $y$ values seem to converge to an $L$ value when $x$ get close to $c$---allows us to gather quickly if a function seems to have a limit at $c$, though the precise value of $L$ may be hard to identify. ##### Example -This example illustrates the same limit a different way. Sliding the $x$ value towards $0$ shows $f(x) = \sin(x)/x$ approaches a value of $1$. +This example illustrates the same limit a different way. Sliding the $x$ value towards $0$ in @fig-jsxgraph-sinx-over-x shows $f(x) = \sin(x)/x$ approaches a value of $1$. +::: {#fig-jsxgraph-sinx-over-x} ```{=html}
@@ -450,6 +473,9 @@ txt = b.create('text', [2, 1, function() { }]); ``` +Interactive animation suggesting a limit for $\sin(x)/x$ at $x=0$ +::: + ##### Example @@ -470,16 +496,18 @@ c = 2 f(c) ``` -The `NaN` indicates that this function is indeterminate at $c=2$. A quick plot gives us an idea that the limit exists and is roughly $-0.2$: +The `NaN` indicates that this function is indeterminate at $c=2$. A quick plot (@fig-quick-plot-of-rational-function-over-1-to-3) gives us an idea that the limit exists and is roughly $-0.2$. The graph looks "continuous." In fact, the value $c=2$ is termed a *removable singularity* as redefining $f(x)$ to be $-0.2$ when $x=2$ results in a "continuous" function. +::: {#fig-quick-plot-of-rational-function-over-1-to-3} ```{julia} #| hold: true c, delta = 2, 1 plot(f, c - delta, c + delta) ``` -The graph looks "continuous." In fact, the value $c=2$ is termed a *removable singularity* as redefining $f(x)$ to be $-0.2$ when $x=2$ results in a "continuous" function. +A quick plot of the rational function $f(x)$ over $[2-1, 2+1]$ (centered about $2$) +::: As an aside, we can redefine `f` using the "ternary operator": @@ -490,7 +518,7 @@ As an aside, we can redefine `f` using the "ternary operator": f(x) = x == 2.0 ? -0.2 : (x^2 - 5x + 6) / (x^2 + x - 6) ``` -This particular case is a textbook example: one can easily factor $f(x)$ to get: +This particular case is a textbook example---one can easily factor $f(x)$ to get: $$ @@ -568,7 +596,7 @@ Same story. The numeric evidence supports a limit of $L=0.6$. ::: {.callout-note} ### The `lim` function -The `CalculusWithJulia` package provides a convenience function `lim(f, c)` to create tables to showcase limits. The `dir` keyword can be `"+-"` (the default) to show values from both the left and the right; `"+"` to only show values to the right of `c`; and `"-"` to only show values to the left of `c`: +The `CalculusWithJulia` package provides a convenience function `lim(f, c)` to create tables to showcase limits.^[The `limit` function, by default, only takes *right* limits; to be defined in the next section. The `dir` keyword argument can change this with `dir="+-"` for the two-sided limits discussed here, `dir="-"` for left limits, and the default `dir="+"` for right limits.] For example: @@ -576,11 +604,7 @@ For example: lim(f, c) ``` -The numbers are displayed in decreasing order so the values on the left side of $c$ are read from bottom to top: - -```{julia} -lim(f, c; dir="-") # or even lim(f, c, -) -``` +The numbers evaluated by `f` are displayed in decreasing order so the values on the left side of $c$ are read from bottom to top. ::: @@ -652,17 +676,66 @@ Looking at the bottom of the second column reveals the error. The value of `1 - Not that we needed to. The answer would have been clear if we had stopped with `x=1e-6` (with `n=6`) say. -In general, some functions will frustrate the numeric approach. It is best to be wary of results. At a minimum they should confirm what a quick graph shows, though even that isn't enough, as this next example shows. +In general, some functions will frustrate the numeric approach. It is best to be wary of results. At a minimum they should confirm what a quick graph shows, though even that isn't always enough. + +### Richardson extrapolation + +The [`Richardson`](https://github.com/JuliaMath/Richardson.jl) package can numerically compute limits of a wide class of functions in a way that avoids numeric issues, as above. + + +```{julia} +using Richardson +f(x) = (1 - cos(x)) / x^2 +c, h0 = 0, 1 +extrapolate(f, h0; x0=c) +``` + +The value `h0=1` in this case is just a starting point of the algorithm and is near `x0` but otherwise need not be worried about. (It is like the value in the left column of the first line output by `lim` for a right-hand limit, not the "going to" part.) In the answer the first value is the estimated limit, the second an error that is used to track when the computed values differ from the mathematically expected values. + +Let $f_0(x) = f(x)$. The basic [algorithm](https://en.wikipedia.org/wiki/Richardson_extrapolation) assumes we can express the value for $L$ in a form such as: + + +$$ +L = f_0(h) + a_0 h^{k_0} + a_1 h^{k_1} + a_2 h^{k_2} + \text{error}. +$$ + +Such a form is known as a series expansion; later the Taylor series expansion will be presented. +The error is understood to depend on $h^{k_3}$ in a more precise way than we describe here. But the key is that the $L - f_0(h)$ is not zero and the most important term in how far from zero is $a_0 h^{k_0}$. The trick of Richardson is to evaluate the above at $h/t$ for some $t$ to get another estimate for $L$, then multiply this new estimate by $t^{k_0}$, and then subtract the old estimate to get: + +$$ +\begin{align*} +(t^{k_0} - 1) L &= (t^{k_0} f_0\left(\frac{h}{t}\right) - f_0(h)) \\ +&+ (t^{k_0} a_0 \left(\frac{h}{t}\right)^{k_0} - a_0 h^{k_0}) \\ +&+ (t^{k_0} a_1 \left(\frac{h}{t}\right)^{k_1} - a_1 h^{k_1}) \\ +&+ (t^{k_0} a_2 \left(\frac{h}{t}\right)^{k_2} - a_2 h^{k_2}) + \text{error} +\end{align*} +$$ + +The rationale for this choice is that the second line, with $a_0$, will cancel out leaving a new expression for $L$: + +$$ +L = f_1(h) + \tilde{a}_1 h^{k_1} + \tilde{a}_2 h^{k_2} + \text{error} +$$ + +where + +$$ +f_1(h) = \frac{t^{k_0} f_0\left(\frac{h}{t}\right) - f_0(h)}{t^{k_0} - 1}. +$$ + +This changes the important term in the difference between $L$ and $f_1(h)$ to an expression in $h^{k_1}$, a presumably smaller term. This process can be repeated to produce $f_2$, $f_3$, etc. However, as seen, even if mathematically this can get better an better, it may not be the case computationally. But these differences between the computational and +their mathematical expectation can be evaluated so that the `extrapolate` function can stop its work before issues arise. + ##### Example -Let $h(x)$ be defined by +Some problems are beyond a numeric approach. Let $h(x)$ be defined by $$ -h(x) = x^2 + 1 + \log(| 11 \cdot x - 15 |)/99. +h(x) = x^2 + 1 + \log(\lvert 11 \cdot x - 15 \rvert)/99. $$ The question is to investigate @@ -672,14 +745,17 @@ $$ \lim_{x \rightarrow 15/11} h(x) $$ -A plot shows the answer appears to be straightforward: - +The plot in @fig-k-obryants-sneaky-function shows the answer appears to be straightforward. +::: {#fig-k-obryants-sneaky-function} ```{julia} h(x) = x^2 + 1 + log(abs(11*x - 15))/99 plot(h, 15/11 - 1, 15/11 + 1) ``` +Plot of a function with the property that it has a vertical asymptote with left and right limits of $-\infty$ and yet the function is never negative when evaluated at a 64-bit floating point number +::: + Taking values near $15/11$ shows nothing perhaps too unusual: @@ -692,64 +768,7 @@ lim(h, c; n = 16) (Though the graph and table do hint at something a bit odd---the graph shows a blip, the table doesn't show values in the second column going towards a specific value.) -However the limit in this case is $-\infty$ (or DNE), as there is an aysmptote at $c=15/11$. The problem is the asymptote due to the logarithm is extremely narrow and happens between floating point values to the left and right of $15/11$. - - -### Richardson extrapolation - - -The [`Richardson`](https://github.com/JuliaMath/Richardson.jl) package provides a function to extrapolate a function `f(x)` to `f(x0)`, as the numeric limit does. We illustrate its use by example: - - -```{julia} -#| hold: true -f(x) = sin(x)/x -extrapolate(f, 1) -``` - -The answer involves two terms, the second being an estimate for the error in the estimation of `f(0)`. - - -The values the method chooses could be viewed as follows: - - -```{julia} -#| term: true -extrapolate(1) do x # using `do` notation for the function - @show x - sin(x)/x -end -``` - -The `extrapolate` function avoids the numeric problems encountered in the following example - - -```{julia} -#| hold: true -f(x) = (1 - cos(x)) / x^2 -extrapolate(f, 1) -``` - -To find limits at a value of `c` not equal to `0`, we set the `x0` argument. For example, - - -```{julia} -#| hold: true -f(x) = (sqrt(x) - 5) / (sqrt(x-16) - 3) -c = 25 -extrapolate(f, 1, x0=25) -``` - -This value can also be `Inf`, in anticipation of infinite limits to be discussed in a subsequent section: - - -```{julia} -#| hold: true -f(x) = (x^2 - 2x + 1)/(x^3 - 3x^2 + 2x + 1) -extrapolate(f, 10, x0=Inf) -``` - -(The starting value should be to the right of any zeros of the denominator.) +However the limit in this case is $-\infty$ (or DNE), as there is an asymptote at $c=15/11$. The problem is the asymptote due to the logarithm is extremely narrow and happens between floating point values to the left and right of $15/11$. ## Symbolic approach to limits @@ -766,17 +785,17 @@ For example, the limit at $0$ of $(1-\cos(x))/x^2$ is easily handled: limit((1 - cos(x)) / x^2, x => 0) ``` -The pair notation (`x => 0`) is used to indicate the variable and the value it is going to. A `dir` argument is used to indicate $x \rightarrow c+$ (the default, or `dir="+"`), $x \rightarrow c-$ (`dir="-"`), and $x \rightarrow c$ (`dir="+-"`). +The pair notation (`x => 0`) is used to indicate the variable and the value it is going to. In the next section we introduce the `dir` keyword argument. ##### Example -We look again at this function which despite having a vertical asymptote at $x=15/11$ has the property that it is positive for all floating point values, making both a numeric and graphical approach impossible: +We look again at the function which despite having a vertical asymptote at $x=15/11$ has the property that it is positive for all floating point values, making both a numeric and graphical approach impossible: $$ -h(x) = x^2 + 1 + \log(| 11 \cdot x - 15 |)/99. +h(x) = x^2 + 1 + \log(\lvert 11 \cdot x - 15 \rvert)/99. $$ We find the limit symbolically at $c=15/11$ as follows, taking care to use the exact value `15//11` and not the *floating point* approximation returned by `15/11`: @@ -800,14 +819,15 @@ $$ \lim_{\rho \rightarrow 1} \frac{x^{1-\rho} - 1}{1 - \rho}. $$ -We have for the first: +We have for the first:^[This and the next two `limit` calls should have `dir="+-"` specified, as the default of SymPy is to only compute a limit from the right side of `c`.] + ```{julia} -limit( (2sin(x) - sin(2x)) / (x - sin(x)), x => 0; dir="+-") +limit( (2sin(x) - sin(2x)) / (x - sin(x)), x => 0) ``` -(The `dir = "+-"` indicates take both a right and left limit and ensure both exist and are equal.) + The second is similarly done, though here we define a function for variety: @@ -815,7 +835,7 @@ The second is similarly done, though here we define a function for variety: ```{julia} #| hold: true f(x) = (exp(x) - 1 - x) / x^2 -limit(f(x), x => 0; dir="+-") +limit(f(x), x => 0) ``` Finally, for the third we define a new variable and proceed: @@ -823,7 +843,7 @@ Finally, for the third we define a new variable and proceed: ```{julia} @syms rho::real -limit( (x^(1-rho) - 1) / (1 - rho), rho => 1; dir="+-") +limit( (x^(1-rho) - 1) / (1 - rho), rho => 1) ``` This last limit demonstrates that the `limit` function of `SymPy` can readily evaluate limits that involve parameters, though at times some assumptions on the parameters may be needed, as was done through `rho::real`. @@ -856,13 +876,18 @@ The value is not `NaN`, rather `Inf`. This is because `cos(pi/2)` is not exactly limit(j(x), x => PI/2) ``` -The value is not right, as this simple graph suggests the limit is in fact $-1$: +The value is not right, as this simple graph (@fig-simple-graph-cosx-over-x-minus-pi-over-2) suggests the limit is in fact $-1$: +::: {#fig-simple-graph-cosx-over-x-minus-pi-over-2} ```{julia} +#| echo: false plot(j, pi/4, 3pi/4) ``` +Plot of $f(x) = \cos(x) / (x - \pi/2)$ over $[-\pi/4, 3\pi/4]$ +::: + The difference between `pi` and `PI` can be significant, and though usually `pi` is silently converted to `PI`, it doesn't happen here as the division by `2` happens first, which turns the symbol into an approximate floating point number. Hence, `SymPy` is giving the correct answer for the problem it is given, it just isn't the problem we wanted to look at. @@ -871,34 +896,20 @@ Trying again, being more aware of how `pi` and `PI` differ, we have: ```{julia} #| hold: true -f(x) = cos(x) / (x - PI/2) -limit(f(x), x => PI/2) +j(x) = cos(x) / (x - PI/2) +limit(j(x), x => PI/2) ``` -(The value `pi` is able to be exactly converted to `PI` when used in `SymPy`, as it is of type `Irrational`, and is not a floating point value. However, the expression `pi/2` converts `pi` to a floating point value and then divides by `2`, hence the loss of exactness when used symbolically.) - - -##### Example: left and right limits - - -Right and left limits will be discussed in the next section; here we give an example of the idea. The mathematical convention is to say a limit exists if both the left *and* right limits exist and are equal. Informally a right (left) limit at $c$ only considers values of $x$ more (less) than $c$. The `limit` function of `SymPy` finds directional limits by default, a right limit, where $x > c$. - - -The left limit can be found by passing the argument `dir="-"`. Passing `dir="+-"` (and not `"-+"`), as done in a few examples above, will compute the mathematical limit, throwing an error in `Python` if no limit exists. - - -```{julia} -limit(ceil(x), x => 0), limit(ceil(x), x => 0, dir="-") -``` - -This accurately shows the limit does not exist mathematically, but `limit(ceil(x), x => 0)` does exist (as it finds a right limit). ## Rules for limits -The `limit` function doesn't compute limits from the definition, rather it applies some known facts about functions within a set of rules. Some of these rules are the following. Suppose the individual limits of $f$ and $g$ always exist (and are finite) below. +The `limit` function doesn't compute limits from the definition, rather it applies some known facts about functions within a set of rules. Some of these rules are the following. +::: {.relationship title="Rules for limits"} + +Suppose the individual limits of $f$ and $g$ exist (and are finite) on the right-hand sides below. Then: $$ \begin{align*} @@ -912,43 +923,55 @@ $$ %% \lim_{x \rightarrow c} \frac{f(x)}{g(x)} &= \frac{\lim_{x \rightarrow c} f(x)}{\lim_{x \rightarrow c} g(x)} - &(\text{provided }\lim_{x \rightarrow c} g(x) \neq 0)\\ + \quad\quad(\text{provided }\lim_{x \rightarrow c} g(x) \neq 0)\\ \end{align*} $$ +::: -These are verbally described as follows, when the individual limits exist and are finite then: +These rules are verbally described as follows, when the individual limits exist and are finite then: - * Limits involving sums, differences or scalar multiples of functions *exist* **and** can be **computed** by first doing the individual limits and then combining the answers appropriately. - * Limits of products exist and can be found by computing the limits of the individual factors and then combining. - * Limits of ratios *exist* and can be found by computing the limit of the individual terms and then dividing **provided** you don't divide by $0$. The last part is really important, as this rule is no help with the common indeterminate form $0/0$. +* Limits involving sums, differences or scalar multiples of functions *exist* *and* can be *computed* by first doing the individual limits and then combining the answers appropriately. + +* Limits of products exist and can be found by computing the limits of the individual factors and then combining. + +* Limits of ratios *exist* and can be found by computing the limit of the individual terms and then dividing **provided** you don't divide by $0$. The last part is really important, as this rule is no help with the common indeterminate form $0/0$. -In addition, consider the composition: +In addition + +::: {.relationship title="Limit rule for a composition"} +Consider the limit of a composition: $$ -\lim_{x \rightarrow c} f(g(x)) +\lim_{x \rightarrow c} f(g(x)). $$ Suppose that - * The outer limit, $\lim_{x \rightarrow b} f(x) = L$, exists, and - * the inner limit, $\lim_{x \rightarrow c} g(x) = b$, exists **and** - * for some neighborhood around $c$ (not including $c$) $g(x)$ is not $b$, +* The outer limit, $\lim_{x \rightarrow b} f(x) = L$, exists, and + +* the inner limit, $\lim_{x \rightarrow c} g(x) = b$, exists, **and** + +* for some neighborhood around $c$ (not including $c$) $g(x)$ is not $b$. Then the limit exists and equals $L$: -$\lim_{x \rightarrow c} f(g(x)) = \lim_{u \rightarrow b} f(u) = L.$ +$$ +\lim_{x \rightarrow c} f(g(x)) = \lim_{u \rightarrow b} f(u) = L. +$$ +::: + An alternative, is to assume $f(x)$ is defined at $b$ and equal to $L$ (which is the definition of continuity), but that isn't the assumption above, hence the need to exclude $g$ from taking on a value of $b$ (where $f$ may not be defined) near $c$. -These rules, together with the fact that our basic algebraic functions have limits that can be found by simple evaluation, mean that many limits are easy to compute. +These limit rules, together with the fact that our basic algebraic functions have limits that can be found by simple evaluation, mean that many limits are easy to compute. ##### Example: composition @@ -961,7 +984,7 @@ $$ \lim_{x \rightarrow 0} \frac{\sin(kx)}{x}. $$ -This is clearly related to the function $f(x) = \sin(x)/x$, which has a limit of $1$ as $x \rightarrow 0$. We see $g(x) = k f(kx)$ is the limit in question. As $kx \rightarrow 0$, though not taking a value of $0$ except when $x=0$, the limit above is $k \lim_{x \rightarrow 0} f(kx) = k \lim_{u \rightarrow 0} f(u) = k$. +This is clearly related to the function $f(x) = \sin(x)/x$, which has a limit of $1$ as $x \rightarrow 0$. We see $g(x) = k f(kx)$ is the function in the limit of the question. As $kx \rightarrow 0$, though not taking a value of $0$ except when $x=0$, the limit above is $k \lim_{x \rightarrow 0} f(kx) = k \lim_{u \rightarrow 0} f(u) = k$. Basically when taking a limit as $x$ goes to $0$ we can multiply $x$ by any constant and figure out the limit for that. (It is as though we "go to" $0$ faster or slower, but are still going to $0$.) @@ -980,7 +1003,7 @@ as this is the limit of $f(g(x))$ with $f$ as above and $g(x) = x^2$. We need $ ##### Example: products -Consider this more complicated limit found on this [Wikipedia](http://en.wikipedia.org/wiki/L%27H%C3%B4pital%27s_rule) page. +Consider this more complicated limit found on this [Wikipedia](http://en.wikipedia.org/wiki/L%27H%C3%B4pital%27s_rule) page: $$ @@ -1007,29 +1030,40 @@ limit(sin(PI*x)/(PI*x) * l(x), x => 1//2) Consider again the limit of $\cos(\pi x) / (1 - (2x)^2)$ at $c=1/2$. A graph of both the top and bottom functions shows the indeterminate, $0/0$, form: - +::: {#fig-plot-cos-pix-and-1-minus-2x-squared} ```{julia} +#| echo: false plot(cos(pi*x), 0.4, 0.6) plot!(1 - (2x)^2) ``` -However, following Euler's insight that $\sin(x)/x$ will have a limit at $0$ of $1$ as $\sin(x) \approx x$, and $x/x$ has a limit of $1$ at $c=0$, we can see that $\cos(\pi x)$ looks like $-\pi\cdot (x - 1/2)$ and $(1 - (2x)^2)$ looks like $-4(x-1/2)$ around $x=1/2$: +Plot of $f(x) = \cos(\pi x)$ and $g(x) = 1 - (2x)^2$ over $[0.5 - 0.1, 0.5 + 0.1]$. The plot shows at $x=0.5$ the ratio will be indeterminate. +::: +However, following Euler's insight that $\sin(x)/x$ will have a limit at $0$ of $1$ as $\sin(x) \approx x$, and $x/x$ has a limit of $1$ at $c=0$, in @fig-plot-cos-pi-x-tangent-line-and-1-minus-2x-squared-and-tangent-line we can see that $\cos(\pi x)$ looks like $-\pi\cdot (x - 1/2)$ and $(1 - (2x)^2)$ looks like $-4(x-1/2)$ around $x=1/2$. These lines are "tangent lines". + + +::: {#fig-plot-cos-pi-x-tangent-line-and-1-minus-2x-squared-and-tangent-line} ```{julia} -plot(cos(pi*x), 0.4, 0.6) -plot!(-pi*(x - 1/2)) +#| echo: false + +p1 = plot(cos(pi*x), 0.4, 0.6; label = "cos(pi*x)") +plot!(p1, -pi*(x - 1/2); label="tangent line") + +p2 = plot(1 - (2x)^2, 0.4, 0.6; label="1 - (2x)^2") +plot!(p2, -4(x - 1/2); label="tangent line") + +plot(p1, p2) ``` -```{julia} -plot(1 - (2x)^2, 0.4, 0.6) -plot!(-4(x - 1/2)) -``` +Plot of $\cos(\pi x)$ and its tangent line and a plot of $1 - (2x)^2$ and its tangent line over $[0.5 - 0.1, 0.5 + 0.1]$ +::: -So around $c=1/2$ the ratio should look like $-\pi (x-1/2) / ( -4(x - 1/2)) = \pi/4$, which indeed it does, as that is the limit. +So around $c=1/2$ the ratio of the functions should look like the ratio of the lines or $-\pi (x-1/2) / ( -4(x -1/2)) = \pi/4$, which indeed it does, as that is the limit. -This is the basis of L'Hôpital's rule, which we will return to once the derivative is discussed. +This is the basis of L'Hospital's rule, which we will return to once the derivative is discussed so that a tangent line can be mathematically described. ##### Example: sums @@ -1054,10 +1088,10 @@ Why? We can express the function $e^{\csc(x)}/e^{\cot(x)}$ as the above function ### The squeeze theorem -Sometimes limits can be found by bounding more complicated functions by easier functions. +Sometimes limits can be found by bounding more complicated functions by easier to reason about functions. + +::: {.theorem title="Theorem: squeeze theorem"} -::: {.callout-note icon=false} -## The [squeeze theorem](http://en.wikipedia.org/wiki/Squeeze_theorem) Fix $c$ in $I=(a,b)$. Suppose for all $x$ in $I$, except possibly $c$, there are two functions $l$ and $u$, satisfying: @@ -1077,10 +1111,12 @@ $$ \lim_{x\rightarrow c} f(x) = L. $$ +The functions $l$ and $u$ [squeeze](http://en.wikipedia.org/wiki/Squeeze_theorem)) $f$. ::: -The figure shows a usage of the squeeze theorem to show $\sin(x)/x \rightarrow 1$ as $\cos(x) \leq \sin(x)x \leq 1$ for $x$ close to $0$. +The figure shows a usage of the squeeze theorem to show $\sin(x)/x \rightarrow 1$ as $\cos(x) \leq \sin(x)/x \leq 1$ for $x$ close to $0$. +::: {#fig-animation-for-square-theorem-cos-sinx-over-x} ```{julia} #| hold: true #| echo: false @@ -1102,13 +1138,14 @@ imgfile = tempname() * ".gif" gif(anim, imgfile, fps = 1) -caption = """ -As ``x`` goes to ``0``, the values of ``\\sin(x)/x`` are squeezed between ``\\cos(x)`` and ``1`` which both converge to ``1``. -""" +caption = "" plotly() ImageFile(imgfile, caption) ``` +As $x$ goes to $0$, the values of $\sin(x)/x$ are squeezed between the values of $\cos(x)$ and $1$, and both functions converge to $1$. +::: + ## Limits from the definition @@ -1178,7 +1215,7 @@ plotly() nothing ``` -Figure illustrating requirements of $\epsilon-\delta$ definition of the limit. The image (shaded red on $y$ axis) of the $x$ within $\delta$ of $c$ (except for $c$ and shaded blue on the $x$ axis) must stay within the bounds of $L-\epsilon$ and $L+ \epsilon$, where $\delta$ may be chosen based on $\epsilon$ but needs to be chosen for every positive $\epsilon$, not just a fixed one as in this figure. +Figure illustrating requirements of $\epsilon-\delta$ definition of the limit. The image (shaded red on $y$ axis) of the $x$ within $\delta$ of $c$ (except for $c$ and shaded blue on the $x$ axis) must stay within the bounds of $L-\epsilon$ and $L+ \epsilon$, where $\delta$ may be chosen based on $\epsilon$ but needs to be chosen for every positive $\epsilon$, not just a fixed one. ::: A simple case is the linear case. Consider the function $f(x) = 3x + 2$. Verify that the limit at $c=1$ is $5$. @@ -1191,17 +1228,14 @@ We show "numerically" that $\delta = \epsilon/3$. #| hold: true f(x) = 3x + 2 c, L = 1, 5 -epsilon = rand() # some number in (0,1) +epsilon = rand() # some number in (0,1) delta = epsilon / 3 xs = c .+ delta * rand(100) # 100 numbers, c < x < c + delta as = [abs(f(x) - L) < epsilon for x in xs] -all(as) # are all the as true? +all(as) # are all the as true? ``` -These lines produce a random $\epsilon$, the resulting $\delta$, and then verify for 100 numbers within $(c, c+\delta)$ that the inequality $\lvert f(x) - L \rvert < \epsilon$ holds for each. Running them again and again should always produce `true` if $L$ is the limit and $\delta$ is chosen properly. - - -(Of course, we should also verify values to the left of $c$.) +These lines produce a random $\epsilon$, the resulting $\delta$, and then verify for 100 numbers within $(c, c+\delta)$ that the inequality $\lvert f(x) - L \rvert < \epsilon$ holds for each. Running them again and again should always produce `true` if $L$ is the limit and $\delta$ is chosen properly. Of course, we should also verify values to the left of $c$. (The random numbers are technically in $[0,1)$, so in theory `epsilon` could be `0`. So the above approach would be more solid if some guard, such as `epsilon = max(eps(), rand())`, was used. As the formal definition is the domain of paper-and-pencil, we don't fuss.) @@ -1286,13 +1320,14 @@ With this result, the rules of limits can immediately extend this to any polynom ###### Question -From the graph, find the limit: +From the graph in @fig-limit-rational-function-x-squared-minus-3x-plus-2-over-x-squared-minus-6x-plua-5, find the limit: $$ L = \lim_{x\rightarrow 1} \frac{x^2−3x+2}{x^2−6x+5} $$ +::: {#fig-limit-rational-function-x-squared-minus-3x-plus-2-over-x-squared-minus-6x-plua-5} ```{julia} #| hold: true #| echo: false @@ -1300,11 +1335,14 @@ f(x) = (x^2 - 3x +2) / (x^2 - 6x + 5) plot(f, 0,2) ``` +Plot of $f(x)$ over $[1-1, 1 + 1]$. +::: + ```{julia} #| hold: true #| echo: false -answ = 1/4 -numericq(answ, 1e-1) +answer = 1/4 +numericq(answer, 1e-1) ``` ###### Question @@ -1317,6 +1355,7 @@ $$ L = \lim_{x \rightarrow -2} \frac{x}{x+1} \frac{x^2}{x^2 + 4} $$ +::: {#fig-plot-x-over-x-plus-1-times-x-squared-over-x-squared-plus-4} ```{julia} #| hold: true #| echo: false @@ -1324,6 +1363,9 @@ f(x) = x/(x+1)*x^2/(x^2+4) plot(f, -3, -1.25) ``` +Plot of $f(x)$ over $[-3, -1.25]$ +::: + ```{julia} #| hold: true #| echo: false @@ -1335,22 +1377,24 @@ numericq(val, 1e-1) ###### Question -Graphically investigate the limit +Graphically investigate the limit of $$ L = \lim_{x \rightarrow 0} \frac{e^x - 1}{x}. $$ -What is the value of $L$? - +using @fig-exp-x-minus-1-over-x. What is the value of $L$? +::: {#fig-exp-x-minus-1-over-x} ```{julia} #| hold: true #| echo: false f(x) = (exp(x) - 1)/x p = plot(f, -1, 1) ``` +Plot of $f(x) = (e^x - 1)/x$ over $[-1,1]$ +::: ```{julia} #| hold: true @@ -1382,9 +1426,9 @@ numericq(val, 1e-2) ###### Question -Select the graph for which there is no limit at $a$. - +In @fig-which-graph-has-no-limit-at-a select the graph for which there is no limit at $a$. +::: {#fig-which-graph-has-no-limit-at-a} ```{julia} #| hold: true #| echo: false @@ -1423,6 +1467,9 @@ let end ``` +Select the graph with no limit at $a$ +::: + ```{julia} #| echo: false plotly(); @@ -1452,8 +1499,8 @@ What is $L$? #| hold: true #| echo: false choices = ["``0``", "``1``", "``e^x``"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` ###### Question @@ -1480,8 +1527,8 @@ Using the last result, what is the value of $L$? #| hold: true #| echo: false choices = ["``\\cos(x)``", "``\\sin(x)``", "``1``", "``0``", "``\\sin(h)/h``"] -answ = 1 -radioq(choices, answ) +answer = 1 +buttonq(choices, answer) ``` ###### Question @@ -1565,8 +1612,8 @@ The limit of $\sin(x)/x$ at $0$ has a numeric value. This depends upon the fact #| hold: true #| echo: false choices = [q"0", q"1", q"pi/180", q"180/pi"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` What is the limit `limit(sinpi(x)/x, x => 0)`? @@ -1576,8 +1623,8 @@ What is the limit `limit(sinpi(x)/x, x => 0)`? #| hold: true #| echo: false choices = [q"0", q"1", q"pi", q"1/pi"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` ###### Question: limit properties @@ -1656,8 +1703,8 @@ choices = [ "Yes, the value is `-11.5123`", "No, the value heads to negative infinity" ]; -answ = 3; -radioq(choices, answ) +answer = 3; +buttonq(choices, answer) ``` ###### Question @@ -1752,8 +1799,8 @@ What is `limit(ex, x => 0)`? #| hold: true #| echo: false choices = ["``e^{km}``", "``e^{k/m}``", "``k/m``", "``m/k``", "``0``"] -answwer = 1 -radioq(choices, answwer) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1866,31 +1913,37 @@ buttonq(choices, 2; explanation=explanation) ###### Question: The squeeze theorem -Let's look at the function $f(x) = x \sin(1/x)$. A graph around $0$ can be made with: - +Let's look at the function $f(x) = x \sin(1/x)$. @fig-squeeze-theorem-x-sin-1-over-x-and-abs-x shows the graph of the function near $0$ and also the graph of $\lvert x \rvert$ and $-\lvert x \rvert$. +::: {#fig-squeeze-theorem-x-sin-1-over-x-and-abs-x} ```{julia} #| hold: true +#| echo: false f(x) = x == 0 ? NaN : x * sin(1/x) c, delta = 0, 1/4 -plot(f, c - delta, c + delta) -plot!(abs) -plot!(x -> -abs(x)) +plot(f, c - delta, c + delta; legend=false, line=(:blue,)) +plot!(abs; line=(:black,)) +plot!(x -> -abs(x); line=(:black,)) ``` -This graph clearly oscillates near $0$. To the graph of $f$, we added graphs of both $g(x) = \lvert x\rvert$ and $h(x) = - \lvert x\rvert$. From this graph it is easy to see by the "squeeze theorem" that the limit at $x=0$ is $0$. Why? +Plot of $f(x) = x \sin(1/x)$ over $[-1/4, 1/4]$ along with $\lvert x\rvert$ and $-\lvert x \rvert$. +::: +This graph clearly oscillates near $0$. To the graph of $f$, we added graphs of both +$g(x) = \lvert x\rvert$ and $h(x) = - \lvert x\rvert$. +From the graph in @fig-squeeze-theorem-x-sin-1-over-x-and-abs-x it is easy to see by the "squeeze theorem" that the limit at $x=0$ is $0$. Why? + ```{julia} #| hold: true #| echo: false choices=[L"""The functions $g$ and $h$ both have a limit of $0$ at $x=0$ and the function $f$ is in between both $g$ and $h$, so must to have a limit of $0$. """, - L"The functions $g$ and $h$ squeeze each other as $g(x) > h(x)$", + L"The functions $g$ and $h$ squeeze each other as $g(x) > h(x)$", L"The function $f$ has no limit - it oscillates too much near $0$"] -answ = 1 -radioq(choices, answ) +answer = 1 +buttonq(choices, answer) ``` (The [Wikipedia](https://en.wikipedia.org/wiki/Squeeze_theorem) entry for the squeeze theorem has this unverified, but colorful detail: diff --git a/quarto/limits/limits_extensions.qmd b/quarto/limits/limits_extensions.qmd index 045640c..d70025b 100644 --- a/quarto/limits/limits_extensions.qmd +++ b/quarto/limits/limits_extensions.qmd @@ -13,17 +13,14 @@ plotly() using SymPy ``` -```{julia} -#| echo: false -#| results: "hidden" -using DataFrames -nothing -``` --- -![To infinity and beyond](figures/buzz-infinity.jpg){width=40%} +::: {#fig-buzz-lightyear-likely-not-AI-generated} +![](figures/buzz-infinity.jpg){width=40%} +To infinity and beyond +::: The limit of a function at $c$ need not exist for one of many different reasons. Some of these reasons can be handled with extensions to the concept of the limit, others are just problematic in terms of limits. This section covers examples of each. @@ -50,7 +47,7 @@ plot(f, range(-1, stop=1, length=1000)) Graph of the function $f(x) = \sin(1/x)$ near $0$. It oscillates infinitely many times around $0$. ::: -The graph oscillates between $-1$ and $1$ infinitely many times on this interval - so many times, that no matter how close one zooms in, the graph on the screen will fail to capture them all. Graphically, there is no single value of $L$ that the function gets close to, as it varies between all the values in $[-1,1]$ as $x$ gets close to $0$. A simple proof that there is no limit, is to take any $\epsilon$ less than $1$, then with any $\delta > 0$, there are infinitely many $x$ values where $f(x)=1$ and infinitely many where $f(x) = -1$. That is, there is no $L$ with $|f(x) - L| < \epsilon$ when $\epsilon$ is less than $1$ for all $x$ near $0$. +The graph oscillates between $-1$ and $1$ infinitely many times on this interval---so many times, that no matter how close one zooms in, the graph on the screen will fail to capture them all. Graphically, there is no single value of $L$ that the function gets close to, as it varies between all the values in $[-1,1]$ as $x$ gets close to $0$. A simple proof that there is no limit, is to take any $\epsilon$ less than $1$, then with any $\delta > 0$, there are infinitely many $x$ values where $f(x)=1$ and infinitely many where $f(x) = -1$. That is, there is no $L$ with $|f(x) - L| < \epsilon$ when $\epsilon$ is less than $1$ for all $x$ near $0$. This function basically has too many values it gets close to. Another favorite example of such a function is the function that is $0$ if $x$ is rational and $1$ if not. This function will have no limit anywhere, not just at $0$, and for basically the same reason as above. @@ -63,17 +60,21 @@ $$ -|x| \leq x \sin(1/x) \leq |x|. $$ -The following figure illustrates: +@fig-squeze-x-sin-1-over-x-minus1-1 illustrates: +::: {#fig-squeze-x-sin-1-over-x-minus1-1} ```{julia} +#| echo: false f(x) = x * sin(1/x) plot(f, -1, 1; label="f") plot!(abs; label="|.|") plot!(x -> -abs(x); label="-|.|") ``` +Plot of $f(x) = x \sin(1/x)$ over $[-1,1]$ with graphs of $\lvert x\rvert$ and $-\lvert x \rvert$ added +::: -The [squeeze](http://en.wikipedia.org/wiki/Squeeze_theorem) theorem of calculus is the formal reason $f$ has a limit at $0$, as both the upper function, $|x|$, and the lower function, $-|x|$, have a limit of $0$ at $0$. +As seen in the last section, the [squeeze](http://en.wikipedia.org/wiki/Squeeze_theorem) theorem of calculus is the formal reason $f$ has a limit at $0$, as both the upper function, $|x|$, and the lower function, $-|x|$, have a limit of $0$ at $0$. ## Right and left limits @@ -82,27 +83,29 @@ The [squeeze](http://en.wikipedia.org/wiki/Squeeze_theorem) theorem of calculus Another example where $f(x)$ has no limit is the function $f(x) = x /|x|, x \neq 0$. This function is $-1$ for negative $x$ and $1$ for positive $x$. Again, this function will have a limit everywhere except possibly at $x=0$, where division by $0$ is possible. -It's graph is - +::: {#fig-plot-abs-x-over-x-minus2-2} ```{julia} -#| hold: true +#| echo: false f(x) = abs(x)/x plot(f, -2, 2) ``` -The sharp jump at $0$ is misleading - again, the plotting algorithm just connects the points, it doesn't handle what is a fundamental discontinuity well - the function is not defined at $0$ and jumps from $-1$ to $1$ there. Similarly to our example of $\sin(1/x)$, near $0$ the function get's close to both $1$ and $-1$, so will have no limit. (Again, just take $\epsilon$ smaller than $1$.) +Plot of $f(x) = \lvert x \rvert / x$ over $[-2,2]$ +::: + +The sharp jump in @fig-plot-abs-x-over-x-minus2-2 at $0$ is misleading---again, the plotting algorithm just connects the points, it doesn't handle what is a fundamental discontinuity well---the function is not defined at $0$ and jumps from $-1$ to $1$ there. Similarly to our example of $\sin(1/x)$, near $0$ the function get's close to both $1$ and $-1$, so will have no limit. (Again, just take $\epsilon$ smaller than $1$.) -But unlike the previous example, this function *would* have a limit if the definition didn't consider values of $x$ on both sides of $c$. The limit on the right side would be $1$, the limit on the left side would be $-1$. This distinction is useful, so there is an extension of the idea of a limit to *one-sided limits*. +But unlike the previous example of $\sin(1/x)$, this function *would* have a limit if the definition didn't consider values of $x$ on both sides of $c$. The limit on the right side would be $1$, the limit on the left side would be $-1$. This distinction is useful, so there is an extension of the idea of a limit to *one-sided limits*. -Let's loosen up the language in the definition of a limit to read: +Let's loosen up the language in the definition of a limit: -::: {.callout-note icon=false} +::: {.definition title="Loosened definition of a limit"} -The limit of $f(x)$ as $x$ approaches $c$ is $L$ if for every neighborhood, $V$, of $L$ there is a neighborhood, $U$, of $c$ for which $f(x)$ is in $V$ for every $x$ in $U$, except possibly $x=c$. +The limit of $f(x)$ as $x$ approaches $c$ is $L$ if for every *neighborhood*, $V$, of $L$ there is a *neighborhood*, $U$, of $c$ for which $f(x)$ is in $V$ for every $x$ in $U$, except possibly $x=c$. ::: @@ -112,18 +115,16 @@ The $\epsilon-\delta$ definition has $V = (L-\epsilon, L + \epsilon)$ and $U=(c- Now for the definition: -::: {.callout-note icon=false} -## The $\epsilon-\delta$ Definition of a right limit +::: {.theorem title="The epsilon-delta definition of a right (and left) limit"} -A function $f(x)$ has a limit on the right of $c$, written $\lim_{x \rightarrow c+}f(x) = L$ if for every $\epsilon > 0$, there exists a $\delta > 0$ such that whenever $0 < x - c < \delta$ it holds that $|f(x) - L| < \epsilon$. That is, $U$ is $(c, c+\delta)$ +A function $f(x)$ has a limit on the *right* of $c$, written $\lim_{x \rightarrow c+}f(x) = L$ if for every $\epsilon > 0$, there exists a $\delta > 0$ such that whenever $0 < x - c < \delta$ it holds that $|f(x) - L| < \epsilon$. That is, $U$ is $(c, c+\delta)$ Similarly, a limit on the left is defined where $U=(c-\delta, c)$. + +A (two-sided) *limit* exists if and only if both the left- and right-hand limits exist *and* are equal. ::: - - - The `SymPy` function `limit` has a keyword argument `dir="+"` or `dir="-"` to request that a one-sided limit be formed. The default is `dir="+"`. Passing `dir="+-"` will compute both one side limits, and throw an error if the two are not equal, in agreement with no limit existing. @@ -139,20 +140,23 @@ limit(f(x), x=>0, dir="+"), limit(f(x), x=>0, dir="-") :::{.callout-warning} ## Warning -That means the mathematical limit need not exist when `SymPy`'s `limit` returns an answer, as `SymPy` is only carrying out a one sided limit. Explicitly passing `dir="+-"` or checking that both `limit(ex, x=>c)` and `limit(ex, x=>c, dir="-")` are equal would be needed to confirm a limit exists mathematically. +That means the mathematical limit need not exist when `SymPy`'s `limit` returns an answer, as `SymPy` is only carrying out a one-sided limit. Explicitly passing `dir="+-"` or checking that both `limit(ex, x=>c)` and `limit(ex, x=>c, dir="-")` exist and are equal would be needed to confirm a limit exists mathematically. ::: The relation between the two concepts is that a function has a limit at $c$ if and only if the left and right limits exist and are equal. This function $f$ has both existing, but the two limits are not equal. -There are other such functions that jump. Another useful one is the floor function, which just rounds down to the nearest integer. A graph shows the basic shape: - +There are other such functions that jump. Another useful one is the floor function, which just rounds down to the nearest integer. @fig-floor-function-over-minus5-5-without-care shows the basic shape: +::: {#fig-floor-function-over-minus5-5-without-care} ```{julia} +#| echo: false plot(floor, -5,5) ``` +Plot of `floor` over $[-5,5]$. The vertical lines are artifacts of plotting. +::: Again, the (nearly) vertical lines are an artifact of the graphing algorithm and not actual points that solve $y=f(x)$. The floor function has limits except at the integers. There the left and right limits differ. @@ -176,9 +180,9 @@ However, not all such functions with indeterminate forms of $0^0$ will have a li ##### Example -Consider this funny graph: - +Consider the funny graph in @fig-funny-shape-plot-for-limit-questions. +::: {#fig-funny-shape-plot-for-limit-questions} ```{julia} #| hold: true #| echo: false @@ -188,10 +192,10 @@ plot(; legend=false, aspect_ratio=true, xticks = -4:4) plot!([(-4, -1.5),(-2,4)]; line=(:black,1)) plot!(x->x^2, -2, -1; line=(:black,1)) -plot!(exp, -1,0) -plot!(x -> 1-2x, 0, 1) -plot!(sqrt, 1, 2) -plot!(x -> 1-x, 2,3) +plot!(exp, -1,0; line=(:black,1)) +plot!(x -> 1-2x, 0, 1; line=(:black,1)) +plot!(sqrt, 1, 2; line=(:black,1)) +plot!(x -> 1-x, 2, 3; line=(:black,1)) S = Plots.scale(Shape(:circle), 0.05) plot!(Plots.translate(S, -4, -1.5); fill=(:black,)) @@ -205,15 +209,18 @@ S = Plots.scale(Shape(:circle), 0.05) end ``` +Funny graph where limits are not always obvious from continuity +::: + Describe the limits at $-1$, $0$, and $1$. -* At $-1$ we see a jump, there is no limit but instead a left limit of 1 and a right limit appearing to be $1/2$. +* At $-1$ we see a jump, there is no limit but instead a left limit of 1 and a right limit of $1/2$, or so. * At $0$ we see a limit of $1$. -* Finally, at $1$ again there is a jump, so no limit. Instead the left limit is about $-1$ and the right limit $1$. +* Finally, at $1$ again there is a jump, so no limit. Instead the left limit is $-1$ and the right limit $1$. ## Limits at infinity @@ -238,13 +245,13 @@ The function $f(x) = \sin(x)$ will not have a limit at $+\infty$ for exactly the limit(sin(x), x => oo) ``` -(We used `SymPy`'s `oo` for $\infty$ and not `Inf`.) +(We use `SymPy`'s variable `oo` for $\infty$ and not `Inf`, though `Inf` can be used.) --- -However, a damped oscillation, such as $f(x) = e^{-x} \sin(x)$ will have a limit: +However, a damped oscillation, such as $f(x) = e^{-x} \sin(x)$ will have a limit at $\infty$: ```{julia} @@ -254,7 +261,7 @@ limit(exp(-x)*sin(x), x => oo) --- -We have rational functions will have the expected limit. In this example $m = n$, so we get a horizontal asymptote that is not $y=0$: +Rational functions will have the expected limit at $\infty$ found by comparing leading terms. In this example $m = n$, so we get a horizontal asymptote that is not $y=0$, rather $y=1/4$: ```{julia} @@ -293,7 +300,7 @@ $$ \lim_{x \rightarrow 0+} f(1/x) $$ -So whether $\lim_{x \rightarrow 0+} \sin(1/x)$ exists is equivalent to whether $\lim_{x\rightarrow \infty} \sin(x)$ exists, which clearly does not due to the oscillatory nature of $\sin(x)$. +So whether $\lim_{x \rightarrow 0+} \sin(1/x)$ exists is equivalent to whether $\lim_{x\rightarrow \infty} \sin(x)$ exists; clearly it does not due to the oscillatory nature of $\sin(x)$. Similarly, one can make this reduction @@ -311,35 +318,42 @@ That is, right limits can be analyzed as limits at $\infty$ or right limits at $ ## Limits of infinity -Vertical asymptotes are nicely defined with, as with horizontal asymptotes, by the graph getting close to some line. However, the formal definition of a limit won't be the same. For a vertical asymptote, the value of $f(x)$ heads towards positive or negative infinity, not some finite $L$. As such, a neighborhood like $(L-\epsilon, L+\epsilon)$ will no longer make sense, rather we replace it with an expression like $(M, \infty)$ or $(-\infty, M)$. As in: the limit of $f(x)$ as $x$ approaches $c$ is *infinity* if for every $M > 0$ there exists a $\delta>0$ such that if $0 < |x-c| < \delta$ then $f(x) > M$. Approaching $-\infty$ would conclude with $f(x) < -M$ for $M>0$. +Vertical asymptotes are nicely defined with by the graph getting close to some vertical line. Similar in spirit to horizontal asymptotes but the formal definition of a limit won't be the same. For a vertical asymptote, the value of $f(x)$ heads towards positive or negative infinity, not some finite $L$. As such, a neighborhood like $(L-\epsilon, L+\epsilon)$ will no longer make sense, rather we replace it with an expression like $(M, \infty)$ or $(-\infty, M)$. As in: the limit of $f(x)$ as $x$ approaches $c$ is *infinity* if for every $M > 0$ there exists a $\delta>0$ such that if $0 < |x-c| < \delta$ then $f(x) > M$. Approaching $-\infty$ would conclude with $f(x) < -M$ for $M>0$. -##### Examples +##### Example -Consider the function $f(x) = 1/x^2$. This will have a limit at every point except possibly $0$, where division by $0$ is possible. In this case, there is a vertical asymptote, as seen in the following graph. The limit at $0$ is $\infty$, in the extended sense above. For $M>0$, we can take any $0 < \delta < 1/\sqrt{M}$. The following graph shows $M=25$ where the function values are outside of the box, as $f(x) > M$ for those $x$ values with $0 < |x-0| < 1/\sqrt{M}$. +Consider the function $f(x) = 1/x^2$. This will have a limit at every point except possibly $0$, where division by $0$ is possible. In this case, there is a vertical asymptote, as seen in @fig-plot-1-over-xsquqred-showing-limit-at-0. The limit at $0$ is $\infty$, in the extended sense above. For $M>0$, we can take any $0 < \delta < 1/\sqrt{M}$. The following graph shows $M=25$ where the function values are outside of the box, as $f(x) > M$ for those $x$ values with $0 < |x-0| < 1/\sqrt{M}$. +::: {#fig-plot-1-over-xsquqred-showing-limit-at-0} ```{julia} -#| hold: true #| echo: false f(x) = 1/x^2 M = 25 delta = 1/sqrt(M) f(x) = 1/x^2 > 50 ? NaN : 1/x^2 -plot(f, -1, 1, legend=false) +plot(f, -1, 1; ylim=(-5, 50), framestyle=:origin, legend=false) plot!([-delta, delta], [M,M], color=colorant"orange") plot!([-delta, -delta], [0,M], color=colorant"red") plot!([delta, delta], [0,M], color=colorant"red") +annotate!([(1/sqrt(M), 0, text(L"\frac{1}{\sqrt{M}}", :top)), + (0, M, text(L"M", :right)) + ]) ``` ---- +Plot of $f(x) = 1/x^2$ over $[-1,1]$ with overlay suggesting how $\delta$ can be found for a given $M$. +::: + +##### Example -The function $f(x)=1/x$ requires us to talk about left and right limits of infinity, with the natural generalization. We can see that the left limit at $0$ is $-\infty$ and the right limit $\infty$: +The function $f(x)=1/x$ requires us to talk about left- and right-hand limits of infinity, with the natural generalization. We can see that the left limit at $0$ is $-\infty$ and the right limit $\infty$: +::: {#fig-plot-1-over-x-avoiding-asymptote-at-0} ```{julia} #| hold: true #| echo: false @@ -347,6 +361,8 @@ f(x) = 1/x plot(f, 1/50, 1, color=:blue, legend=false) plot!(f, -1, -1/50, color=:blue) ``` +Plot of $f(x) = 1/x$ over $[-1, 1]$ +::: `SymPy` agrees: @@ -357,20 +373,23 @@ f(x) = 1/x limit(f(x), x=>0, dir="-"), limit(f(x), x=>0, dir="+") ``` ---- +##### Example -Consider the function $g(x) = x^x(1 + \log(x)), x > 0$. Does this have a *right* limit at $0$? +Consider the function $g(x) = x^x(1 + \log(x)), x > 0$ in @fig-plot-x-to-x-times-1-plus-logx. Does this have a *right* limit at $0$? A quick graph shows that a limit may be $-\infty$: - +::: {#fig-plot-x-to-x-times-1-plus-logx} ```{julia} g(x) = x^x * (1 + log(x)) plot(g, 1/100, 1) ``` +Plot of $x^x(1 + \log(x))$ over $(0,1)$ +::: + We can check with `SymPy`: @@ -413,52 +432,71 @@ $$ \lim_{x \rightarrow 0} \frac{e^x - 1}{x} = 1, $$ -is an important limit. Using the definition of $e^x$ by an infinite sequence: +is an important limit. We show how to mathematically find it assuming +this definition of $e^x$ by an infinite sequence: $$ -e^x = \lim_{n \rightarrow \infty} (1 + \frac{x}{n})^n, +e^x = \lim_{n \rightarrow \infty} \left(1 + \frac{x}{n}\right)^n, $$ -we can establish the limit using the squeeze theorem. First, +The limit of this sequence is known to `SymPy`: + +```{julia} +@syms x n +limit((1 + x/n)^n, n=>oo) +``` + + +We use this definition to establish the limit above using the squeeze theorem. First, define a new sequence: $$ -A = |(1 + \frac{x}{n})^n - 1 - x| = |\Sigma_{k=0}^n {n \choose k}(\frac{x}{n})^k - 1 - x| = |\Sigma_{k=2}^n {n \choose k}(\frac{x}{n})^k|, +\begin{align*} +A_n +&= \lvert \left(1 + \frac{x}{n}\right)^n - 1 - x\rvert \\ +&= \lvert \sum_{k=0}^n {n \choose k}\left(\frac{x}{n}\right)^k - 1 - x\rvert \\ +&= \lvert \sum_{k=2}^n {n \choose k}\left(\frac{x}{n}\right)^k\rvert, +\end{align*} $$ -the first two sums cancelling off. The above comes from the binomial expansion theorem for a polynomial. Now ${n \choose k} \leq n^k$so we have +the first two sums cancelling off. The second line in the above comes from applying binomial expansion theorem for an integer power. Now ${n \choose k} \leq n^k$so we have $$ -A \leq \Sigma_{k=2}^n |x|^k = |x|^2 \frac{1 - |x|^{n+1}}{1 - |x|} \leq -\frac{|x|^2}{1 - |x|}. +\begin{align*} +A_n +&\leq \lvert \sum_{k=2}^n n^k\left(\frac{x}{n}\right)^k\rvert \\ +&= \sum_{k=2}^n \lvert x\rvert^k \\ +&= \lvert x\rvert^2 \frac{1 - \lvert x\rvert^{n+1}}{1 - \lvert x\rvert} \\ +&\leq \frac{\lvert x\rvert^2}{1 - \lvert x\rvert}. +\end{align*} $$ -using the *geometric* sum formula with $x \approx 0$ (and not $1$): +We used this *geometric* sum formula with $x \approx 0$ (and not $1$) in the third line: ```{julia} #| hold: true -@syms x n i -summation(x^i, (i,0,n)) +@syms r::positive n i +summation(r^i, (i, 0, n)) ``` -As this holds for all $n$, as $n$ goes to $\infty$ we have: +As the above holds for all $n$, as $n$ goes to $\infty$ we have: $$ -|e^x - 1 - x| \leq \frac{|x|^2}{1 - |x|} +\lvert e^x - 1 - x\rvert \leq \frac{\lvert x\rvert^2}{1 - \lvert x\rvert} $$ -Dividing both sides by $x$ and noting that as $x \rightarrow 0$, $|x|/(1-|x|)$ goes to $0$ by continuity, the squeeze theorem gives the limit: +Dividing both sides by $x$ and noting that as $x \rightarrow 0$, $\lvert x\rvert/(1-\lvert x\rvert)$ goes to $0$ by continuity, the squeeze theorem gives the limit: $$ \lim_{x \rightarrow 0} \frac{e^x -1}{x} - 1 = 0. $$ -That ${n \choose k} \leq n^k$ can be viewed as the left side counts the number of combinations of $k$ choices from $n$ distinct items, which is less than the number of permutations of $k$ choices, which is less than the number of choices of $k$ items from $n$ distinct ones without replacement – what $n^k$ counts. +That ${n \choose k} \leq n^k$ can be viewed as the left side counts the number of combinations of $k$ choices from $n$ distinct items, which is less than the number of permutations of $k$ choices, which is less than the number of choices of $k$ items from $n$ distinct ones without replacement---what $n^k$ counts. @@ -466,60 +504,24 @@ That ${n \choose k} \leq n^k$ can be viewed as the left side counts the number o ## Summary -The following table captures the various changes to the definition of the limit to accommodate some of the possible behaviors. +@tbl-various-modifications-to-limit-definition captures the various changes to the definition of the limit to accommodate some of the possible behaviors. [Ross](https://doi.org/10.1007/978-1-4614-6271-2) summarizes all this by enumerating the 15 different *related* definitions for $\lim_{x \rightarrow a} f(x) = L$ that arise from $L$ being either finite, $-\infty$, or $+\infty$ and $a$ being any of $c$, $c-$, $c+$, $-\infty$, or $+\infty$. -```{julia} -#| echo: false -limit_type=[ -"limit", -"right limit", -"left limit", -L"limit at $\infty$", -L"limit at $-\infty$", -L"limit of $\infty$", -L"limit of $-\infty$", -"limit of a sequence" -] -Notation=[ -L"\lim_{x\rightarrow c}f(x) = L", -L"\lim_{x\rightarrow c+}f(x) = L", -L"\lim_{x\rightarrow c-}f(x) = L", -L"\lim_{x\rightarrow \infty}f(x) = L", -L"\lim_{x\rightarrow -\infty}f(x) = L", -L"\lim_{x\rightarrow c}f(x) = \infty", -L"\lim_{x\rightarrow c}f(x) = -\infty", -L"\lim_{n \rightarrow \infty} a_n = L" -] +::: {#tbl-various-modifications-to-limit-definition .hover .striped tbl-colwidths="[31,23,23,23]"} +| Type | Notation | V | U | +| -------------------:| -------------------------------------:| --------------------------:| ------------------------:| +| limit at $c$| $\lim_{x\rightarrow c}f(x) = L$ | $(L-\epsilon, L+\epsilon)$ | $(c - \delta, c+\delta)$ | +| right limit at $c$ | $\lim_{x\rightarrow c+}f(x) = L$ | $(L-\epsilon, L+\epsilon)$ | $(c, c+\delta)$ | +| left limit at $c$| $\lim_{x\rightarrow c-}f(x) = L$ | $(L-\epsilon, L+\epsilon)$ | $(c - \delta, c)$ | +| limit *at* $\infty$ | $\lim_{x\rightarrow \infty}f(x) = L$ | $(L-\epsilon, L+\epsilon)$ | $(M, \infty)$ | +| limit *at* $-\infty$ | $\lim_{x\rightarrow -\infty}f(x) = L$ | $(L-\epsilon, L+\epsilon)$ | $(-\infty, M)$ | +| limit *of* $\infty$ | $\lim_{x\rightarrow c}f(x) = \infty$ | $(M, \infty)$ | $(c - \delta, c+\delta)$ | +| limit *of* $-\infty$ | $\lim_{x\rightarrow c}f(x) = -\infty$ | $(-\infty, M)$ | $(c - \delta, c+\delta)$ | +| limit of a sequence | $\lim_{n \rightarrow \infty} a_n = L$ | $(L-\epsilon, L+\epsilon)$ | $(M, \infty)$ | -Vs = [ -L"(L-\epsilon, L+\epsilon)", -L"(L-\epsilon, L+\epsilon)", -L"(L-\epsilon, L+\epsilon)", -L"(L-\epsilon, L+\epsilon)", -L"(L-\epsilon, L+\epsilon)", -L"(M, \infty)", -L"(-\infty, M)", -L"(L-\epsilon, L+\epsilon)" -] - -Us = [ -L"(c - \delta, c+\delta)", -L"(c, c+\delta)", -L"(c - \delta, c)", -L"(M, \infty)", -L"(-\infty, M)", -L"(c - \delta, c+\delta)", -L"(c - \delta, c+\delta)", -L"(M, \infty)" -] - -d = DataFrame(Type=limit_type, Notation=Notation, V=Vs, U=Us) -table(d) -``` - -[Ross](https://doi.org/10.1007/978-1-4614-6271-2) summarizes this by enumerating the 15 different *related* definitions for $\lim_{x \rightarrow a} f(x) = L$ that arise from $L$ being either finite, $-\infty$, or $+\infty$ and $a$ being any of $c$, $c-$, $c+$, $-\infty$, or $+\infty$. +: Table illustrating various modifications to the basic limit definition to accommodate a wider range of applicability +::: ## Rates of growth @@ -595,7 +597,7 @@ A negative test for compatibility is the following: if $$ -\lim_{x \rightarrow \infty} \frac{\log(|f(x)|)}{\log(|g(x)|)} = 0, +\lim_{x \rightarrow \infty} \frac{\log(\lvert f(x)\rvert)}{\log(\lvert g(x)\rvert)} = 0, $$ Then $f$ and $g$ are not compatible (and $g$ grows faster than $f$). Applying this to the last two values of $f$ and $g$, we have @@ -616,7 +618,7 @@ so $f(x) = \exp(x^2)$ grows faster than $g(x) = \exp(x)^2$. Keeping in mind that logarithms grow slower than powers which grow slower than exponentials ($a > 1$) can help understand growth at $\infty$ as a comparison of leading terms does for rational functions. -We can immediately put this to use to compute $\lim_{x\rightarrow 0+} x^x$. We first express this problem using $x^x = (\exp(\ln(x)))^x = e^{x\ln(x)}$. Rewriting $u(x) = \exp(\ln(u(x)))$, which only uses the basic inverse relation between the two functions, can often be a useful step. +We can immediately put this to use to compute $\lim_{x\rightarrow 0+} x^x$. We first express this problem using $x^x = (\exp(\ln(x)))^x = e^{x\ln(x)}$. Rewriting $u(x) = \exp(\ln(u(x)))$ can often be a useful step. This rewriting only uses the basic inverse relation between the exponential and logarithmic functions. As $f(x) = e^x$ is a suitably nice function (continuous) so that the limit of a composition can be computed through the limit of the inside function, $x\ln(x)$, it is enough to see what $\lim_{x\rightarrow 0+} x\ln(x)$ is. We *re-express* this as a limit at $\infty$ @@ -636,9 +638,10 @@ The last equality follows, as the function $x$ dominates the function $\ln(x)$. ###### Question -Select the graph for which the limit at $a$ is infinite. +Select the graph in @fig-select-graph-limit-at-at-infinite for which the limit at $a$ is infinite. +::: {#fig-select-graph-limit-at-at-infinite} ```{julia} #| hold: true #| echo: false @@ -677,12 +680,15 @@ plotly() hotspotq(imgfile, (1/2,1), (1/2,1)) ``` +Select the graph for which the limit $a$ is infinity +::: + ###### Question -Select the graph for which the limit at $\infty$ appears to be defined. - +Select the graph in @fig-select-graph-limit-at-infty-exists for which the limit at $\infty$ appears to be defined. +::: {#fig-select-graph-limit-at-infty-exists} ```{julia} #| hold: true #| echo: false @@ -715,6 +721,8 @@ savefig(p, imgfile) plotly() hotspotq(imgfile, (1/2,1), (1/2,1)) ``` +Select the graph for which a limit (finite) at $\infty$ appears to exist +::: ###### Question @@ -804,8 +812,8 @@ Find $\lim_{x \rightarrow 2+} (x-3)/(x-2)$. #| hold: true #| echo: false choices=["``L=-\\infty``", "``L=-1``", "``L=0``", "``L=\\infty``"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` Find $\lim_{x \rightarrow -3-} (x-3)/(x+3)$. @@ -815,8 +823,8 @@ Find $\lim_{x \rightarrow -3-} (x-3)/(x+3)$. #| hold: true #| echo: false choices=["``L=-\\infty``", "``L=-1``", "``L=0``", "``L=\\infty``"] -answ = 4 -radioq(choices, answ) +answer = 4 +radioq(choices, answer) ``` ###### Question @@ -866,7 +874,7 @@ Use this fact, to find $L$ ```{julia} -limit(ex1 - (w/2 - 1), w=>0) +limit(ex1 - (w/2 - 1), w=>0; dir="+") ``` $L$ is: @@ -907,7 +915,8 @@ For which value(s) of $k$ in $1,2,3$ is the limit $0$? (Do the above $3$ times u #| hold: true #| echo: false choices = ["``1``", "``2``", "``3``", "``1,2``", "``1,3``", "``2,3``", "``1,2,3``"] -radioq(choices, 1, keep_order=true) +explanation = "Fix `k` to be an integer (`1`, `2`, or `3`) and then compute `limit(sin(sin(x^2))/x^k, x=>0)`" +buttonq(choices, 1; explanation) ``` ###### Question: No limit @@ -923,8 +932,8 @@ choices=["The limit does exist - it is any number from -1 to 1", "Err, the limit does exists and is 1", "The function oscillates too much and its y values do not get close to any one value", "Any function that oscillates does not have a limit."] -answ = 3 -radioq(choices, answ) +answer = 3 +buttonq(choices, answer) ``` ###### Question $0^0$ is not *always* $1$ @@ -947,8 +956,8 @@ Consider different values of $k$ to see if this limit depends on $k$ or not. Wha #| hold: true #| echo: false choices = ["``1``", "``k``", "``\\log(k)``", "The limit does not exist"] -answ = 1 -radioq(choices, answ) +answer = 1 +buttonq(choices, answer) ``` Now, consider this limit: @@ -958,7 +967,7 @@ $$ \lim_{x \rightarrow 0+} x^{1/\log_k(x)} = L. $$ -In `julia`, $\log_k(x)$ is found with `log(k,x)`. The default, `log(x)` takes $k=e$ so gives the natural log. So, we would define `h`, for a given `k`, with +In `Julia`, $\log_k(x)$ is found with `log(k,x)`. The default, `log(x)` takes $k=e$ so gives the natural log. So, we would define `h`, for a given `k`, with ```{julia} @@ -973,8 +982,8 @@ Consider different values of $k$ to see if the limit depends on $k$ or not. What #| hold: true #| echo: false choices = ["``1``", "``k``", "``\\log(k)``", "The limit does not exist"] -answ = 2 -radioq(choices, answ) +answer = 2 +buttonq(choices, answer) ``` ###### Question @@ -996,8 +1005,9 @@ choices=[ "the first, second and third ones", "the first, second, third, and fourth ones", "all of them"] -answ = 5 -radioq(choices, answ, keep_order=true) +answer = 5 +explanation = L"Try them all, you will get $\infty$" +buttonq(choices, answer; explanation) ``` ###### Question @@ -1015,8 +1025,8 @@ L"We can talk about the limit at $\infty$ of $f(x) - mx$ being $b$", L"We can say $f(x) - (mx+b)$ has a horizontal asymptote $y=0$", L"We can say $f(x) - mx$ has a horizontal asymptote $y=b$", "Any of the above"] -answ = 5 -radioq(choices, answ, keep_order=true) +answer = 5 +buttonq(choices, answer) ``` ###### Question @@ -1056,6 +1066,6 @@ choices = [L" $f(x)$ has a limit of $1$ as $x \rightarrow 0$", L" $f(x)$ has a limit of $-1$ as $x \rightarrow 0$", L" $f(x)$ does not have a limit as $x \rightarrow 0$" ] -answ = 3 -radioq(choices, answ) +answer = 3 +buttonq(choices, answer) ``` diff --git a/quarto/limits/sequences_series.qmd b/quarto/limits/sequences_series.qmd index 4cb0644..fc675c4 100644 --- a/quarto/limits/sequences_series.qmd +++ b/quarto/limits/sequences_series.qmd @@ -21,9 +21,13 @@ nothing --- -![Smaller and smaller and smaller..., the [Droste effect](https://en.wikipedia.org/wiki/Droste_effect)](./figures/mona-lisa-recursive.png) +::: {#fig-mona-mona-mona-lisa} +![](./figures/mona-lisa-recursive.png) -This section expands on limits of infinite sequences and their sums. +Smaller and smaller and smaller..., the [Droste effect](https://en.wikipedia.org/wiki/Droste_effect) +::: + +This section expands on limits of infinite sequences and their sums. The material is typically part of the second semester of calculus, but doesn't rely on any new material beyond that on limits already introduced. ## Definitions @@ -37,7 +41,7 @@ A series is a sum of an infinite sequence $\sum_{i=1}^\infty a_i = a_1 + a_2 + a As mentioned, a sequence converges to $L$ if for any $\epsilon$ we can find an $M$ such that if $n > M$ then $|a_n - L| < \epsilon$. A non-convergent sequence is called *divergent*. Series too may be convergent of divergent, with details to come. -## Examples +##### Example Some examples of sequences are: @@ -53,29 +57,26 @@ Some examples of sequences are: ### Some limit theorems for sequences -The limit theorems apply to limits of sequences as well: +The limit theorems apply to limits of sequences as well, including the following: -:::{.callout-note appearance="minimal"} -#### The squeeze theorem +:::{.theorem title="The squeeze theorem"} If $l_n < a_n < r_n$ and *both* $l_n$ and $r_n$ converge to $L$ then $a_n$ converges to $L$. ::: -:::{.callout-note appearance="minimal"} -#### Linear combinations +:::{.theorem title="Linear combinations"} If $a_n \rightarrow L$ and $b_n \rightarrow M$ then for constants $c$ and $d$: $c\cdot a_n + d \cdot b_n \rightarrow c\cdot L + d\cdot M$. ::: -:::{.callout-note appearance="minimal"} -#### Products and ratios -If $a_n \rightarrow L$ and $b_n \rightarrow M$ -$a_n \cdot b_n \rightarrow LM$ and if $M != 0$ -$a_n / b_n \rightarrow L/M$. +:::{.theorem title="Products and ratios"} +If $a_n \rightarrow L$ and $b_n \rightarrow M$ then +$a_n \cdot b_n \rightarrow LM$. + +Further, if $M \neq 0$ then $a_n / b_n \rightarrow L/M$. ::: -:::{.callout-note appearance="minimal"} -#### Composition +:::{.theorem title="Composition"} If the function $f(x)$ has a limit of $L$ at $b$ and $a_n$ converges to $b$ *and* $a_n \neq b$ for large $n$ then @@ -89,17 +90,18 @@ $$ We mention a few other limit theorems for sequences. -One fact about convergent series is +One fact about convergent series is: -> Convergent series are bounded. +:::{.relationship title="Bounded"} +Convergent series are bounded. +::: (That is, there exists an $M>0$ with $-M \leq a_n \leq M$ for all $n$.) Not all bounded sequences converge; however: -:::{.callout-note appearance="minimal"} -##### Bounded and monotone +:::{.relationship title="Bounded and monotone"} If $a_n$ is monotone increasing ($a_n \leq a_{n+1}$ for all $n$) and *bounded* then $a_n$ converges. This is a [monotone convergence theorem](https://en.wikipedia.org/wiki/Monotone_convergence_theorem). It's proof shows the least upper bound is the limit. A similar statement holds for bounded, monotone decreasing sequences. @@ -107,24 +109,20 @@ This is a [monotone convergence theorem](https://en.wikipedia.org/wiki/Monotone_ The sequence $a_n = (1 + 1/n)^n$ is both monotone increasing and bounded, hence convergent. As mentioned, it converges to $e$. That this limit has a known value is not a result of the theorem, which only says some value exists. - - A *subsequence* of an infinite sequence is an infinite sequence chosen by taking only some of the terms (along the same order). A simple example would be $b_n = a_{2n}$, which would take every other term of the sequence $\{a_n\}$. More generally if $\phi(n)$ is an increasing function with integer values, then $b_n = a_{\phi(n)}$ would be a formal way to define a subsequence. -:::{.callout-note appearance="minimal"} -##### Bolzano-Weierstrass theorem +:::{.theorem title="Bolzano-Weierstrass theorem"} Every bounded sequence has a convergent subsequence. ::: -A sketch is to consider the interval $I_1 = [-M,M]$. It has infinitely many values of the sequence in it when $M$ is a bound. Divide this interval in half. There is a choice of $I_2$ so that it also contains infinitely many points of the sequence. (Maybe both do, in which case either choice can be made.) This splitting and choosing can be repeated to create an infinite sequence of *nested* intervals $\{I_n\}$ each containing infinitely many points of $\{a_n\}$ and each of length $M/2^{n-2}$. As these intervals are nested the left-hand endpoints are bounded and increasing, hence convergent. +A sketch is to consider the interval $I_1 = [-M,M]$. It has infinitely many values of the sequence in it when $M$ is a bound. Divide this interval in half. There is a choice of $I_2$ so that it also contains infinitely many points of the sequence. (Maybe both do, in which case either choice can be made.) This splitting and choosing can be repeated to create an infinite sequence of *nested* intervals $\{I_n\}$ each containing infinitely many points of $\{a_n\}$ and each of length $M/2^{n-2}$. As these intervals are nested the values closest to the left-hand endpoints are bounded and increasing, hence convergent. -There may be many different convergent subsequences, this just identified one. +There may be many different convergent subsequences, this sketch just identified one. Finally, this following fact can be used to reverse the pedagogical approach of defining limits by starting with limits restricted to sequences. -:::{.callout-note appearance="minimal"} -##### Limits of functions +:::{.relationship title="Limits of functions"} The limit of $f(x)$ at $c$ exists and equals $L$ if and only if for *every* sequence $x_n$ in the domain of $f$ converging to $c$ the sequence $s_n = f(x_n)$ converges to $L$. ::: @@ -170,10 +168,9 @@ simplify(out) We can't set a *value* for `r` symbolically and were `r=1` this sum is different, but we have $|r| < 1$ so the proposed is indeed the partial sum. -As $n \rightarrow \infty$ for $|r| < 1$ this expression goes to $s = r / (1-r)^2$. It would diverge otherwise. +As $n \rightarrow \infty$ for $|r| < 1$ this expression goes to $s = r / (1-r)^2$. It would diverge otherwise.^[This fact has other derivations which don't require knowing the special formula.] - -##### Example: Sum of inverse factorials +##### Example: sum of inverse factorials Consider these two sequences: @@ -182,14 +179,14 @@ s_n = \sum_{k=0}^n \frac{1}{k!}, \quad p_n = \left(1 + \frac{1}{n}\right)^n $$ -We know $p_n \rightarrow e$. We will see that it also follows that $s_n \rightarrow e$. That is the series for the sequence $a_k = 1/k!$ converges to $e$. +We know $p_n \rightarrow e$. With a fair amount of effort, we will see that it also follows that $s_n \rightarrow e$. That is the series for the sequence $a_k = 1/k!$ converges to $e$. -First we see that $e$ is in the ballpark. +First we confirm that $e$ is in the ballpark. -We bound each term of $s_n$ for $k \geq 2$ +For $k \geq 2$ bound each term of $s_n$ with: $$ -a_k = \frac{1}{k!} \leq \frac{1}{(k(k-1))} = \frac{1}{k-1} - \frac{1}{k}, +a_k = \frac{1}{k!} \leq \frac{1}{k\cdot(k-1)} = \frac{1}{k-1} - \frac{1}{k}, $$ With this, we can identify a telescoping sum: @@ -203,7 +200,7 @@ s_n &= \sum_{k=0}^n a_k\\ \end{align*} $$ -This is in agreement with $s = e$. To get that value takes more effort and different bounds. +With this bound the sequence $s_n$ is bounded and monotone, hence convergent. To identify the value it converges to is $e$ takes more effort and different bounds. @@ -217,7 +214,7 @@ p_n &= \left(1 + \frac{1}{n}\right)^n\\ &= \sum_{k=0}^n \frac{n!}{k!(n-k)!}\frac{1}{n^k}\\ &= \sum_{k=0}^n \frac{1}{k!}\frac{n!}{(n-k)!}\frac{1}{n^k}\\ &= \sum_{k=0}^n \frac{1}{k!} \cdot \left(1 - \frac{1}{n}\right) \cdot \left(1-\frac{2}{n}\right) \cdot \cdots \cdot \left(1 - \frac{k-1}{n}\right)\\ -&= \sum_{k=0}^n \frac{1}{k!} b_{n,k} +&= \sum_{k=0}^n \frac{1}{k!} b_{n,k}. \end{align*} $$ @@ -242,7 +239,7 @@ $$ \left(1 - \frac{1}{n}\right) &\left(1 - \frac{2}{n}\right)\cdot\left(1 - \frac{k}{n}\right)\cdot\left(1 - \frac{k+1}{n}\right)\\ &> \left(1 - \frac{1 + 2 + \cdots + k}{n}\right)\left(1 + \frac{k+1}{n}\right)\\ &= 1 - \frac{1 + 2 + \cdots + k + (k+1)}{n} + \frac{(1+2+\cdots+k)(k+1)}{n}\\ -&> 1 - \frac{1 + 2 + \cdots + k + (k+1)}{n} +&> 1 - \frac{1 + 2 + \cdots + k + (k+1)}{n}. \end{align*} $$ @@ -255,7 +252,7 @@ p_n &= \sum_{k=0}^n \frac{1}{k!}b_{n,l}\\ &= s_n - \sum_{k=0}^n \frac{1}{k!}\frac{(k-1)k}{2n}\\ &= s_n - \frac{1}{2n} \sum_{k=2}^n \frac{1}{(k-2)!}\\ &= s_n - \frac{1}{2n} s_{n-2}\\ -&> s_n - \frac{3}{2n} +&> s_n - \frac{3}{2n}. \end{align*} $$ @@ -279,16 +276,25 @@ There are some general things that are the case for series to understand converg First we consider only sequences with non-negative terms. -:::{.callout-note appearance="minimal"} -##### Necessary condition for convergence +:::{.relationship title="Necessary condition for convergence"} If $a_n \geq 0$ for each $n$ then a necessary condition that $s_n \rightarrow s$ is that $a_n \rightarrow 0$. ::: -This says if $a_n$ does not converge to $0$ then $s_n$ diverges. It is definitely not the case that a sequence that converges to $0$ will lend itself to a convergent series. A famous example would be $\sum_{i=1}^n 1/i$ which diverges. The partial sums of this series are termed the [harmonic series](https://tinyurl.com/ua4893w5) and have the property that $s_n = \ln(n) + \gamma + 1/(2n) + \epsilon_n$ where $e_n \rightarrow 0$ and $\gamma \approx 0.5772$ is a constant termed the Euler-Mascheroni constant. (See `MathConstants.γ`.) +This says if $a_n$ does not converge to $0$ then $s_n$ diverges. It is definitely not the case that a sequence that converges to $0$ will lend itself to a convergent series. An important example is the series known as the harmonic series: + +$$ +s_n = 1 + \frac{1}{2} + \frac{1}{3} + \cdots + \frac{1}{n} = \sum_{i=1}^n \frac{1}{i}. +$$ + +This series has $1/n \rightarrow 0$, but $s_n \rightarrow \infty$. It is [known](https://en.wikipedia.org/wiki/Euler%27s_constant) that + +$$ +\lim_{n \rightarrow \infty} \left( \sum_{i=1}^n \frac{1}{i} - \ln(i) \right) = \gamma = 0.57721\cdots +$$ -:::{.callout-note appearance="minimal"} -##### Only the tail terms determine convergence +:::{.relationship title="Only the tail terms determine convergence"} + Convergence of $\sum_n a_n$ only depends on the terms for $n > N$ for any fixed $N$. Only the tail terms determine convergence, but every term determines the value of the series when it converges. @@ -297,8 +303,8 @@ Only the tail terms determine convergence, but every term determines the value o Fix any $N > 0$, the partial sums with $n>N$ satisfy: $$ -s_n = \sum_{k=1}^n a_k -=\sum_{k=1}^N a_k + \sum_{k=N}^n a_k +s_n = \sum_{i=1}^n a_i +=\sum_{i=1}^N a_i + \sum_{i=N}^n a_i = s_N + (s_n - s_N) $$ @@ -306,13 +312,13 @@ The limit as $n \rightarrow \infty$ does not depend on the constant $s_N$. -:::{.callout-note appearance="minimal"} -##### Comparison test -If $0 \leq c_n \leq a_n \leq b_n$ for each $n$ then +:::{.theorem title="Comparison test"} -* if $\sum_{i=1}^n b_i$ converges then $\sum_{i=0}^n a_i$ converges; +If $0 \leq c_n \leq a_n \leq b_n$ for each $n$ then: -* if $\sum_{i=1}^n c_i$ diverges then $\sum_{i=0}^n a_i$ diverges. +* if $\sum_{i=1}^n b_i$ converges then $\sum_{i=1}^n a_i$ converges; + +* if $\sum_{i=1}^n c_i$ diverges then $\sum_{i=1}^n a_i$ diverges. ::: This can be used to prove, for example, that if a series based on a non-negative sequence converges, any series based on a subsequence will also converge. @@ -348,12 +354,11 @@ We can use the comparison test to say $s_n$ converges, as we earlier saw the bou There are other tests that, when applicable, are more direct and avoid needing to identify a bound. -:::{.callout-note appearance="minimal"} -##### Ratio test +:::{.theorem title="Ratio test"} Consider the series formed from the sequence $\{a_n\}$ with $a_n \geq 0$. The ratios $a_{n+1}/a_n$ can determine if the series converges or diverges: -* if $a_{n+1}/a_n \rightarrow L$ and $L < 1$ then the series converges -* if $a_{n+1}/a_n \rightarrow L$ and $L > 1$ then the series diverges +* if $a_{n+1}/a_n \rightarrow L$ and $L < 1$ then the series converges; +* if $a_{n+1}/a_n \rightarrow L$ and $L > 1$ then the series diverges; and * if $a_{n+1}/a_n \rightarrow L$ and $L = 1$ then the series may or may not converge. ::: @@ -363,7 +368,7 @@ $$ a_{n+1} \leq a_n r \leq a_{n-1}r^2 \leq \cdots \leq a_1 r^n $$ -By the comparison test, the series $\sum a_k$ converges, since $0 < r < 1$. +By the comparison test, the series $\sum a_i$ converges, since $0 < r < 1$. The case for $L > 1$ is similar, only we find a lower bound on each term. @@ -371,9 +376,9 @@ For the case $L=1$---which is where most hard problems fall---we have examples w A similar type of theorem involves powers of the terms in the sequence -:::{.callout-note appearance="minimal"} -##### Root test -Consider the series formed from the sequence $\{a_n\}$ with $a_n \geq 0$. The values of $(a_n)^{1/n}$ can determine if the series converges or diverges: +:::{.theorem title="Root test"} + +Consider the series formed from the sequence $\{a_n\}$ with $a_n \geq 1$. The values of $(a_n)^{1/n}$ can determine if the series converges or diverges: * if $(a_n)^{1/n} \rightarrow L$ and $L < 1$ then the series converges * if $(a_n)^{1/n} \rightarrow L$ and $L > 1$ then the series diverges @@ -386,21 +391,32 @@ $a_0 + a_1 + a_2 + \cdots + a_n < r^0 + r^1 + r^2 + \cdots + r^n$. The geometric The same two examples ($a_n = 1/n$ and $a_n = 1/n^2$) give examples of divergent and convergent series when $L = 1$. -:::{.callout-note appearance="minimal"} -##### The p-series test +:::{.theorem title="The p-series test"} Fix $p>0$. Consider the series $$ -s = \sum_{i=1}^\infty \frac{1}{i^p} +s = \sum_{i=1}^\infty \frac{1}{i^p}. $$ -If $p > 1$ this series is convergent; if $0 < p \leq 1$ the series is divergent. +* if $p > 1$ this series is convergent; +* if $0 < p \leq 1$ the series is divergent. ::: This test is a consequence of a more general integral test which will be discussed later, below we offer a specific proof. -When $p = 1$ this clearly diverges, it being the harmonic series. +When $p = 1$ this diverges, it being the harmonic series. A short [proof](https://web.williams.edu/Mathematics/lg5/harmonic.pdf) is to assign the answer a value, say $H$ and then: -When $p < 1$, we have $i^p < i$ so $1/i^p > 1/i$. By the comparison test (to the harmonic series) the series $s$ will diverge. +$$ +\begin{array}{lllllllllllllll} +H &=& 1 &+& \frac{1}{2} &+& \frac{1}{3} &+& \frac{1}{4} &+& \frac{1}{5} &+& \frac{1}{6} &+& \cdots\\ +&\geq& 1 &+& \frac{1}{2} &+& \frac{1}{4} &+& \frac{1}{4} &+& \frac{1}{6} &+& \frac{1}{6} &+& \cdots\\ +&=& 1 &+& \frac{1}{2} &+& \frac{1}{2} & & &+& \frac{1}{3} & & &+& \cdots\\ +&=& \frac{1}{2} &+& H & & & & & & & & & & +\end{array} +$$ + +The second line comes by replacing denominators of the form $2n-1$ by the larger value $2n$ and then adding $1/(2n)$ to itself to get $1/n$. Rearranging leaves $H$ and an extra $1/2$. No finite $H$ can satisfy $H \geq 1/2 + H$, so $H$ must be infinite. + +When $p < 1$, we have for $i \geq 1$ that $i^p < i$ so $1/i^p > 1/i$. By the comparison test (to the harmonic series) the series $s$ will diverge. To prove this series converges when $p > 1$, we split the sum up into different pieces from $2^k$ to $2^{k+1}-1$. Call this sum $t_k$ then $s = \sum_k t_k$ with @@ -417,14 +433,15 @@ The above replaces each term with the smallest value over $2^k$ to $2^{k+1}-1$ a For $p > 1$, the value of $2/2^p < 1$. That is, $s$ is bounded by a geometric series hence convergent. ##### Example -Consider this series +Consider the series $$ \sum_{n=1}^\infty \frac{n^k}{5^n}, $$ + for some positive integer $k$. -Will this converge? +Will this series converge? The ratio of $a_{n+1}$ to $a_n$ is: @@ -439,10 +456,10 @@ As the limit of this ratio is less than $1$, the series converges. ##### Example -Let $a$ be a positive number and consider this series +Let $a$ be a positive number and consider this series: $$ -\sum_{n=1}^\infty \frac{a^n}{n^n} +\sum_{n=1}^\infty \frac{a^n}{n^n}. $$ Does this converge? @@ -462,14 +479,14 @@ That this is less than $1$ says the series converges. Consider this series: $$ -\sum_{n=3}^\infty \frac{1}{n^{3/2} \log(n)} +\sum_{n=3}^\infty \frac{1}{n^{3/2} \log(n)}. $$ This isn't exactly in the format for the $p$-series test, but we note that $\log(n) \geq 1$ for $n\geq 3$. So $$ a_n = \frac{1}{n^{3/2} \log(n)} -\leq \frac{1}{n^{3/2}} +\leq \frac{1}{n^{3/2}}. $$ The series $\sum n^{-3/2}$ converges by the $p$-series test, hence the series in question must also converge. @@ -484,9 +501,10 @@ First, we say that a series is *absolutely convergent* if $\sum |a_n|$ converges A series which is convergent but not absolutely convergent is termed *conditionally convergent*. -:::{.callout-note appearance="minimal"} -##### Absolute convergence implies convergence +:::{.theorem title="Absolute convergence implies convergence"} + If the series $\sum a_k$ is *absolutely convergent* then it is convergent. + ::: @@ -520,8 +538,7 @@ This series is conditionally convergent---not absolutely convergent. Why follows immediately from a more general statement. -:::{.callout-note appearance="minimal"} -##### Alternating series test +:::{.theorem title="Alternating series test"} If $a_n$ and $a_{n+1}$ have different signs for each $n$ *and* $|a_n| \rightarrow 0$ *monotonically* then $$ s = \sum_{k=1}^\infty a_k = a_1 + a_2 + a_3 + \cdots @@ -580,7 +597,7 @@ Take the sequence $\{ a_n \}$. This is simply a list of values with some indicat * If the series is *conditionally convergent* but not *absolutely convergent* then a reordering may converge (conditionally) to a different value or even diverge. -The case of the latter is the alternating harmonic series. to see it diverges, we note that the subsequences of both negative and positive terms diverge. Call these $\{b_n\}$ and $\{c_n\}$. To see that we can make these diverge, we note that for any $N>0$ and $L$ we can find $k$ so that $b_N + b_{N+1} + \cdots + b_{N+k} > L$. Now for each $i$, we take a consecutive subsequence of values of $b_n$ which sum to $i + |c_i|$. Then the subsequence formed by a group of $b$s followed by $c_i$ will have partial sums always bigger than $i$, hence divergent. +The case of the latter is the alternating harmonic series. to see it diverges, we note that the subsequences of both negative and positive terms diverge. Call these $\{b_n\}$ and $\{c_n\}$. To see that we can make these diverge, we note that for any $N>0$ and $L$ we can find $k$ so that $b_N + b_{N+1} + \cdots + b_{N+k} > L$. Now for each $i$, we take a consecutive subsequence of values of $b_n$ which sum to $i + \lvert c_i\rvert$. Then the subsequence formed by a group of $b$s followed by $c_i$ will have partial sums always bigger than $i$, hence divergent. ## Power series @@ -597,24 +614,24 @@ A typical case is $c=0$. For any fixed $x$ this is simply a series. Convergence Suppose the series $\sum a_n$ is absolutely convergent. Then there are some values around $c$ for which the series converges ($x=c$ is one). The *radius of convergence* is a value $r$ for which -* if $|x-c| < r$ then the power series converges absolutely; and -* if $|x-c| > r$ the power series *diverges*. +* if $\lvert x-c\rvert < r$ then the power series converges absolutely; and +* if $\lvert x-c\rvert > r$ the power series *diverges*. The root test indicates why such a value exists. Suppose we consider the term $b_n = a_n (x-c)^n$. Then $$ -(|b_n|)^{1/n} = (|a_n|)^{1/n} |x-c| +\lvert b_n\rvert^{1/n} = \lvert a_n\rvert^{1/n} \lvert x-c \rvert $$ -If $(|a_n|^{1/n}) \rightarrow L$ then the root test indicates if the power series converges absolutely or not. In particular. If $L|x-c| < 1$ it converges and if $L|x-c| > 1$ it diverges. That is if $|x-c| < 1/L$ or $|x-c| > 1/L$. Taking $r=1/L$ yields the radius of convergence. (For cases where the limit does not exist, a relaxed version of the root test with the *limit inferior* is applicable.) +If $\lvert a_n \rvert^{1/n} \rightarrow L$ then the root test indicates if the power series converges absolutely or not. In particular. If $L\lvert x-c \rvert < 1$ it converges and if $L\lvert x-c \rvert > 1$ it diverges. That is if $\lvert x-c \rvert < 1/L$ or $\lvert x-c \rvert > 1/L$. Taking $r=1/L$ yields the radius of convergence. (For cases where the limit does not exist, a relaxed version of the root test with the *limit inferior* is applicable.) The ratio test can also be used to establish the radius of convergence provided: $$ -\frac{|a_{n+1}||(x-c)^{n+1}|}{|a_n||(x-c)^n|} = -\frac{|a_{n+1}|}{|a_n|} |x-c| \rightarrow L |x-c|. +\frac{\lvert a_{n+1} \rvert\lvert (x-c)^{n+1} \rvert}{\lvert a_n \rvert\lvert (x-c)^n \rvert} = +\frac{\lvert a_{n+1} \rvert}{\lvert a_n \rvert} \lvert x-c \rvert \rightarrow L \lvert x-c \rvert. $$ Again, $r=1/L$. @@ -630,9 +647,9 @@ $$ This has $c=0$ and $a_n = 1/n!$. The ratio test simplifies to $$ -\frac{|x-c|^{n+1}}{(n+1)!} \cdot -\frac{n!}{|x-c|^n} = -\frac{|x-c|}{n} \rightarrow 0 +\frac{\lvert x-c \rvert^{n+1}}{(n+1)!} \cdot +\frac{n!}{\lvert x-c \rvert^n} = +\frac{\lvert x-c \rvert}{n} \rightarrow 0 $$ Hence $r=\infty$ and the power series is always absolutely convergent. @@ -647,10 +664,10 @@ $$ This has term $a_n = (-1)^{n} x^{2n+1}/(2n+1)$, $n\geq 0$. The root test for absolute convergence has $$ -(|a_n|)^{1/n} = \frac{x^{2 + 1/n}}{(2n+1)^{1/n}} \rightarrow x^2 +(\lvert a_n \rvert)^{1/n} = \frac{x^{2 + 1/n}}{(2n+1)^{1/n}} \rightarrow x^2 $$ -Provided $|x| < 1$ the power series will converge. +Provided $\lvert x \rvert < 1$ the power series will converge. Both these examples are related to *Taylor series* for some function. @@ -1077,8 +1094,8 @@ radioq(choices, 2) The following extends the comparison test: -:::{.callout-note appearance="minimal"} -### Limit comparison test +:::{.theorem title="Limit comparison test"} + Take two series with *positive* terms $\sum a_n$ and $\sum b_n$. If $\lim_{n\rightarrow \infty} a_n/b_n = c$ with $0 < c < \infty$ then either both series converge of both series diverge. @@ -1154,7 +1171,7 @@ L"s - t", L"s \cdot t", L"s / t" ] -radioq(choices, 31; keep_order=true) +radioq(choices, 3; keep_order=true) ``` For completeness, [Wikipedia](https://en.wikipedia.org/wiki/Power_series#Multiplication_and_division) gives this formula for division $s/t = \sum_{n=0}^\infty d_n \cdot (x-c)^n$ where @@ -1164,15 +1181,18 @@ d_0 = \frac{a_0}{b_0} $$ and + $$ d_n = \frac{1}{b_0^{n+1}} \cdot -\begin{vmatrix} -a_n & b_1 & b_2 & \cdots & b_n\\ -a_{n-1} & b_0 & b_1 & \cdots & b_{n-1}\\ -a_{n-2} & 0 & b_0 & \cdots & b_{n-1}\\ -\vdots & \vdots & \vdots & \ddots & \vdots\\\ -a_0 & 0 & 0 & \cdots & b_0 -\end{vmatrix} +\det\left( +\begin{array}{ccccc} +a_n & b_1 & b_2 & \cdots & b_n\\ +a_{n-1} & b_0 & b_1 & \cdots & b_{n-1}\\ +a_{n-2} & 0 & b_0 & \cdots & b_{n-1}\\ +\vdots & \vdots & \vdots & \ddots & \vdots\\ +a_0 & 0 &0 & \cdots & b_0\\ +\end{array} +\right) $$ -The last operation is called the determinant and will be discussed later on. +The last operation, $\det$, is called the *determinant* and will be discussed later on. diff --git a/quarto/misc/Project.toml b/quarto/misc/Project.toml index 671818d..aa7a6f7 100644 --- a/quarto/misc/Project.toml +++ b/quarto/misc/Project.toml @@ -4,6 +4,8 @@ ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210" HCubature = "19dc6840-f33b-545b-b366-655c7e3ffd49" LaTeXStrings = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f" Mustache = "ffc61752-8dc7-55ee-8c37-f3e9cdd09e70" +PlotlyBase = "a03496cd-edff-5a9b-9e67-9cda94a718b5" +PlotlyKaleido = "f2990250-8cf9-495f-b13a-cce12b45703c" Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80" QuadGK = "1fd47b50-473d-5c70-9696-f719f8f3bcdc" QuizQuestions = "612c44de-1021-4a21-84fb-7261cf5eb2d4" diff --git a/quarto/misc/getting_started_with_julia.qmd b/quarto/misc/getting_started_with_julia.qmd index 04a978c..71b54bc 100644 --- a/quarto/misc/getting_started_with_julia.qmd +++ b/quarto/misc/getting_started_with_julia.qmd @@ -25,7 +25,7 @@ There are a few services for running `Julia` through the web. Mentioned here is [launch binder](https://mybinder.org/v2/gh/CalculusWithJulia/CwJScratchPad.git/master) -Clicking the launch link above will open a web page which provides a blank notebook, save for a package used by these notes. However, `Binder` is nowhere near as reliable as a local installation. +Clicking the launch link above will open a web page which provides a blank notebook, save for a package used by these notes. However, `Binder` is nowhere near as reliable as a local installation; it has timeout issues, memory limitations, and can be slow to load, as would be expected of a free service. ## Installing Julia locally @@ -49,25 +49,28 @@ The base `Julia` provides a *command-line interface*, or REPL (read-evaluate-par Once installed, `Julia` can be started by clicking on an icon or typing `julia` at the command line. Either will open a *command line interface* for a user to interact with a `Julia` process. The basic workflow is easy: commands are typed then sent to a `Julia` process when the "return" key is pressed for a complete expression. Then the output is displayed. -A command is typed following the *prompt*. An example might be `2 + 2`. To send the command to the `Julia` interpreter the "return" key is pressed. A complete expression or expressions will then be parsed and evaluated (executed). If the expression is not complete, `julia`'s prompt will still accept input to complete the expression. Type `2 +` to see. (The expression `2 +` is not complete, as the infix operator `+` expects two arguments, one on its left and one on its right.) +The REPL is shown when `Julia` is started from the command line and has a banner that looks like this: ```{julia} -#| eval: false - _ - _ _ _(_)_ | Documentation: https://docs.julialang.org - (_) | (_) (_) | - _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. - | | | | | | |/ _` | | - | | |_| | | | (_| | | Version 1.7.0 (2021-11-30) - _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release -|__/ | +#| echo: false +using REPL +REPL.banner() +``` + + +In the REPL a command is typed following the *prompt*. The prompt would look like `julia> `. + +```{julia} +#| eval: false julia> 2 + 2 4 ``` -Above, `julia>` is the prompt. These notes will not include the prompt, so that copying-and-pasting can be more easily used. Input and output cells display similarly, though with differences in coloring. For example: +In the above, the command (`2+2`) was sent to the the `Julia` interpreter when the "return" key is pressed. A complete expression or expressions will then be parsed and evaluated (executed). If the expression is not complete, `julia`'s prompt will still accept input to complete the expression. Type `2 +` to see. (The expression `2 +` is not complete, as the infix operator `+` expects two arguments, one on its left and one on its right.) + +These notes will not include the prompt, so that copying-and-pasting can be more easily used. Input and output cells display similarly, though with differences in coloring. For example: ```{julia} diff --git a/quarto/misc/julia_interfaces.qmd b/quarto/misc/julia_interfaces.qmd index c527f59..82aeef2 100644 --- a/quarto/misc/julia_interfaces.qmd +++ b/quarto/misc/julia_interfaces.qmd @@ -27,8 +27,8 @@ Base `Julia` comes with a `REPL` package, which provides a means to interact wi (_) | (_) (_) | _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. | | | | | | |/ _` | | - | | |_| | | | (_| | | Version 1.7.0 (2021-11-30) - _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release + | | |_| | | | (_| | | Version 1.12.0 (2025-10-07) + _/ |\__'_|_|_|\__'_| | Official https://julialang.org release |__/ | julia> 2 + 2 diff --git a/quarto/misc/quick_notes.qmd b/quarto/misc/quick_notes.qmd index b3e4c82..fe71e6f 100644 --- a/quarto/misc/quick_notes.qmd +++ b/quarto/misc/quick_notes.qmd @@ -523,15 +523,15 @@ Arguments of interest include | Attribute | Value | -|:--------------:|:------------------------------------------------------:| -| `legend` | A boolean, specify `false` to inhibit drawing a legend | -| `aspect_ratio` | Use `:equal` to have x and y axis have same scale | -| `linewidth` | Integers greater than 1 will thicken lines drawn | -| `color` | A color may be specified by a symbol (leading `:`). | -| | E.g., `:black`, `:red`, `:blue` | +|:---------------|:-------------------------------------------------------| +| `legend` | A boolean, specify `false` to inhibit drawing a legend | +| `aspect_ratio` | Use `:equal` to have x and y axis have same scale | +| `linewidth` | Integers greater than 1 will thicken lines drawn | +| `color` | A color may be specified by a symbol (leading `:`). E.g., `:black`, `:red`, `:blue` | +| `line` | A shorthands, allowing specification by type, e.g. `(1, :black, 0.25, :dot)` | - * using `plot(xs, ys)` +* using `plot(xs, ys)` The lower level interface to `plot` involves directly creating x and y values to plot: diff --git a/quarto/precalc/Project.toml b/quarto/precalc/Project.toml index 8701513..28ea850 100644 --- a/quarto/precalc/Project.toml +++ b/quarto/precalc/Project.toml @@ -1,4 +1,5 @@ [deps] +AbbreviatedStackTraces = "ac637c84-cc71-43bf-9c33-c1b4316be3d4" CalculusWithJulia = "a2e0e22d-7d4c-5312-9169-8b992201a882" DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" IJulia = "7073ff75-c697-5162-941a-fcdaad2a7d2a" diff --git a/quarto/precalc/exp_log_functions.qmd b/quarto/precalc/exp_log_functions.qmd index e3363a7..bd93996 100644 --- a/quarto/precalc/exp_log_functions.qmd +++ b/quarto/precalc/exp_log_functions.qmd @@ -15,7 +15,6 @@ plotly() --- - The family of exponential functions is used to model growth and decay. The family of logarithmic functions is defined here as the inverse of the exponential functions, but have reach far outside of that. @@ -45,7 +44,7 @@ For $a \neq 0$, $a^0$ is defined to be $1$. For positive, integer values of $n$, we have by definition that $a^{-n} = 1/a^n$. -For $n$ a positive integer, we can define $a^{1/n}$ to be the unique positive solution to $x^n=a$. +For $n$ a positive integer, we can define $a^{1/n}$ to be the unique, positive, real solution to $x^n=a$. Using the key properties of exponents we can extend this to a definition of $a^x$ for any rational $x$. @@ -57,31 +56,45 @@ Defining $a^x$ for any real number requires some more sophisticated mathematics. One method is to use a [theorem](http://tinyurl.com/zk86c8r) that says a *bounded* monotonically increasing sequence will converge. (This uses the [Completeness Axiom](https://en.wikipedia.org/wiki/Completeness_of_the_real_numbers).) Then for $a > 1$ we have if $q_n$ is a sequence of rational numbers increasing to $x$, then $a^{q_n}$ will be a bounded sequence of increasing numbers, so will converge to a number defined to be $a^x$. Something similar is possible for the $0 < a < 1$ case. -This definition can be done to ensure the rules of exponents hold for $a > 0$: +This definition can be done to ensure the rules of exponents hold. + +::: {.relationship title="Rules of exponents"} +For $a > 0$: $$ -a^{x + y} = a^x \cdot a^y, \quad (a^x)^y = a^{x \cdot y}. +\begin{align*} +a^{x + y} &= a^x \cdot a^y,\\ +(a^x)^y &= a^{x \cdot y}. +\end{align*} $$ +::: In `Julia` these functions are implemented using `^`. A special value of the base, $e$, may be defined as well in terms of a limit. The exponential function $e^x$ is implemented in `exp`. +::: {#fig-family-of-exponential-functions} ```{julia} -#| hold: true +#| echo: false plot(x -> (1/2)^x, -2, 2, label="1/2") plot!(x -> 1^x, label="1") plot!(x -> 2^x, label="2") plot!(x -> exp(x), label="e") ``` +Plots of different exponential functions, when $0 < a < 1$ there is decay; when $a=1$ a constant; and when $a > 1$ exponential growth. +::: + We see examples of some general properties: - * The domain is all real $x$ and the range is all *positive* $y$ (provided $a \neq 1$). - * For $0 < a < 1$ the functions are monotonically decreasing. - * For $a > 1$ the functions are monotonically increasing. - * If $1 < a < b$ and $x > 0$ we have $a^x < b^x$. +* The domain is all real $x$ and the range is all *positive* $y$ (provided $a \neq 1$). + +* For $0 < a < 1$ the functions are monotonically decreasing. + +* For $a > 1$ the functions are monotonically increasing. + +* If $1 < a < b$ and $x > 0$ we have $a^x < b^x$. ##### Example @@ -102,6 +115,7 @@ P0 * exp(r2*t), P0 * exp(r8*t) As can be seen, there is quite a bit of difference. +##### Example: Rule of 72 In $1494$, [Pacioli](http://tinyurl.com/gsy939y) gave the "Rule of $72$", stating that to find the number of years it takes an investment to double when continuously compounded one should divide the interest rate into $72$. @@ -138,7 +152,7 @@ n = 2 * 24 That would be an enormous growth. Don't worry: "Exponential growth cannot continue indefinitely, however, because the medium is soon depleted of nutrients and enriched with wastes." -:::{.callout-note} +::: {.callout-note} ## Note The value of `2^n` and `2.0^n` is different in `Julia`. The former remains an integer and is subject to integer overflow for `n > 62`. As used above, `2^(n/6)` will not overflow for larger `n`, as when the exponent is a floating point value, the base is promoted to a floating point value. @@ -156,7 +170,7 @@ That is evidence that the $F_n \approx c\cdot 1.618^n$. (See [Relation to golden ##### Example -In the previous example, the exponential family of functions is used to describe growth. Polynomial functions also increase. Could these be used instead? If so that would be great, as they are easier to reason about. +In a previous example, the exponential family of functions is used to describe growth. Polynomial functions also increase. Could these be used instead? If so that would be great, as they are easy to reason about. The key fact is that exponential growth is much greater than polynomial growth. That is for large enough $x$ and for any fixed $a>1$ and positive integer $n$ it is true that $a^x \gg x^n$. @@ -171,34 +185,6 @@ Later we will see an easy way to certify this statement. Euler's number, $e$, may be defined several ways. One way is to define $e^x$ by the limit as $n$ grows infinitely large of $(1+x/n)^n$. Then $e=e^1$. The value is an irrational number. This number turns up to be the natural base to use for many problems arising in calculus. In `Julia` there are a few mathematical constants that get special treatment, so that when needed, extra precision is available. The value `e` is not immediately assigned to this value, rather `ℯ` is. This is typed `\euler[tab]`. The label `e` is thought too important for other uses to reserve the name for representing a single number. However, users can issue the command `using Base.MathConstants` and `e` will be available to represent this number. When the `CalculusWithJulia` package is loaded, the value `e` is defined to be the floating point number returned by `exp(1)`. This loses the feature of arbitrary precision, but has other advantages. -A [cute](https://www.mathsisfun.com/numbers/e-eulers-number.html) appearance of $e$ is in this problem: Let $a>0$. Cut $a$ into $n$ equal pieces and then multiply them. What $n$ will produce the largest value? Note that the formula is $(a/n)^n$ for a given $a$ and $n$. - - -Suppose $a=5$ then for $n=1,2,3$ we get: - - -```{julia} -#| hold: true -a = 5 -(a/1)^1, (a/2)^2, (a/3)^3 -``` - -We'd need to compare more, but at this point $n=2$ is the winner when $a=5$. - - -With calculus, we will be able to see that the function $f(x) = (a/x)^x$ will be maximized at $a/e$, but for now we approach this in an exploratory manner. Suppose $a=5$, then we have: - - -```{julia} -#| hold: true -a = 5 -n = 1:10 -f(n) = (a/n)^n -@. [n f(n) (a/n - e)] # @. just allows broadcasting -``` - -We can see more clearly that $n=2$ is the largest value for $f$ and $a/2$ is the closest value to $e$. This would be the case for any $a>0$, pick $n$ so that $a/n$ is closest to $e$. - ##### Example: The limits to growth @@ -206,45 +192,48 @@ We can see more clearly that $n=2$ is the largest value for $f$ and $a/2$ is the The $1972$ book [The limits to growth](https://donellameadows.org/wp-content/userfiles/Limits-to-Growth-digital-scan-version.pdf) by Meadows et. al. discusses the implications of exponential growth. It begins stating their conclusion (emphasis added): "If the present *growth* trends in world population, industrialization, pollution, food production, and resource depletion continue *unchanged*, the limits to growth on this planet will be reached sometime in the next *one hundred* years." They note it is possible to alter these growth trends. We are now half way into this time period. -Let's consider one of their examples, the concentration of carbon dioxide in the atmosphere. In their Figure $15$ they show data from $1860$ onward of CO$_2$ concentration extrapolated out to the year $2000$. At [climate.gov](https://www.climate.gov/news-features/understanding-climate/climate-change-atmospheric-carbon-dioxide) we can see actual measurements from $1960$ to $2020$. Numbers from each graph are read from the graphs, and plotted in the code below: - +Let's consider one of their examples, the concentration of carbon dioxide in the atmosphere. In their Figure $15$ they show data from $1860$ onward of CO$_2$ concentration extrapolated out to the year $2000$. At [climate.gov](https://www.climate.gov/news-features/understanding-climate/climate-change-atmospheric-carbon-dioxide) we can see actual measurements from $1960$ to $2020$. Numbers from each graph are read from the graphs, and plotted in the code below. See @fig-plot-of-climate-data-1960-2020. +::: {#fig-plot-of-climate-data-1960-2020} ```{julia} -co2_1970 = [(1860, 293), (1870, 293), (1880, 294), (1890, 295), (1900, 297), - (1910, 298), (1920, 300), (1930, 303), (1940, 305), (1950, 310), - (1960, 313), (1970, 320), (1980, 330), (1990, 350), (2000, 380)] -co2_2021 = [(1960, 318), (1970, 325), (1980, 338), (1990, 358), (2000, 370), - (2010, 390), (2020, 415)] +co2_1970 = [(1860, 293), (1870, 293), (1880, 294), (1890, 295), + (1900, 297), (1910, 298), (1920, 300), (1930, 303), + (1940, 305), (1950, 310), (1960, 313), (1970, 320), + (1980, 330), (1990, 350), (2000, 380)] +co2_2021 = [(1960, 318), (1970, 325), (1980, 338), (1990, 358), + (2000, 370), (2010, 390), (2020, 415)] plot(co2_1970, legend=false) # vector of points interface plot!(co2_2021) -exp_model(;r, x0, P0) = x -> P0 * exp(r * (x - x0)) - +# add two exponential models r = 0.002 x0, P0 = 1960, 313 -plot!(exp_model(; r, x0, P0), 1950, 1990, linewidth=5, alpha=0.25) +m1(x) = P0 * exp(r * (x - x0)) +plot!(m1, 1950, 1990, line=(:blue, 5, 0.25)) r = 0.005 x0, P0 = 2000, 370 - -plot!(exp_model(; r, x0, P0), 1960, 2020, linewidth=5, alpha=0.25) +m2(x) = P0 * exp(r * (x - x0)) +plot!(m2, 1960, 2020, line=(:blue, 5, 0.25)) ``` +Plot of $CO_2$ concentration over time with two exponential models layered on +::: We can see that the projections from the year $1970$ hold up fairly well. -On this plot we added two *exponential* models. at $1960$ we added a *roughly* $0.2$ percent per year growth (a rate mentioned in an accompanying caption) and at $2000$ a roughly $0.5$ percent per year growth. The former barely keeping up with the data. (To do so, we used a parameterized function making for easier code reuse.) +On this plot we added two *exponential* models. at $1960$ we added a *roughly* $0.2$ percent per year growth (a rate mentioned in an accompanying caption) and at $2000$ a roughly $0.5$ percent per year growth. The former barely keeping up with the data. -The word **roughly** above could be made exact. Suppose we knew that between $1960$ and $1970$ the rate went from $313$ to $320$. If this followed an exponential model, then $r$ above would satisfy: +The word *roughly* above could be made exact. Suppose we knew that between $1960$ and $1970$ the rate went from $313$ to $320$. If this followed an exponential model, then $r$ above would satisfy: $$ P_{1970} = P_{1960} e^{r \cdot (1970 - 1960)} $$ -or on division $320/313 = e^{r\cdot 10}$. Solving for $r$ can be done – as explained next – and yields $0.002211\dots$. +or on division $320/313 = e^{r\cdot 10}$. Solving for $r$ can be done---as explained next---and yields $0.002211\dots$. ## Logarithmic functions @@ -262,9 +251,10 @@ That is $a^{\log_a(x)} = x$ for $x > 0$ and $\log_a(a^x) = x$ for all $x$. To see how a logarithm is mathematically defined will have to wait, though the family of functions---one for each $a>0$---are implemented in `Julia` through the function `log(a,x)`. There are special cases requiring just one argument: `log(x)` will compute the natural log, base $e$---the inverse of $f(x) = e^x$; `log2(x)` will compute the log base $2$---the inverse of $f(x) = 2^x$; and `log10(x)` will compute the log base $10$- the inverse of $f(x)=10^x$. (Also `log1p` computes an accurate value of $\log(1 + p)$ when $p \approx 0$.) -To see this in an example, we plot for base $2$ the exponential function $f(x)=2^x$, its inverse, and the logarithm function with base $2$: +To see this in an example, we plot for base $2$ the exponential function $f(x)=2^x$, its inverse, and the logarithm function with base $2$. Though we made three graphs, only two are seen in @fig-plot-2-to-x-inverse-log2, as the graph of `log2` matches that of the inverse function. +::: {#fig-plot-2-to-x-inverse-log2} ```{julia} #| hold: true f(x) = 2^x @@ -277,7 +267,8 @@ xs = range(1/4, stop=4, length=100) plot!(xs, log2.(xs), color=:green, label="log₂") # plot log2 ``` -Though we made three graphs, only two are seen, as the graph of `log2` matches that of the inverse function. +Plot of $f(x) = 2^x$, its inverse function (e.g. `plot(ys,xs)`), and the function $\log_2(x)$. +::: Note that we needed a bit of care to plot the inverse function directly, as the domain of $f$ is *not* the domain of $f^{-1}$. Again, in this case the domain of $f$ is all $x$, but the domain of $f^{-1}$ is only all *positive* $x$ values. @@ -318,7 +309,7 @@ If $1/10$ of the original carbon $14$ remains, how old is the item? This amounts -5730 * log2(1/10) ``` -:::{.callout-note} +::: {.callout-note} ## Note (Historically) Libby and James Arnold proceeded to test the radiocarbon dating theory by analyzing samples with known ages. For example, two samples taken from the tombs of two Egyptian kings, Zoser and Sneferu, independently dated to $2625$ BC plus or minus $75$ years, were dated by radiocarbon measurement to an average of $2800$ BC plus or minus $250$ years. These results were published in Science in $1949$. Within $11$ years of their announcement, more than $20$ radiocarbon dating laboratories had been set up worldwide. Source: [Wikipedia](http://tinyurl.com/p5msnh6). @@ -327,15 +318,19 @@ If $1/10$ of the original carbon $14$ remains, how old is the item? This amounts ### Properties of logarithms -The basic graphs of logarithms ($a > 1$) are all similar, though as we see larger bases lead to slower growing functions, though all satisfy $\log_a(1) = 0$: - +The basic graphs of logarithms ($a > 1$) are all similar with growth from $-\infty$ to $\infty$ as $x$ goes from just bigger than $0$ towards $\infty$. @fig-basic-logarithmic-graphs shows larger bases lead to slower growing functions, though all satisfy $\log_a(1) = 0$: +::: {#fig-basic-logarithmic-graphs} ```{julia} -plot(log2, 1/2, 10, label="2") # base 2 -plot!(log, 1/2, 10, label="e") # base e -plot!(log10, 1/2, 10, label="10") # base 10 +#| echo: false +plot(log2, 1/5, 10, label="log2") # base 2 +plot!(log, 1/5, 10, label="log") # base e +plot!(log10, 1/5, 10, label="log10") # base 10 ``` +Graphs of `log2`, `log` and `log10` +::: + Now, what do the properties of exponents imply about logarithms? @@ -384,7 +379,11 @@ a^{(\log_b(x)/\log_b(a))} = (b^{\log_b(a)})^{(\log_b(x)/\log_b(a))} = b^{\log_b(a) \cdot \log_b(x)/\log_b(a) } = b^{\log_b(x)} = x. $$ -In short, we have these three properties of logarithmic functions when $a, b$ are positive bases; $u,v$ are positive numbers; and $x$ is any real number: +In short, we have three properties of logarithmic functions. + +::: {.relationship title="Basic properties of logarithmic functions"} + +If $a, b$ are positive bases; $u,v$ are positive numbers; and $x$ is any real number, then $$ @@ -394,6 +393,7 @@ $$ \log_a(u) &= \log_b(u)/\log_b(a). \end{align*} $$ +::: ##### Example @@ -534,7 +534,7 @@ radioq(choices, answ, keep_order=true) ###### Question -The [Loudest band](https://en.wikipedia.org/wiki/Loudest_band) can possibly be measured in [decibels](https://en.wikipedia.org/wiki/Decibel). In $1976$ the Who recorded $126$ db and in $1986$ Motorhead recorded $130$ db. Suppose both measurements record power through the formula $db = 10 \log_{10}(P)$. What is the ratio of the Motorhead $P$ to the $P$ for the Who? +The [Loudest band](https://en.wikipedia.org/wiki/Loudest_band) can possibly quantified by measuring in [decibels](https://en.wikipedia.org/wiki/Decibel). In $1976$ the Who recorded $126$ db and in $1986$ Motorhead recorded $130$ db. Suppose both measurements record power through the formula $db = 10 \log_{10}(P)$. What is the ratio of the Motorhead $P$ to the $P$ for the Who? ```{julia} diff --git a/quarto/precalc/functions.qmd b/quarto/precalc/functions.qmd index d27aac2..8936a06 100644 --- a/quarto/precalc/functions.qmd +++ b/quarto/precalc/functions.qmd @@ -12,32 +12,34 @@ using Plots plotly() ``` - --- A mathematical [function](http://en.wikipedia.org/wiki/Function_(mathematics)) is defined abstractly by: - -> **Function:** A function is a *relation* which assigns to each element in the domain a *single* element in the range. A **relation** is a set of ordered pairs, $(x,y)$. The set of first coordinates is the domain, the set of second coordinates the range of the relation. - +::: {.definition title="A function"} +A function is a *relation* which assigns to each element in the domain a *single* element in the range. A **relation** is a set of ordered pairs, $(x,y)$. The set of first coordinates is the domain, the set of second coordinates the range of the relation. +::: That is, a function gives a correspondence between values in its domain with values in its range. -This definition is abstract, as functions can be very general. With single-variable calculus, we generally specialize to real-valued functions of a single variable (*univariate, scalar functions*). These typically have the correspondence given by a rule, such as $f(x) = x^2$ or $f(x) = \sqrt{x}$. The function's domain may be implicit (as in all $x$ for which the rule is defined) or may be explicitly given as part of the rule. The function's range is then the image of its domain, or the set of all $f(x)$ for each $x$ in the domain ($\{f(x): x \in \text{ domain}\}$). +This definition is abstract, as functions can be very general. With single-variable calculus, we generally specialize to real-valued functions of a single variable (*univariate, scalar-valued functions*). These typically have the correspondence given by a rule, such as $f(x) = x^2$ or $f(x) = \sqrt{x}$. The function's domain may be implicit (as in all $x$ for which the rule is defined) or may be explicitly given as part of the rule. The function's range is then the image of its domain, or the set of all $f(x)$ for each $x$ in the domain---$\{f(x): x \in \text{ domain}\}$. Some examples of mathematical functions are: $$ -f(x) = \cos(x), \quad g(x) = x^2 - x, \quad h(x) = \sqrt{x}, \quad -s(x) = \begin{cases} -1 & x < 0\\1&x>0\end{cases}. +\begin{array}{rclrcl} +f(x) &=& \cos(x),& g(x) &=& x^2 - x, \\ +h(x) &=& \sqrt{x},& +s(x) &=& \begin{cases} -1 & x < 0\\1&x>0\end{cases}. +\end{array} $$ -For these examples, the domain of both $f(x)$ and $g(x)$ is all real values of $x$, where as for $h(x)$ it is implicitly just the set of non-negative numbers, $[0, \infty)$. Finally, for $s(x)$, we can see that the domain is defined for every $x$ but $0$. +For these examples, the domain for both $f(x)$ and $g(x)$ is all real values of $x$, whereas for $h(x)$ it is implicitly just the set of non-negative numbers, $[0, \infty)$. Finally, for $s(x)$, we can see that the domain is defined for every $x$ but $0$. In general the range is harder to identify than the domain, and this is the case for these functions too. For $f(x)$ we may know the $\cos$ function is trapped in $[-1,1]$ and it is intuitively clear than all values in that set are possible. The function $h(x)$ would have range $[0,\infty)$. The $s(x)$ function is either $-1$ or $1$, so only has two possible values in its range. What about $g(x)$? It is a parabola that opens upward, so any $y$ values below the $y$ value of its vertex will not appear in the range. In this case, the symmetry indicates that the vertex will be at $(1/2, -1/4)$, so the range is $[-1/4, \infty)$. @@ -145,11 +147,11 @@ This figure shows that the domain of a function may be a collection of intervals :::{.callout-note} ## Note **Thanks to Euler (1707-1783):** The formal idea of a function is a relatively modern concept in mathematics. According to [Dunham](http://www.maa.org/sites/default/files/pdf/upload_library/22/Ford/dunham1.pdf), +Euler defined a function as an "analytic expression composed in any way whatsoever of the variable quantity and numbers or constant quantities." He goes on to indicate that as Euler matured, so did his notion of function, ending up closer to the modern idea of a correspondence not necessarily tied to a particular formula or “analytic expression.” He finishes by saying: "It is fair to say that we now study functions in analysis because of him." + ::: -Euler defined a function as an "analytic expression composed in any way whatsoever of the variable quantity and numbers or constant quantities." He goes on to indicate that as Euler matured, so did his notion of function, ending up closer to the modern idea of a correspondence not necessarily tied to a particular formula or “analytic expression.” He finishes by saying: "It is fair to say that we now study functions in analysis because of him." - We will see that defining functions within `Julia` can be as simple a concept as Euler started with, but that the more abstract concept has a great advantage that is exploited in the design of the language. @@ -181,9 +183,23 @@ For typical cases like the three above, there isn't really much new to learn. :::{.callout-note} ## The equals sign is used differently between math and Julia -The equals sign in `Julia` always indicates either an assignment or a mutation of the object on the left side. The definition of a function above is an *assignment*, in that a function is added (or modified) in a table holding the methods associated with the function's name. -The equals sign restricts the expressions available on the *left*-hand side to a) a variable name, for assignment; b) mutating an object at an index, as in `xs[1]`; c) mutating a property of a struct; or d) a function assignment following this form `function_name(args...)`. +In math, the equals sign is often associated with an equation. +However, the equals sign in `Julia` always indicates either an +assignment or a mutation of the object on the left side. The +definition of a function above is an *assignment*, in that a function +is added (or modified) in a table holding the methods associated with +the function's name. + +The equals sign restricts the expressions available on the *left*-hand side to either + +* a variable name, for assignment; + +* mutating an object at an index, as in `xs[1]`; + +* mutating a property of a struct; or + +* a function assignment following this form `function_name(args...)`. Whereas function definitions and usage in `Julia` mirrors standard math notation; equations in math are not so mirrored in `Julia`. In mathematical equations, the left-hand of an equation is typically a complicated algebraic expression. Not so with `Julia`, where the left hand side of the equals sign is prescribed and quite limited. @@ -200,7 +216,7 @@ Functions in `Julia` have an implicit domain, just as they do mathematically. In h(-1) ``` -The `DomainError` is one of many different error types `Julia` has, in this case it is quite apt: the value $-1$ is not in the domain of the function. +The `DomainError` is one of many different error types `Julia` has, in this case it is quite apt---the value $-1$ is not in the domain of the function. ### Equations, functions, calling a function @@ -344,7 +360,7 @@ else end ``` -The conditions for the `if` statements are expressions that evaluate to either `true` or `false`, such as generated by the Boolean operators `<`, `<=`, `==`, `!=`, `>=`, and `>`. +The conditions for the `if` statements are expressions that evaluate to either `true` or `false`, such as generated by the comparison operators `<`, `<=`, `==`, `!=`, `>=`, and `>`. If familiar with `if` conditions, they are natural to use. However, for simpler cases of "if-else" `Julia` provides the more convenient *ternary* operator: `cond ? if_true : if_false`. (The name comes from the fact that there are three arguments specified.) The ternary operator checks the condition and if true returns the first expression, whereas if the condition is false the second condition is returned. (Another useful control flow construct is [short-circuit](https://docs.julialang.org/en/v1/manual/control-flow/#Short-Circuit-Evaluation) evaluation.) @@ -400,8 +416,10 @@ The function `s(x)` isn't quite so easy to implement, as there isn't an "otherwi ```{julia} -s(x) = x < 0 ? -1 : - x > 0 ? 1 : error("0 is not in the domain") +s(x) = + x < 0 ? -1 : + x > 0 ? 1 : + error("0 is not in the domain") ``` With nested ternary operators, the advantage over the `if` condition is not always compelling, but for simple cases the ternary operator is quite useful. @@ -469,12 +487,11 @@ trajectory(100) By using a multi-line function our work is much easier to look over for errors. -##### Example: the secant method for finding a solution to $f(x) = 0$. - - This next example, shows how using functions to collect a set of computations for simpler reuse can be very helpful. +##### Example: the secant method for finding a solution to $f(x) = 0$. + An old method for finding a zero of an equation is the [secant method](https://en.wikipedia.org/wiki/Secant_method). We illustrate the method with the function $f(x) = x^2 - 2$. In an upcoming example we see how to create a function to evaluate the secant line between $(a,f(a))$ and $(b, f(b))$ at any point. In this example, we define a function to compute the $x$ coordinate of where the secant line crosses the $x$ axis. This can be defined as follows: @@ -556,38 +573,41 @@ end Now our guess $c$ is basically the same as `sqrt(2)`. Repeating the above leads to only a slight improvement in the guess, as we are about as close as floating point values will allow. +In most cases, this method can fairly quickly find a zero provided two good starting points are used. -Here we see a visualization with all these points. As can be seen, it quickly converges at the scale of the visualization, as we can't see much closer than `1e-2`. + +@fig-secant-method-illustration shows a visualization with all these points. As can be seen, it quickly converges at the scale of the visualization, as we can't see much closer than `1e-2`. + +::: {#fig-secant-method-illustration} ```{julia} -#| hold: true #| echo: false f(x) = x^2 - 2 a, b = 1, 2 c = secant_intersection(f, a, b) -p = plot(f, a, b, linewidth=5, legend=false) -plot!(p, zero, a, b) -scatter!([a,b], [f(a), f(b)]; marker=(:square,)) +plt = plot(f, a, b, linewidth=5, legend=false) +plot!(plt, zero, a, b) +scatter!(plt, [(a, f(a)), (b, f(b))]; marker=(:square,)) -plot!(p, [a,b], f.([a,b])); -scatter!(p, [c], [f(c)]) +plot!(plt, [(a, f(a)), (b, f(b))]) +scatter!(plt, [(c, f(c))]) a, b = b, c c = secant_intersection(f, a, b) -plot!(p, [a,b], f.([a,b])); -scatter!(p, [c], [f(c)]) +plot!(plt, [(a, f(a)), (b, f(b))]) +scatter!(plt, [(c, f(c))]) a, b = b, c c = secant_intersection(f, a, b) -plot!(p, [a,b], f.([a,b])); -scatter!(p, [c], [f(c)]) -p +plot!(plt, [(a, f(a)), (b, f(b))]) +scatter!(plt, [(c, f(c))]) +plt ``` +::: -In most cases, this method can fairly quickly find a zero provided two good starting points are used. ## Parameters, function context (scope), keyword arguments @@ -605,7 +625,9 @@ learn that this is just a dummy variable to be substituted for and so could have any name. Both also share a variable $m$ for a slope. Where does the value for $m$ come from? -In practice, there is a context that gives an answer. Despite the same name, there is no expectation that the slope will be the same for each function if the context is different. So when parameters are involved, a function involves a rule and a context to give specific values to the parameters. Euler had said initially that functions composed of "the variable quantity and numbers or constant quantities." The term "variable," we still use, but instead of "constant quantities," we use the name "parameters." In computer language, instead of context, we use the word *scope*. +In practice, there is a context that gives an answer. Despite the same name, there is no expectation that the slope will be the same for each function if the context is different. So when parameters are involved, a function involves a rule and a context to give specific values to the parameters. + +Euler had said initially that functions composed of "the variable quantity and numbers or constant quantities." The term "variable," we still use, but instead of "constant quantities," we use the name "parameters." In computer language, instead of context, we use the word *scope*. Something similar is also true with `Julia`. Consider the example of writing a function to model a linear equation with slope $m=2$ and $y$-intercept $3$. A typical means to do this would be to define constants, and then use the familiar formula: @@ -675,10 +697,12 @@ mxplusb(0; m=3, b=2) Keywords are used to mark the parameters whose values are to be changed from the default. Though one can use *positional arguments* for parameters---and there are good reasons to do so---using keyword arguments is a good practice if performance isn't paramount, as their usage is more explicit yet the defaults mean that a minimum amount of typing needs to be done. -Keyword arguments are widely used with plotting commands, as there are numerous options to adjust, but typically only a handful adjusted per call. The `Plots` package whose commands we illustrate throughout these notes starting with the next section has this in its docs: `Plots.jl` follows two simple rules with data and attributes: +Keyword arguments are widely used with plotting commands, as there are numerous options to adjust, but typically only a handful adjusted per call. The `Plots` package whose commands we illustrate throughout these notes starting with the next section has this in its docs: -* Positional arguments correspond to input data -* Keyword arguments correspond to attributes +> `Plots.jl` follows two simple rules with data and attributes: +> +> * Positional arguments correspond to input data +> * Keyword arguments correspond to attributes @@ -732,7 +756,7 @@ The *big* advantage of bundling parameters into a container is consistency-–-t ::: {.callout-note} ## Avoid global variables -Referring to a global parameter is common in math, but has a significant performance impact in `Julia`. Save for the simplest usage, it is much better to pass parameters to the function through one of several means that too rely on the value of the global state. +Referring to a global parameter is common in math, but has a significant performance impact in `Julia`. Save for the simplest usage, it is much better to pass parameters to the function through one of means just illustrated. ::: @@ -749,22 +773,22 @@ Volume(r, h) = pi * r^2 * h # of a cylinder SurfaceArea(r, h) = pi * r * (r + sqrt(h^2 + r^2)) # of a right circular cone, including the base ``` -The right-hand sides may or may not be familiar, but it should be reasonable to believe that if push came to shove, the formulas could be looked up. However, the left-hand sides are subtly different---they have two arguments, not one. In `Julia` it is trivial to define functions with multiple arguments---we just did. +The right-hand sides may or may not be familiar, but it should be reasonable to believe that if push came to shove, the formulas could be looked up. However, the left-hand sides are subtly different---they have two arguments, not one. In `Julia` it is non-eventful to define functions with multiple arguments---we just did. -Earlier we saw the `log` function can use a second argument to express the base. This function is basically defined by `log(b,x)=log(x)/log(b)`. The `log(x)` value is the natural log, and this definition just uses the change-of-base formula for logarithms. +In a previous section we saw the `log` function can use a second argument to express the base. This function is basically defined by `log(b,x)=log(x)/log(b)`. The `log(x)` value is the natural log, and this definition just uses the change-of-base formula for logarithms. -But not so fast, on the left side is a function with two arguments and on the right side the functions have one argument---yet they share the same name. How does `Julia` know which to use? `Julia` uses the number, order, and *type* of the positional arguments passed to a function to determine which function definition to use. This is technically known as [multiple dispatch](http://en.wikipedia.org/wiki/Multiple_dispatch) or **polymorphism**. As a feature of the language, it can be used to greatly simplify the number of functions the user must learn. The basic idea is that many functions are "generic" in that they have methods which will work differently in different scenarios. +But not so fast, on the left side is a function with two arguments and on the right side the functions have one argument---yet they share the same name. How does `Julia` know which to use? `Julia` uses the number, order, and *type* of the positional arguments passed to a function to determine which function definition to use. This is technically known as [multiple dispatch](http://en.wikipedia.org/wiki/Multiple_dispatch) or *polymorphism*. As a feature of the language, it can be used to greatly simplify the number of functions the user must learn. The basic idea is that many functions are "generic" in that they have methods which will work differently in different scenarios but still compute the same general thing. -:::{.callout-warning} -## Warning +:::{.callout-note} +## Multiple dispatch is familiar Multiple dispatch is very common in mathematics. For example, we learn different ways to add: integers (fingers, carrying), real numbers (align the decimal points), rational numbers (common denominators), complex numbers (add components), vectors (add components), polynomials (combine like monomials), ... yet we just use the same `+` notation for each operation. The concepts are related, the details different. ::: -`Julia` is similarly structured. `Julia` terminology would be to call the operation "`+`" a *generic function* and the different implementations *methods* of "`+`". This allows the user to just need to know a smaller collection of generic concepts yet still have the power of detail-specific implementations. To see how many different methods are defined in the base `Julia` language for the `+` operator, we can use the command `methods(+)`. As there are so many (well over $100$ when `Julia` is started), we illustrate how many different logarithm methods are implemented for "numbers:" +`Julia` is similarly structured. `Julia` terminology would be to call the operation "`+`" a *generic function* and the different implementations *methods* of "`+`". This allows the user to just need to know a smaller collection of generic concepts yet still have the power of detail-specific implementations. To see how many different methods are defined in the base `Julia` language for the `+` operator, we can use the command `methods(+)`. As there are so many (well over $100$ when `Julia` is started), we illustrate how many different logarithm methods are implemented for "numbers": ```{julia} @@ -818,35 +842,36 @@ But the other fact about this problem---that the perimeter is $20$---means that height(w) = (20 - 2*w)/2 ``` -By hand we would substitute this last expression into that for the area and simplify (to get $A=w\cdot (20-2 \cdot w)/2 = -w^2 + 10w$). However, within `Julia` we can let *composition* do the substitution and leave the algebraic simplification for `Julia` to do: +By hand we would *substitute* this last expression into that for the area and simplify (to get $A=w\cdot (20-2 \cdot w)/2 = -w^2 + 10w$). However, within `Julia` we can let *composition* do the substitution and leave the algebraic simplification for `Julia` to do: ```{julia} Area(w) = Area(w, height(w)) ``` + This might seem odd, just like with `log`, we now have two *different* but related functions named `Area`. Julia will decide which to use based on the number of arguments when the function is called. This setup allows both to be used on the same line, as above. This usage style is not so common with many computer languages, but is a feature of `Julia` which is built around the concept of *generic* functions with multiple dispatch rules to decide which rule to call. -For example, jumping ahead a bit, the `plot` function of `Plots` expects functions of a single numeric variable. Behind the scenes, then the function `A(w)` will be used in this graph: +For example, jumping ahead a bit, the `plot` function of `Plots` expects functions of a single numeric variable. Behind the scenes, then the function `A(w)` will be used in forming @fig-plot-of-area-to-see-how-easy-plotting-is. From the graph of the `Area` function we can see that the width for maximum area is $w=5$ and so $h=5$ as well. +::: {#fig-plot-of-area-to-see-how-easy-plotting-is} ```{julia} plot(Area, 0, 10) ``` - -From the graph, we can see that the width for maximum area is $w=5$ and so $h=5$ as well. +Plot of the `Area` function (of a single variable) over $[0,10]$ +::: -## Other types of functions + +## Anonymous functions `Julia` has both *generic* functions *and* *anonymous* functions. Generic functions participate in *multiple dispatch*, a central feature of `Julia`. Anonymous functions are very useful with higher-order programming (passing functions as arguments). These notes occasionally take advantage of anonymous functions for convenience. -### Anonymous functions - Simple mathematical functions have a domain and range which are a subset of the real numbers, and generally have a concrete mathematical rule. However, the definition of a function is much more abstract. We've seen that functions for computer languages can be more complicated too, with, for example, the possibility of multiple input values. Things can get more abstract still. @@ -917,7 +942,9 @@ Generic versus anonymous functions. Julia has two types of functions, generic on This comes up when we use functions that return functions as we have different styles that can be used: When we defined `l = shift_right(f, c=3)` the value of `l` is assigned to name an anonymous function for later use. This binding can be reused to define other variables. -However, we could have defined the function `l` through `l(x) = shift_right(f, c=3)(x)`, being explicit about what happens to the variable `x`. This would add a method to the generic function `l`. Meaning, we get an error if we tried to assign a variable to `l`, such as an expression like `l=3`. The latter style is inefficient, so is not preferred. +However, we could have defined the function `l` through `l(x) = shift_right(f, c=3)(x)`, being explicit about what happens to the variable `x`. This would add a method to the generic function `l`. Meaning, we get an error if we tried to assign a variable to `l`, such as an expression like `l=3`. + +The latter style is inefficient, so is not preferred. ::: @@ -975,7 +1002,7 @@ specific_line(m,b) = x -> mxplusb(x; m=m, b=b) The returned object will have its parameters (`m` and `b`) fixed when used. -In `Julia`, the functions `Base.Fix1` and `Base.Fix2` are provided to take functions of two variables and create callable objects of just one variable, with the other argument fixed. This partial function application is provided by a some of the logical comparison operators, which can be useful with filtering, say. +In `Julia`, the functions `Base.Fix1` and `Base.Fix2` are provided to take functions of two variables and create callable objects of just one variable, with the other argument fixed. This partial function application is provided by some of the logical comparison operators, which can be useful with filtering, say. For example, `<(2)` is a funny looking way of expressing the function `x -> x < 2`. (Think of `x < y` as `<(x,y)` and then "fix" the value of `y` to be `2`.) This is useful with filtering by a predicate function, for example: @@ -1001,7 +1028,7 @@ In Julia v1.12 the `Fix` constructor can fix an arbitrary position of a variadic Many functions in `Julia` accept a function as the first argument. A common pattern for calling some function is `action(f, args...)` where `action` is the function that will act on another function `f` using the value(s) in `args...`. There `do` notation is syntactical sugar for creating an anonymous function which is useful when more complicated function bodies are needed. -Here is an artificial example to illustrate of a task we won't have cause to use in these notes, but is an important skill in some contexts. The `do` notation can be confusing to read, as it moves the function definition to the end and not the beginning, but is convenient to write and is used very often with the task of this example. +Here is an artificial example to illustrate of a task we won't have cause to use in these notes. The `do` notation can be confusing to read, as it moves the function definition to the end and not the beginning, but is convenient to write and is used very often with the task of this example. To save some text to a file requires a few steps: opening the file; writing to the file; closing the file. The `open` function does the first. One method has this signature `open(f::Function, args...; kwargs....)` and is documented to "Apply the function f to the result of `open(args...; kwargs...)` and close the resulting file descriptor upon completion." Which is great, the open and close stages are handled by `Julia` and only the writing is up to the user. @@ -1051,7 +1078,7 @@ For heavy use of chaining through function application there are various package ## Questions -##### Question +###### Question State the domain and range of $f(x) = |x + 2|$. @@ -1066,11 +1093,11 @@ choices = [ "Domain is all non-negative numbers, range is all real numbers", "Domain is all non-negative numbers, range is all non-negative numbers" ] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` -##### Question +###### Question State the domain and range of $f(x) = 1/(x-2)$. @@ -1085,11 +1112,11 @@ L"Domain is all real numbers except $2$, range is all real numbers except $0$", L"Domain is all non-negative numbers except $0$, range is all real numbers except $2$", L"Domain is all non-negative numbers except $-2$, range is all non-negative numbers except $0$" ] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` -##### Question +###### Question Which of these functions has a domain of all real $x$, but a range of $x > 0$? @@ -1103,11 +1130,11 @@ raw"``f(x) = 2^x``", raw"``f(x) = 1/x^2``", raw"``f(x) = |x|``", raw"``f(x) = \sqrt{x}``"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` -##### Question +###### Question ::: {#fig-floor-function} @@ -1172,7 +1199,7 @@ choices = [ "The integers" ] answer = 1 -radioq(choices, answ) +radioq(choices, answer) ``` From the graph, what is the range of the function? @@ -1187,7 +1214,7 @@ choices = [ "The integers" ] answer = 3 -radioq(choices, answ) +radioq(choices, answer) ``` (This graphic uses the convention that a filled in point is present, but an open point is not, hence each bar represents some $[k, k+1)$.) @@ -1206,8 +1233,8 @@ q"function f(x) = sin(x + pi/3)", q"f(x) = sin(x + pi/3)", q"f: x -> sin(x + pi/3)", q"f x = sin(x + pi/3)"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` ###### Question @@ -1225,8 +1252,8 @@ q"f(x) := (1 + x^2)^(-1)", q"f[x] = (1 + x^2)^(-1)", q"def f(x): (1 + x^2)^(-1)" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1386,8 +1413,8 @@ Will the call `C(1, mu=70)` use a value of `70` for `mu`? #| echo: false choices = ["Yes, this will work just as it does for keyword arguments", "No, there will be an error that the function does not accept keyword arguments"] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question @@ -1411,8 +1438,8 @@ choices = [ "If `x` is in `[a,b]` it returns `x`, otherwise it returns `NaN`", "`x` is the larger of the minimum of `x` and `a` and the value of `b`, aka `max(min(x,a),b)`" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1446,8 +1473,8 @@ L"You still get $0.649...$", "You get a `MethodError`, as `cos(pi/4)` is evaluated as a number and `∘` is not defined for functions and numbers", "You get a `generic` function, but this won't be callable. If tried, it will give an method error." ] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` :::{.callout-note} @@ -1468,8 +1495,8 @@ choices = [ "It is `0.6663667453928805`, the same as `cos(sin(1))`", "It is `0.5143952585235492`, the same as `sin(cos(1))`", "It gives an error"] -answ = 1 -radioq(choices, answ, keep_order=true) +answer = 1 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -1488,8 +1515,8 @@ fn(3) #| hold: true #| echo: false choices = ["`true`","`false`"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -1516,8 +1543,8 @@ Repeat the secant method two more times to find a better approximation for $\sqr #| hold: true #| echo: false choices = [q"4//3", q"7//5", q"58//41", q"816//577"] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` How small is the value of $f(c)$ for this value? @@ -1537,8 +1564,8 @@ How close is this answer to the true value of $\sqrt{2}$? #| hold: true #| echo: false choices = [L"about $8$ parts in $100$", L"about $1$ parts in $100$", L"about $4$ parts in $10,000$", L"about $2$ parts in $1,000,000$"] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` (Finding a good approximation to $\sqrt{2}$ would be helpful to builders, for example, as it could be used to verify the trueness of a square room, say.) @@ -1557,8 +1584,8 @@ choices = ["Just use `f = h - g`", "Define `f(x) = h(x) - g(x)`", "Use `x -> h(x) - g(x)` when the difference is needed" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question diff --git a/quarto/precalc/inversefunctions.qmd b/quarto/precalc/inversefunctions.qmd index 51dca2d..5f38948 100644 --- a/quarto/precalc/inversefunctions.qmd +++ b/quarto/precalc/inversefunctions.qmd @@ -3,13 +3,12 @@ {{< include ../_common_code.qmd >}} -In this section we will use these add-on packages: +In this section we will use this add-on package: ```{julia} -using CalculusWithJulia using Plots -plotly() +plotly(); ``` --- @@ -37,8 +36,9 @@ Why is this useful? When available, it can help us solve equations. If we can wr Let's explore when we can "solve" for an inverse function. -Consider this graph of the function $f(x) = 2^x$ +Consider @fig-plot-2-x-showing-x-to-y-and-y-to-x showing a graph of the function $f(x) = 2^x$. +::: {#fig-plot-2-x-showing-x-to-y-and-y-to-x} ```{julia} #| echo: false p = let @@ -84,12 +84,15 @@ plotly() p ``` -The graph of a function is a representation of points $(x,f(x))$, so to *find* $y = f(c)$ from the graph, we begin on the $x$ axis at $c$, move vertically to the graph (the point $(c, f(c))$), and then move horizontally to the $y$ axis, intersecting it at $y = f(c)$. The figure shows this for $c=2$, from which we can read that $f(c)$ is about $4$. This is how an $x$ is associated to a single $y$. +Plot of $f(x) = 2^x$ illustrating how to graphically start from $x=c$ to find $f(c)$ and how to start from $y=f(d)$ to find $d$ +::: + +The graph of a function is a representation of points $(x,f(x))$, so to *find* $y = f(c)$ from the graph, we begin on the $x$ axis at $c$, move vertically to the graph (the point $(c, f(c))$), and then move horizontally to the $y$ axis, intersecting it at $y = f(c)$. This is how an $x$ is associated to a single $y$. If we were to *reverse* the direction, starting at $y = f(d)$ on the $y$ axis and then moving horizontally to the graph, and then vertically to the $x$-axis we end up at a value $d$ with the correct value of $f(d)$. This allows solving for $x$ knowing $y$ in $y=f(x)$. -The operation described will form a function **if** the initial movement horizontally is guaranteed to find *no more than one* value on the graph. That is, to have an inverse function, there can not be two $x$ values corresponding to a given $y$ value. This observation is often visualized through the "horizontal line test" - the graph of a function with an inverse function can only intersect a horizontal line at most in one place. +The operation described will form a function **if** the initial movement horizontally is guaranteed to find *no more than one* value on the graph. That is, to have an inverse function, there can not be two $x$ values corresponding to a given $y$ value. This observation is often visualized through the "horizontal line test"---the graph of a function with an inverse function can only intersect a horizontal line at most in one place. More formally, a function is called *one-to-one* *if* for any two $a \neq b$, it must be that $f(a) \neq f(b)$. Many functions are one-to-one, many are not. Familiar one-to-one functions are linear functions ($f(x)=a \cdot x + b$ with $a\neq 0$), odd powers of $x$ ($f(x)=x^{2k+1}$), and functions of the form $f(x)=x^{1/n}$ for $x \geq 0$. In contrast, all *even* functions are *not* one-to-one, as $f(x) = f(-x)$ for any nonzero $x$ in the domain of $f$. @@ -183,7 +186,7 @@ y - 2 &= (x-1)^5\\ \end{align*} $$ -We see that $f^{-1}(x) = 1 + (x - 2)^{1/5}$. The fact that the power $5$ is an odd power is important, as this ensures a unique (real) solution to the fifth root of a value, in the above $y-2$. +We see that $f^{-1}(x) = (x - 2)^{1/5} + 1$. The fact that the power $5$ is an odd power is important, as this ensures a unique (real) solution to the fifth root of a value, in the above $y-2$. In the section on [polynomial roots](../precalc/polynomial_roots.html) we introduce the `solve` function of `SymPy`, which can algebraically solve for inverse functions in easier cases. @@ -196,7 +199,7 @@ In the section on [polynomial roots](../precalc/polynomial_roots.html) we intro The function $f(x) = x^x, x \geq 1/e$ is strictly increasing. However, trying to algebraically solve for an inverse function will quickly run into problems (without using specially defined functions). The existence of an inverse does not imply there will always be luck in trying to find a mathematical rule defining the inverse. -In the section on the [intermediate value theorem](../limits/intermediate_value_theorem.html#the-find_zero-function) we will see how to *numerically* solve for an inverse function. +In the section on the intermediate value theorem we will see how to *numerically* solve for an inverse function. ## Functions which are not always invertible @@ -205,7 +208,7 @@ In the section on the [intermediate value theorem](../limits/intermediate_value_ Consider the function $f(x) = x^2$. The graph---a parabola---is clearly not *monotonic*. Hence no inverse function exists. Yet, we can solve equations $y=x^2$ quite easily: $y=\sqrt{x}$ *or* $y=-\sqrt{x}$. We know the square root undoes the squaring, but we need to be a little more careful to say the square root is the inverse of the squaring function. -The issue is there are generally *two* possible answers. To avoid this, we might choose to only take the *non-negative* answer. To make this all work as above, we restrict the domain of $f(x)$ and now consider the related function $f(x)=x^2, x \geq 0$. This is now a monotonic function, so will have an inverse function. This is clearly $f^{-1}(x) = \sqrt{x}$. (The $\sqrt{x}$ being defined as the principle square root or the unique *non-negative* answer to $u^2-x=0$.) +The issue in this case is there are generally *two* possible answers. To avoid this, we might choose to only take the *non-negative* answer. To make this all work as above, we restrict the domain of $f(x)$ and now consider the related function $f(x)=x^2, x \geq 0$. This is now a monotonic function, so will have an inverse function. This is clearly $f^{-1}(x) = \sqrt{x}$. (The $\sqrt{x}$ being defined as the principle square root or the unique *non-negative* answer to $u^2-x=0$.) The [inverse function theorem](https://en.wikipedia.org/wiki/Inverse_function_theorem) basically says that if $f$ is *locally* monotonic, then an inverse function will exist *locally*. By "local" we mean in a neighborhood of $c$. @@ -234,7 +237,7 @@ Then $f^{-1}(x) = \sqrt{(1-x)/x}$ where $0 < x \leq 1$. The somewhat complicated Consider again the graph of a monotonic function, in this case $f(x) = x^2 + 2, x \geq 0$: - +::: {#fig-plot-xsquared-plus-2-domain-range} ```{julia} #| hold: true f(x) = x^2 + 2 @@ -243,25 +246,31 @@ plot(f, 0, 4; yticks=[2,4,8,16], plot!([(2,0), (2, f(2)), (0, f(2))]) ``` +Plot of $f(x) = x^2 + 2$ over $[0,4]$ +::: + The graph is shown over the interval $(0,4)$, but the *domain* of $f(x)$ is all $x \geq 0$. The *range* of $f(x)$ is clearly $2 \leq y \leq \infty$. The lines layered on the plot show how to associate an $x$ value to a $y$ value or vice versa (as $f(x)$ is one-to-one). The domain then of the inverse function is all the $y$ values for which a corresponding $x$ value exists: this is clearly all values bigger or equal to $2$. The *range* of the inverse function can be seen to be all the images for the values of $y$, which would be all $x \geq 0$. This gives the relationship: -> * the *domain* of $f^{-1}(x)$ is the *range* of $f(x)$; -> * the *range* of $f^{-1}(x)$ is the *domain* of $f(x)$; +::: {.relationship title="The domain and range"} +The *domain* of $f^{-1}(x)$ is the *range* of $f(x)$. + +The *range* of $f^{-1}(x)$ is the *domain* of $f(x)$. +::: From this we can see if we start at $x$, apply $f$ we get $y$, if we then apply $f^{-1}$ we will get back to $x$ so we have: -> For all $x$ in the domain of $f$: $f^{-1}(f(x)) = x$. +::: {.relationship title = "Composition with an inverse"} +For all $x$ in the domain of $f$: $f^{-1}(f(x)) = x$. -Similarly, were we to start on the $y$ axis, we would see: - -> For all $x$ in the domain of $f^{-1}$: $f(f^{-1}(x)) = x$. +For all $x$ in the domain of $f^{-1}$: $f(f^{-1}(x)) = x$. In short $f^{-1} \circ f$ and $f \circ f^{-1}$ are both identity functions, though on possibly different domains. +::: ## The graph of the inverse function @@ -269,37 +278,41 @@ In short $f^{-1} \circ f$ and $f \circ f^{-1}$ are both identity functions, thou The graph of $f(x)$ is a representation of all values $(x,y)$ where $y=f(x)$. As the inverse flips around the role of $x$ and $y$ we have: - -> If $(x,y)$ is a point on the graph of $f(x)$, then $(y,x)$ will be a point on the graph of $f^{-1}(x)$. - +::: {.relationship title="Mirror points"} + If $(x,y)$ is a point on the graph of $f(x)$, then $(y,x)$ will be a point on the graph of $f^{-1}(x)$. +::: Let's see this in action. Take the function $2^x$. We can plot it by generating points to plot as follows: - +::: {#fig-plot-2-to-x-and-inverse-with-xs-ys} ```{julia} #| hold: true f(x) = 2^x xs = range(0, 2, length=50) ys = f.(xs) -plot(xs, ys; color=:blue, label="f", - aspect_ratio=:equal, framestyle=:origin, xlims=(0,4)) +plot(xs, ys; aspect_ratio=:equal, framestyle=:origin, + xlims=(0,4), + color=:blue, label="f") plot!(ys, xs; color=:red, label="f⁻¹") # the inverse ``` +Plot of $f(x) = 2^x$ using `plot(xs, ys)` and its inverse produced with `plot!(ys, xs)` +::: + By flipping around the $x$ and $y$ values in the `plot!` command, we produce the graph of the inverse function---when viewed as a function of $x$. We can see that the domain of the inverse function (in red) is clearly different from that of the function (in blue). -The inverse function graph can be viewed as a symmetry of the graph of the function. Flipping the graph for $f(x)$ around the line $y=x$ will produce the graph of the inverse function: Here we see for the graph of $f(x) = x^{1/3}$ and its inverse function: - +The inverse function graph can be viewed as a symmetry of the graph of the function. Flipping the graph for $f(x)$ around the line $y=x$ will produce the graph of the inverse function: @fig-plot-cbrt-and-its-inverse show the graph of $f(x) = x^{1/3}$ and its inverse function: +::: {#fig-plot-cbrt-and-its-inverse} ```{julia} #| hold: true f(x) = cbrt(x) xs = range(-2, 2, length=150) ys = f.(xs) -plot(xs, ys; color=:blue, - aspect_ratio=:equal, legend=false) +plot(xs, ys; aspect_ratio=:equal, legend=false, + line=(:blue,)) plot!(ys, xs; line=(:red,)) plot!(identity; line=(:green, :dash)) x = 1/4 @@ -307,11 +320,15 @@ y = f(x) plot!([(x,y), (y,x)]; line=(:green, :dot)) ``` -We drew a line connecting $(1/4, f(1/4))$ to $(f(1/4),1/4)$. We can see that it crosses the line $y=x$ perpendicularly, indicating that points are symmetric about this line. (The plotting argument `aspect_ratio=:equal` ensures that the $x$ and $y$ axes are on the same scale, so that this type of line will look perpendicular.) +Plot of $f(x) = x^{1/3}$ and its inverse +::: + +In @fig-plot-cbrt-and-its-inverse we drew a line connecting $(1/4, f(1/4))$ to $(f(1/4),1/4)$. We can see that this line crosses the line $y=x$ perpendicularly, indicating that points are symmetric about the $y=x$ line. (The plotting argument `aspect_ratio=:equal` ensures that the $x$ and $y$ axes are on the same scale, so that this type of line will look perpendicular.) One consequence of this symmetry, is that if $f$ is strictly increasing, then so is its inverse. ::: {.callout-note} +## `cbrt(x)` is different from `x^(1/3)` In the above we used `cbrt(x)` and not `x^(1/3)`. The latter usage assumes that $x \geq 0$ as it isn't guaranteed that for all real exponents the answer will be a real number. The `cbrt` function knows there will always be a real answer and provides it. ::: @@ -321,12 +338,12 @@ In the above we used `cbrt(x)` and not `x^(1/3)`. The latter usage assumes that The slope of $f(x) = 9/5 \cdot x + 32$ is clearly $9/5$ and the slope of the inverse function $f^{-1}(x) = 5/9 \cdot (x-32)$ is clearly $5/9$ - or the reciprocal. This makes sense, as the slope is the rise over the run, and by flipping the $x$ and $y$ values we merely flip over the rise and the run. -Now consider the graph of the *tangent line* to a function. This concept will be better defined later, for now, it is a line "tangent" to the graph of $f(x)$ at a point $x=c$. +Now consider the graph of the *tangent line* to a function. This concept will be better defined later. -For concreteness, we consider $f(x) = \sqrt{x}$ at $c=2$. The tangent line will have slope $1/(2\sqrt{2})$ and will go through the point $(2, f(2))$. We graph the function, its tangent line, and their inverses: - +For concreteness, we consider $f(x) = \sqrt{x}$ at $c=2$. The tangent line will have slope $1/(2\sqrt{2})$ and will go through the point $(2, f(2))$. In @fig-plot-sqrt-x-its-inverse-two-tangent-lines we graph the function, its tangent line, and their inverses. +::: {#fig-plot-sqrt-x-its-inverse-two-tangent-lines} ```{julia} #| hold: true f(x) = sqrt(x) @@ -339,8 +356,13 @@ plot(xs, ys, color=:blue, legend=false) plot!(xs, zs, color=:blue) # the tangent line plot!(ys, xs, color=:red) # the inverse function plot!(zs, xs, color=:red) # inverse of tangent line +scatter!([(c, f(c))], marker=(:blue, 3)) +scatter!([(f(c),c)], marker=(:red, 3)) ``` +Plot of $f(x) = \sqrt{x}$, its inverse, and two related tangent lines +::: + What do we see? In blue, we can see the familiar square root graph along with a "tangent" line through the point $(2, f(2))$. The red graph of $f^{-1}(x) = x^2, x \geq 0$ is seen and, perhaps surprisingly, a tangent line. This is at the point $(f(2), 2)$. We know the slope of this tangent line is the reciprocal of the slope of the red tangent line. This gives this informal observation: @@ -493,8 +515,8 @@ Is it possible that a function have two different inverses? #| echo: false choices = [L"No, for all $x$ in the domain and an inverse, the value of any inverse will be the same, hence all inverse functions would be identical.", L"Yes, the function $f(x) = x^2, x \geq 0$ will have a different inverse than the same function $f(x) = x^2, x \leq 0$"] -answ = 1 -radioq(choices, answ) +answer = 1 +buttonq(choices, answer) ``` ###### Question @@ -509,8 +531,8 @@ A function takes a value $x$ adds $1$, divides by $2$, and then subtracts $1$. I choices = [L"Yes, the function is the linear function $f(x)=(x+1)/2 - 1$ and so is monotonic.", L"No, the function is $1$ then $2$ then $1$, but not \"one-to-one\"" ] -answ = 1 -radioq(choices, answ) +answer = 1 +buttonq(choices, answer) ``` ###### Question @@ -525,8 +547,8 @@ Is the function $f(x) = x^5 - x - 1$ one-to-one? choices=[L"Yes, a graph over $(-100, 100)$ will show this.", L"No, a graph over $(-2,2)$ will show this." ] -answ = 2 -radioq(choices, answ) +answer = 2 +buttonq(choices, answer) ``` ###### Question @@ -560,15 +582,17 @@ yesnoq(false) A function is defined by its graph. - +::: {#fig-function-x-sinx-defined-by-graph} ```{julia} #| hold: true #| echo: false f(x) = x - sin(x) plot(f, 0, 6pi) ``` +Plot of $f(x)$ over $[0, 6\pi]$ +::: -Over the domain shown, is the function one-to-one? +Over the domain shown in @fig-function-x-sinx-defined-by-graph , is the function one-to-one? ```{julia} @@ -590,8 +614,8 @@ What is $g(x) = (f(x))^{-1}$? #| hold: true #| echo: false choices = ["``g(x) = x``", "``g(x) = x^{-1}``"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` What is $g(x) = f^{-1}(x)$? @@ -601,8 +625,8 @@ What is $g(x) = f^{-1}(x)$? #| hold: true #| echo: false choices = ["``g(x) = x``", "``g(x) = x^{-1}``"] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question @@ -674,8 +698,8 @@ L"The function that multiplies by $2$, subtracts $1$ and then squares the value. L"The function that divides by $2$, adds $1$, and then takes the square root of the value.", L"The function that takes square of the value, then subtracts $1$, and finally multiplies by $2$." ] -answ = 1 -radioq(choices, answ) +answer = 1 +buttonq(choices, answer) ``` ###### Question @@ -744,8 +768,8 @@ choices = [ "``f^{-1}(x) = (5x-4)^3``", "``f^{-1}(x) = 5/(x^3 + 4)``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -762,8 +786,8 @@ raw"``f^{-1}(x) = (x-e)^{1/\pi}``", raw"``f^{-1}(x) = (x-\pi)^{e}``", raw"``f^{-1}(x) = (x-e)^{\pi}``" ] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -779,8 +803,8 @@ choices = [ raw"``[7, \infty)``", raw"``(-\infty, \infty)``", raw"``[0, \infty)``"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` ###### Question @@ -796,8 +820,8 @@ choices = [ raw"``[7, \infty)``", raw"``(-\infty, \infty)``", raw"``[0, \infty)``"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` ###### Question @@ -805,7 +829,7 @@ radioq(choices, answ) From the plot, are blue and red inverse functions? - +::: {#fig-plot-of-blue-red-inverse-maybe-1} ```{julia} #| hold: true #| echo: false @@ -816,6 +840,8 @@ plot(xs, ys, color=:blue, legend=false) plot!(ys, xs, color=:red) plot!(x->x, linestyle=:dash) ``` +Plot of two functions, one in blue, one in red along with the line $y=x$. +::: ```{julia} #| hold: true @@ -825,7 +851,7 @@ yesnoq(true) From the plot, are blue and red inverse functions? - +::: {#fig-plot-of-blue-red-inverse-maybe-2} ```{julia} #| hold: true #| echo: false @@ -836,6 +862,8 @@ plot(xs, ys, color=:blue, legend=false) plot!(-xs, -ys, color=:red) plot!(x->x, linestyle=:dash) ``` +Plot of two functions, one in blue, one in red along with the line $y=x$. +::: ```{julia} #| hold: true @@ -849,10 +877,13 @@ yesnoq(false) The function $f(x) = (ax + b)/(cx + d)$ is known as a [Mobius](http://tinyurl.com/oemweyj) transformation and can be expressed as a composition of $4$ functions, $f_4 \circ f_3 \circ f_2 \circ f_1$: - * where $f_1(x) = x + d/c$ is a translation, - * where $f_2(x) = x^{-1}$ is inversion and reflection, - * where $f_3(x) = ((bc-ad)/c^2) \cdot x$ is scaling, - * and $f_4(x) = x + a/c$ is a translation. +* where $f_1(x) = x + d/c$ is a translation, + +* where $f_2(x) = x^{-1}$ is inversion and reflection, + +* where $f_3(x) = ((bc-ad)/c^2) \cdot x$ is scaling, + +* and $f_4(x) = x + a/c$ is a translation. For $x=10$, $a=1$, $b=2$, $c=3$ and $d=5$, what is $f(10)$? @@ -886,8 +917,8 @@ choices = [ L"As $f_4(f_3(f_2(f_1(x))))=(f_1 \circ f_2 \circ f_3 \circ f_4)(x)$", "As the latter is more complicated than the former." ] -answ=1 -radioq(choices, answ) +answer=1 +radioq(choices, answer) ``` Let $g_1$, $g_2$, $g_3$, and $g_4$ denote the inverse functions. Clearly, $g_1(x) = x- d/c$ and $g_4(x) = x - a/c$, as the inverse of adding a constant is subtracting the constant. @@ -900,8 +931,8 @@ What is $g_2(x)=f_2^{-1}(x)$? #| hold: true #| echo: false choices = ["``g_2(x) = x^{-1}``", "``g_2(x) = x``", "``g_2(x) = x -1``"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` What is $g_3(x)=f_3^{-1}(x)$? @@ -914,8 +945,8 @@ choices = [ raw"``c^2/(b\cdot c - a\cdot d) \cdot x``", raw"``(b\cdot c-a\cdot d)/c^2 \cdot x``", raw"``c^2 x``"] -answ = 1 -radioq(choices, answ) +answer = 1 +radioq(choices, answer) ``` Given these, what is the value of $g_4(g_3(g_2(g_1(f_4(f_3(f_2(f_1(10))))))))$? diff --git a/quarto/precalc/julia_overview.qmd b/quarto/precalc/julia_overview.qmd index 880206a..4fa6389 100644 --- a/quarto/precalc/julia_overview.qmd +++ b/quarto/precalc/julia_overview.qmd @@ -567,54 +567,67 @@ plotly() # optionally change the backend from the default With `Plots` loaded, we can plot a function by passing the function object by name to `plot`, specifying the range of `x` values to show, as follows: - +::: {#fig-plot-sin-0-to-2pi-illustration} ```{julia} plot(sin, 0, 2pi) ``` +Plot of $f(x) = \sin(x)$ over $[0, 2\pi]$ +::: + ::: {.callout-note} -This is in the form of **the** basic pattern employed: `verb(function_object, arguments...)`. The verb in this example is `plot`, the object `sin`, the arguments `0, 2pi` to specify `[a,b]` domain to plot over. +This plot command is in the form of **the** basic pattern employed throughout: `verb(function_object, arguments...)`. The verb in this example is `plot`, the object `sin`, the arguments `0, 2pi` to specify `[a,b]` domain to plot over. ::: Plotting more than one function over `[a,b]` is achieved through the `plot!` function, which modifies the existing plot (`plot` creates a new one) by adding a new layer: - +::: {#fig-plot-example-sin-cos-zero} ```{julia} plot(sin, 0, 2pi) -plot!(cos, 0, 2pi) -plot!(zero, 0, 2pi) # add the line y=0 +plot!(cos) +plot!(zero) # add the line y=0 ``` +Plot of $f(x) = \sin(x)$, $g(x) = \cos(x)$ and $h(x) = 0 over $[0, 2\pi]$ +::: + (There are alternatives to plot functions or other traces all at once.) -Individual points are added with `scatter` or `scatter!`: - +Individual points are added with `scatter` or `scatter!`. Either two vectors `xs` and `ys` with the coordinates in each *or* a vector of tuples, each tuple representing a point can be used. +::: {#fig-plot-of-sin-cos-scattered-points} ```{julia} plot(sin, 0, 2pi, legend=false) plot!(cos, 0, 2pi) -scatter!([pi/4, pi+pi/4], [sin(pi/4), sin(pi + pi/4)]) +scatter!([(pi/4, sin(pi/4)), ( pi+pi/4, sin(pi + pi/4))]) ``` +Plot of $f(x) = \sin(x)$ and $g(x) = \cos(x)$ over the $[0, 2\pi]$ with intersection points marked with markers through `scatter` +::: + (The extra argument `legend=false` suppresses the automatic legend drawing. There are many other useful keyword arguments to adjust attributes of a trace of a graphic. For example, passing `markersize=10` to the `scatter!` command would draw the points larger than the default.) Plotting an *anonymous* function is a bit more immediate than the two-step approach of defining a named function then calling `plot` with this as an argument: - +::: {#fig-plot-of-anonymous-function-exp-minusx-over-pi-times-sinx} ```{julia} plot( x -> exp(-x/pi) * sin(x), 0, 2pi) ``` +Plot of $f(x) = e^{-x/\pi}\sin(x)$ over $[0, 2\pi]$ using an anonymous function +::: The `scatter!` function used above takes two vectors of values to describe the points to plot, one for the $x$ values and one for the matching $y$ values. The `plot` function can also produce plots with this interface. For example, here we use a comprehension to produce `y` values from the specified `x` values: - +::: {#fig-plot-sine-functions-using-xs-ys-approach} ```{julia} #| hold: true xs = range(0, 2pi, length=251) ys = [sin(2x) + sin(3x) + sin(4x) for x in xs] plot(xs, ys) ``` +Plot of $f(x) = \sin(2x) + \sin(3x) + \sin(4x)$ over $[0, 2\pi]$ made by constructing vectors `xs` , `ys` holding $x$ and $y$ coordiinates of points to include +::: There are different plotting interfaces. Though not shown, all of these `plot` commands produce a plot of `f`, though with minor differences: @@ -679,12 +692,14 @@ p(x=>2), p(x=>2, a=>3, b=>4, c=>1) This is convenient notation for calling the `subs` function for `SymPy`. -SymPy expressions of a single free variable can be plotted directly: - +SymPy expressions of a single free variable can be plotted directly. +::: {#fig-plot-64-16-xsquared-using-sympy-recipe} ```{julia} plot(64 - (1/2)*32 * x^2, 0, 2) ``` +Plot of $f(x) = 64 - 16x^2$ over $[0, 2]$ using a plot recipe +::: * SymPy has functions for manipulating expressions: `simplify`, `expand`, `together`, `factor`, `cancel`, `apart`, $...$ * SymPy has functions for basic math: `factor`, `roots`, `solve`, `solveset`, $\dots$ diff --git a/quarto/precalc/plotting.qmd b/quarto/precalc/plotting.qmd index cbf56bc..2a187e2 100644 --- a/quarto/precalc/plotting.qmd +++ b/quarto/precalc/plotting.qmd @@ -30,10 +30,11 @@ A scalar, univariate function, such as $f(x) = 1 - x^2/2$, can be thought of in * It can be represented through a rule of what it does to $x$, as above. This is useful for computing numeric values. - * it can be interpreted verbally, as in *square* $x$, take half then *subtract* from one. This can give clarity to what the function does. + * it can be interpreted verbally, as in *square* $x$, take half, then *subtract* from one. This can give clarity to what the function does. * It can be thought of in terms of its properties: a polynomial, continuous, upside down $U$-shaped, an approximation for $\cos(x)$ near $0$, $\dots$ * it can be visualized graphically. This is useful for seeing the qualitative behavior of a function. +The last one is the focus of this section. The graph of a univariate function is just a set of points in the Cartesian plane. These points come from the relation $(x,f(x))$ that defines the function. Operationally, a sketch of the graph will consider a handful of such pairs and then the rest of the points will be imputed. @@ -76,7 +77,7 @@ using Plots ::: -Some backends require installation, such as `PyPlot` and `PlotlyJS`. We use `plotly` in these notes, for the most part, which is not the default, so requires an additional command to set the backend: +Some backends require installation, such as `PyPlot` and `PlotlyJS`. We use `plotly` in these notes, for the most part, which is not the default, so requires an additional command to set the backend:^[The `plotly` backend has some interactive features that make non-static graphics.] ```{julia} plotly() @@ -89,12 +90,15 @@ With `Plots` loaded and a backend chosen, it is straightforward to graph a funct For example, to graph $f(x) = 1 - x^2/2$ over the interval $[-3,3]$ we have: - +::: {#fig-one-minus-xsquared-over-2} ```{julia} f(x) = 1 - x^2/2 plot(f, -3, 3) ``` +Plot of $f(x) = 1 - x^2/2$ over $[-3,3]$ +::: + The `plot` command does the hard work behind the scenes. It needs $2$ pieces of information declared: @@ -116,48 +120,64 @@ Let's see some other graphs. The `sin` function over one period is plotted through: - +::: {#fig-plot-sin-0-2pi} ```{julia} plot(sin, 0, 2pi) ``` -We can make a graph of $f(x) = (1+x^2)^{-1}$ over $[-3,3]$ with +Plot of $f(x) = \sin(x)$ over $[0, 2\pi]$ +::: +We can make a graph of $f(x) = (1+x^2)^{-1}$ over $[-3,3]$ with: +::: {#fig-plot-1-over-1-plus-xsquared} ```{julia} #| hold: true f(x) = 1 / (1 + x^2) plot(f, -3, 3) ``` +Plot of $f(x) = 1 / (1 + x^2)$ over $[-3, 3]$ +::: + A graph of $f(x) = e^{-x^2/2}$ over $[-2,2]$ is produced with: - +::: {#fig-plot-exp-minus-xsquared-over-2} ```{julia} #| hold: true f(x) = exp(-x^2/2) plot(f, -2, 2) ``` +Plot of $f(x) = e^{-x^2/2}$ over $[-2, 2]$ +::: + We could skip the first step of defining a function by using an *anonymous function*. For example, to plot $f(x) = \cos(x) - x$ over $[0, \pi/2]$ we could do: - +::: {#fig-plot-cosx-minus-xusing-anonymous-function} ```{julia} plot(x -> cos(x) - x, 0, pi/2) ``` +Plot of $f(x) = \cos(x)-x$ over $[0, \pi/2]$ using an anonymous function +::: + Anonymous functions are especially helpful when parameterized functions are involved: +::: {#fig-plot-mxplusb-with-anonymous-function} ```{julia} #| hold: true mxplusb(x; m=1, b=0) = m*x + b plot(x -> mxplusb(x; m=-1, b=1), -1, 2) ``` +Plot of $f(x) = -x + 1$ over $[-1,2]$ using an anonymous function +::: + Had we parameterized using the `f(x,p)` style, the result would be similar: - +::: {#fig-plot-mxplusb-with-anonymous-function-fxp-style} ```{julia} function mxplusb(x, p) m, b = p.m, p.b @@ -166,6 +186,9 @@ end plot(x -> mxplusb(x, (m=-1, b=1)), -1, 2) ``` +Plot of $f(x) = -x + 1$ over $[-1,2]$ using an anonymous function and the "`f(x,p)`" style of passing parameters +::: + :::{.callout-note} ## Note The function object in the general pattern `action(function, args...)` is commonly specified in one of three ways: by a name, as with `f`; as an anonymous function; or as the return value of some other action through composition. @@ -174,12 +197,14 @@ The function object in the general pattern `action(function, args...)` is common Anonymous functions are also created by `Julia's` `do` notation, which is useful when the first argument to function (like `plot`) accepts a function: - +::: {#fig-plot-cosx-minus-x-using-do-notation} ```{julia} plot(0, pi/2) do x cos(x) - x end ``` +Plot of $f(x) = \cos(x) -x$ over $[0,\pi/2]$ illustrating use of `Julia`'s `do` notation +::: The `do` notation can be a bit confusing to read when unfamiliar, though its convenience makes it appealing. @@ -195,7 +220,7 @@ Some types we will encounter, such as the one for symbolic values or the special The default style for `Plots.jl` is to use a frame style where the viewing window is emphasized. This is a rectangular region, $[x_0, x_1] \times [y_0, y_1]$, which is seen through the tick labeling, the bounding scales on the left and bottom, and emphasized through the grid. -This choices does *not* show the $x-y$ axes. As such, we might layer on the axes when these are of interest. +This choice does *not* show the $x-y$ axes. As such, we might layer on the axes when these are of interest. To emphasize concepts, we may stylize a function graph, rather than display the basic graphic. For example, in this graphic highlighting the amount the function goes up as it moves from $1$ to $x$: @@ -239,17 +264,12 @@ plt = let ((1 + x)/2, f(1), text(L"\Delta x", 10, :top)), (x, (f(1) + f(x))/2, text(L"\Delta y", 10, :left)) ]) + plotly() current() end plt ``` -```{julia} -#| echo: false -plotly() -nothing -``` - ::: @@ -260,12 +280,15 @@ nothing Making a graph with `Plots` is easy, but producing a graph that is informative can be a challenge, as the choice of a viewing window can make a big difference in what is seen. For example, trying to make a graph of $f(x) = \tan(x)$, as below, will result in a bit of a mess---the chosen viewing window crosses several places where the function blows up: +::: {#fig-plot-tan-over-minus10-10-poor-choice} ```{julia} -#| hold: true f(x) = tan(x) plot(f, -10, 10) ``` +Plot of $f(x) = \tan(x)$ over $[-10,10]$. The vertical asymptotes of the function result in a poor graphic. +::: + Though this graph shows the asymptote structure and periodicity, it doesn't give much insight into each period or even into the fact that the function is periodic. @@ -275,22 +298,30 @@ Though this graph shows the asymptote structure and periodicity, it doesn't give The actual details of making a graph of $f$ over $[a,b]$ are pretty simple and follow the steps in making a "T"-table: - * A set of $x$ values are created between $a$ and $b$. - * A corresponding set of $y$ values are created. - * The pairs $(x,y)$ are plotted as points and connected with straight lines. +* A set of $x$ values are created between $a$ and $b$. + +* A corresponding set of $y$ values are created. + +* The pairs $(x,y)$ are plotted as points and connected with straight lines. + + +If the first two lines create values `xs` and `ys` then the last is done by calling `plot(xs, ys)`. The only real difference is that when drawing by hand, we might know to curve the lines connecting points based on an analysis of the function. As `Julia` doesn't consider this, the points are connected with straight lines – like a dot-to-dot puzzle. -In general, the `x` values are often generated by `range` or the `colon` operator and the `y` values produced by mapping or broadcasting a function over the generated `x` values. +In general, the `x` values are often generated by the `range` or the `colon` operator and the `y` values produced by mapping or broadcasting a function over the generated `x` values. -However, the plotting directive `plot(f, xmin, xmax)` calls an adaptive algorithm to use more points where needed, as judged by `PlotUtils.adapted_grid(f, (xmin, xmax))`. It computes both the `x` and `y` values. This algorithm is wrapped up into the `unzip(f, xmin, xmax)` function from `CalculusWithJulia`. The algorithm adds more points where the function is more "curvy" and uses fewer points where it is "straighter." Here we see the linear function is identified as needing far fewer points than the oscillating function when plotted over the same range: +However, the plotting directive `plot(f, xmin, xmax)`, in computing values for `x` to plot, calls an adaptive algorithm to use more points where needed, as judged by `PlotUtils.adapted_grid(f, (xmin, xmax))`. It computes both the `x` and `y` values. This algorithm is wrapped up into the `unzip(f, xmin, xmax)` function from `CalculusWithJulia`. The algorithm adds more points where the function is more "curvy" and uses fewer points where it is "straighter." Here we see the linear function is identified as needing far fewer points than the oscillating function when plotted over the same range: ```{julia} -pts_needed(f, xmin, xmax) = length(unzip(f, xmin, xmax)[1]) +function pts_needed(f, xmin, xmax) + xs, ys = unzip(f, xmin, xmax) + length(xs) +end pts_needed(x -> 10x, 0, 10), pts_needed(x -> sin(10x), 0, 10) ``` @@ -300,22 +331,19 @@ pts_needed(x -> 10x, 0, 10), pts_needed(x -> sin(10x), 0, 10) --- -For instances where a *specific* set of $x$ values is desired to be used, the `range` function or colon operator can be used to create the $x$ values and broadcasting used to create the $y$ values. For example, if we were to plot $f(x) = \sin(x)$ over $[0,2\pi]$ using $10$ points, we might do: - +For instances where a *specific* set of $x$ values is desired to be used, the `range` function or colon operator can be used to create the $x$ values and broadcasting used to create the $y$ values. For example, if we were to plot $f(x) = \sin(x)$ over $[0,2\pi]$ using $10$ points, we might do the three steps, passing the values to `plot` as follows: +::: {#fig-plot-xs-ys-sin} ```{julia} -xs = range(0, 2pi, length=10) +xs = range(0, 2pi, 10) ys = sin.(xs) -``` - -Finally, to plot the set of points and connect with lines, the $x$ and $y$ values are passed along as vectors: - - -```{julia} plot(xs, ys) ``` -This plots the points as pairs and then connects them in order using straight lines. Basically, it creates a dot-to-dot graph. The above graph looks primitive, as it doesn't utilize enough points. +Plot of $f(x) = \sin(x)$ over $[0,2\pi]$ using the `f(xs, ys)` style +::: + +The `plot` function plots the points as pairs and then connects them in order using straight lines. Basically, it creates a dot-to-dot graph. The above graph looks primitive, as it doesn't utilize enough points. @@ -325,96 +353,99 @@ This plots the points as pairs and then connects them in order using straight li The graph of a function may be reflected through a line, as those seen with a mirror. For example, a reflection through the $y$ axis takes a point $(x,y)$ to the point $(-x, y)$. We can easily see this graphically, when we have sets of $x$ and $y$ values through a judiciously placed minus sign. -For example, to plot $\sin(x)$ over $(-\pi,\pi)$ we might do: +For example, to plot $\sin(x)$ over $(-\pi,\pi)$ we might specify `xs`, generate `ys` and then call `plot(xs, ys)`. ```{julia} +#| eval: false xs = range(-pi, pi, length=100) ys = sin.(xs) plot(xs, ys) ``` -To reflect this graph through the $y$ axis, we only need to plot `-xs` and not `xs`: - +To reflect this graph through the $y$ axis, we only need to plot `-xs` and not `xs`. ```{julia} +#| eval: false plot(-xs, ys) ``` -Looking carefully we see there is a difference. (How?) +::: {#fig-plot-xs-ys-xs layout-ncol=1} +```{julia} +#| echo: false +xs = range(-pi, pi, length=100) +ys = sin.(xs) +p1 = plot(xs, ys) +p2 = plot(-xs, ys) +plot(p1, p2) +``` + +Plot of `xs, ys` on left, `-xs, ys` on right. The right graph is the reflection of the left one through the $x$ axis. +::: There are four very common reflections: +* reflection through the $y$-axis takes $(x,y)$ to $(-x, y)$. - * reflection through the $y$-axis takes $(x,y)$ to $(-x, y)$. - * reflection through the $x$-axis takes $(x,y)$ to $(x, -y)$. - * reflection through the origin takes $(x,y)$ to $(-x, -y)$. - * reflection through the line $y=x$ takes $(x,y)$ to $(y,x)$. +* reflection through the $x$-axis takes $(x,y)$ to $(x, -y)$. + +* reflection through the origin takes $(x,y)$ to $(-x, -y)$. + +* reflection through the line $y=x$ takes $(x,y)$ to $(y,x)$. -For the $\sin(x)$ graph, we see that reflecting through the $x$ axis produces the same graph as reflecting through the $y$ axis: +@fig-reflections shows the different graphs. For the $\sin(x)$ graph, we see that reflecting through the $x$ axis produces the same graph as reflecting through the $y$ axis. Doing both reflections (e.g., through the $x$ axis, then the $y$ axis) is the same as reflecting through the origin and for this function leaves the graph unchanged. +> An *even function* is one where reflection through the $y$ axis leaves the graph unchanged. That is, $f(-x) = f(x)$. An *odd function* is one where a reflection through the origin leaves the graph unchanged, or $f(-x) = -f(x)$. + + +::: {#fig-reflections} ```{julia} -plot(xs, -ys) +#| echo: false +let + xs = range(-pi, pi, length=100) + ys = sin.(xs) + p_xy = plot(-xs, ys; title="plot(-xs, ys)", legend=false) + hline!(p_xy, [0]; line=(1, :gray25)) + plot!(p_xy, xs, ys; line=(:dash, :gray75)) + + px_y = plot(xs, -ys; title="plot(xs, -ys)", legend=false) + vline!(px_y, [0]; line=(1, :gray)) + plot!(px_y, xs, ys; line=(:dash, :gray75)) + + p_x_y = plot(xs, ys; title="plot(-xs, -ys)", legend=false) + scatter!(p_x_y, [(0,0)]; marker=(3, :gray)) + plot!(p_x_y, xs, ys; line=(:dash, :gray75)) + + pyx = plot(ys, xs; title="plot(ys, xs)", legend=false, aspect_ratio=:equal) + plot!(pyx, [(-pi,-pi), (pi, pi)]; line=(1, :gray)) + plot!(pyx, xs, ys; line=(:dash, :gray75)) + + plot(p_xy, px_y, p_x_y, pyx) +end ``` -However, reflecting through the origin leaves this graph unchanged: +Plot of the four common reflections for $f(x) = \sin(x)$ along with a line or point indicated what the reflection was through. This function is an odd function, so the graph is left unchanged by reflection through the origin (lower left). +::: +The reflection of $f(x) = \sin(x)$ through the line $y=x$ leads to a graph of the equation $x = \sin(y)$. The result is not the graph of a function as the same $x$ can map to more than one $y$ value. (The new graph does not pass the "vertical line" test.) However, for the sine function we can get a function from this reflection if we choose a narrower viewing window: + +::: {#fig-plot-arcsin-by-generating-points-over-minuspi-over-2-to-pi-over-2} ```{julia} -plot(-xs, -ys) -``` - -> An *even function* is one where reflection through the $y$ axis leaves the graph unchanged. That is, $f(-x) = f(x)$. An *odd function* is one where a reflection through the origin leaves the graph unchanged, or symbolically $f(-x) = -f(x)$. - - - -If we try reflecting the graph of $\sin(x)$ through the line $y=x$, we have: - - -```{julia} -plot(ys, xs) -``` - -This is the graph of the equation $x = \sin(y)$, but is not the graph of a function as the same $x$ can map to more than one $y$ value. (The new graph does not pass the "vertical line" test.) - - -However, for the sine function we can get a function from this reflection if we choose a narrower viewing window: - - -```{julia} -#| hold: true xs = range(-pi/2, pi/2, length=100) ys = sin.(xs) plot(ys, xs) ``` +Inverse plot of $f(x)=\sin(x)$ over $[-\pi/2, \pi/2]$ using `plot(ys, xs)` +::: + The graph is that of the "inverse function" for $\sin(x), x \text{ in } [-\pi/2, \pi/2]$. -#### The `plot(xs, f)` syntax - - -When plotting a univariate function there are three basic patterns that can be employed. We have examples above of: - - -* `plot(f, xmin, xmax)` uses a recipe implementing an adaptive algorithm to identify values for $x$ in the interval `[xmin, xmas]`, - -* `plot(xs, f.(xs))` to manually choose the values of $x$ to plot points for, and - -Finally, there is a merging of the first two following the pattern: - -* `plot(xs, f)` - - -All require a manual choice of the values of the $x$-values to plot, but the broadcasting is carried out in the `plot` command. This style is convenient, for example, to down sample the $x$ range to see the plotting mechanics, such as: - - -```{julia} -plot(0:pi/4:2pi, sin) -``` #### NaN values @@ -424,22 +455,29 @@ At times it is not desirable to draw lines between each successive point. For ex For example,what happens at $0$ with $f(x) = 1/x$. The most straightforward plot is dominated by the vertical asymptote at $x=0$: - +::: {#fig-plot-1-over-x-over-minus1-1} ```{julia} q(x) = 1/x plot(q, -1, 1) ``` +Plot of $f(x) = 1/x$ over $[-1,1]$ showing effect on graphic of vertical asymptote +::: + We can attempt to improve this graph by adjusting the viewport. The *viewport* of a graph is the $x$-$y$ range of the viewing window. By default, the $y$-part of the viewport is determined by the range of the function over the specified interval, $[a,b]$. As just seen, this approach can produce poor graphs. The `ylims=(ymin, ymax)` argument can modify what part of the $y$ axis is shown. (Similarly `xlims=(xmin, xmax)` will modify the viewport in the $x$ direction.) As we see, even with this adjustment, the spurious line connecting the points with $x$ values closest to $0$ is still drawn: - +::: {#fig-plot-1-over-x-over-minus1-1-using-ylims} ```{julia} plot(q, -1, 1, ylims=(-10,10)) ``` +Plot of $f(x) = 1/x$ over $[-1,1]$ using `ylims` to reduce effect on graphic of vertical asymptote + +::: + The dot-to-dot algorithm, at some level, assumes the underlying function is *continuous*; here $q(x)=1/x$ is not. @@ -448,7 +486,7 @@ There is a convention for most plotting programs that **if** the $y$ value for a Here is one way to plot $q(x) = 1/x$ over $[-1,1]$ taking advantage of this convention: - +::: {#fig-plot-1-over-x-over-minus1-1-using-NaN} ```{julia} #| hold: true xs = range(-1, 1, length=251) @@ -457,29 +495,35 @@ ys[xs .== 0.0] .= NaN plot(xs, ys) ``` +Plot of $f(x) = 1/x$ after setting `y` values to `NaN` when `x` is zero +::: + By using an odd number of points, we should have that $0.0$ is amongst the `xs`. The next to last line replaces the $y$ value that would be infinite with `NaN`. -The above is fussy. As a recommended alternative, we might modify the function so that if it is too large, the values are replaced by `NaN`. Here is one such function consuming a function and returning a modified function put to use to make this graph: - +The above is fussy. As a recommended alternative, we might modify the function so that if it is too large, the values are replaced by `NaN`. Here is one such function consuming a function and returning a modified function put to use to make this graph]: +::: {#fig-plot-1-over-x-over-minus1-1-using-CalculusWithJulias-rangeclamp} ```{julia} rangeclamp(f, hi=20, lo=-hi; replacement=NaN) = x -> lo < f(x) < hi ? f(x) : replacement plot(rangeclamp(x -> 1/x), -1, 1) ``` -(The `clamp` function is a base `Julia` function which clamps a number between `lo` and `hi`, returning `lo` or `hi` if `x` is outside that range.) +Plot of $f(x) = 1/x$ over $[-1, 1]$ after wrapping in `rangclamp` to replace large `y` values in (absolute value) with `NaN`. +::: + +(The `clamp` function is a base `Julia` function and has slightly different semantics. It clamps a number between `lo` and `hi`, returning `lo` or `hi` (not `NaN`) if `x`is outside that range. The `rangeclamp` function is part of the `CalculusWithJulia` package and need not be defined if that package is loaded.) ## Layers -Graphing more than one function over the same viewing window is often desirable. Though this is easily done all at once in `Plots` by specifying a vector of functions as the first argument to `plot` instead of a single function object, we instead focus on building the graph layer by layer.^[The style of `Plots` is to combine multiple *series* to plot into one object and let `Plots` sort out which (every column is treated as a separate series). This can be very efficient from a programming perspective, but we leave it for power users. The use of layers, seems much easier to understand.] +Graphing more than one function over the same viewing window is often desirable. Though this is easily done all at once in `Plots`, we instead focus on building the graph layer by layer.^[The style of `Plots` is to combine multiple *series* to plot into one object and let `Plots` sort out which (every column is treated as a separate series). This can be very efficient from a programming perspective, but we leave it for power users. The use of layers, seems much easier to understand.] For example, to see that a polynomial and the cosine function are "close" near $0$, we can plot *both* $\cos(x)$ and the function $f(x) = 1 - x^2/2$ over $[-\pi/2,\pi/2]$: - +::: {#fig-plot-cos-quadratic-approx-using-layers} ```{julia} #| hold: true f(x) = 1 - x^2/2 @@ -487,9 +531,13 @@ plot(cos, -pi/2, pi/2, label="cos") plot!(f, -pi/2, pi/2, label="f") ``` -Another useful function to add to a plot is one to highlight the $x$ axis. This makes identifying zeros of the function easier. The anonymous function `x -> 0` will do this. But, perhaps less cryptically, so will the base function `zero`. For example +Plot of $f(x) = \cos(x)$ and a quadratic polynomial +::: + +Another useful function to add to a plot is one to highlight the $x$ axis. This makes identifying zeros of the function easier.^[The default frame style with `Plots` shows a scale around the boundaries of the graph. The specification `framestyle=:origin` labels the $x$ and $y$ axes] The anonymous function `x -> 0` will do this. But, perhaps less cryptically, so will the base function `zero`. For example +::: {#fig-plot-of-x5-x-plus-1-over-I-add-zero-layer} ```{julia} #| hold: true f(x) = x^5 - x + 1 @@ -497,6 +545,9 @@ plot(f, -1.5, 1.4, label="f") plot!(zero, label="zero") ``` +Plot of a polynomial with a layer showing `zero` (when $y=0$) +::: + (The job of `zero` is to return "$0$" in the appropriate type. There is also a similar `one` function in base `Julia`.) @@ -505,21 +556,25 @@ The `plot!` call adds a layer. We could still specify the limits for the plot, t For another example, suppose we wish to plot the function $f(x)=x\cdot(x-1)$ over the interval $[-1,2]$ and emphasize with points the fact that $0$ and $1$ are zeros. We can do this with three layers: the first to graph the function, the second to emphasize the $x$ axis, the third to graph the points. - +::: {#fig-plot-polynomial-zero-zeros} ```{julia} #| hold: true f(x) = x * (x-1) plot(f, -1, 2; legend=false) # turn off legend plot!(zero) -scatter!([0,1], [0,0]) +zs = [(0,0), (1,0)] # vector of tuples +scatter!(zs) ``` -The $3$ main functions used in these notes for adding layers are: +Plot of a polynomial with the zero line and the zeros of the polynomial added as layers +::: + +The three main functions used in these notes for adding layers are: * `plot!(f, a, b)` to add the graph of the function `f`; also `plot!(xs, ys)` -* `scatter!(xs, ys)` to add points $(x_1, y_1), (x_2, y_2), \dots$. +* `scatter!([(x1, y1), (x2, y2), ...])` or `scatter!(xs, ys)` to add points $(x_1, y_1), (x_2, y_2), \dots$. * `annotate!((x,y, label))` to add a label at $(x,y)$ @@ -536,23 +591,35 @@ The `Plots` package uses positional arguments for input data and keyword argumen The `Plots` package provides many such arguments for adjusting a graphic, here we mention just a few: - * `plot(...; title="main title", xlabel="x axis label", ylabel="y axis label")`: add title and label information to a graphic - * `plot(...; label="a label")` the `label` attribute will show up when a legend is present. Using an empty string, `""`, will suppress add the layer to the legend. - * `plot(...; legend=false)`: by default, different layers will be indicated with a legend, this will turn off this feature - * `plot(...; xlims=(a,b), ylims=(c,d))`: either or both `xlims` and `ylims` can be used to control the viewing window - * `plot(...; xticks=[xs..], yticks=[ys...]: either or both `xticks` and `yticks` can be used to specify where the tick marks are to be drawn - * `plot(...; aspect_ratio=:equal)`: will keep $x$ and $y$ axis on same scale so that squares look square. - * `plot(...; framestyle=:origin)`: The default `framestyle` places $x$-$y$ guides on the edges; this specification places them on the $x-y$ plane. - * `plot(...; color="green")`: this argument can be used to adjust the color of the drawn figure (color can be a string,`"green"`, or a symbol, `:green`, among other specifications) - * `plot(...; linewidth=5)`: this argument can be used to adjust the width of drawn lines - * `plot(...; linestyle=:dash)`: will change the line style of the plotted lines to dashed lines. Also `:dot`, ... +* `plot(...; title="main title", xlabel="x axis label", ylabel="y axis label")`: add a title to a graphic + +* `plot(...; xlabel="x axis label", ylabel="y axis label")`: add label information to a graphic + +* `plot(...; label="a label")` the `label` attribute will show up when a legend is present. Using an empty string, `""`, will suppress add the layer to the legend. + +* `plot(...; legend=false)`: by default, different layers will be indicated with a legend, this will turn off this feature + +* `plot(...; xlims=(a,b), ylims=(c,d))`: either or both `xlims` and `ylims` can be used to control the viewing window + +* `plot(...; xticks=[xs..], yticks=[ys...]` either or both `xticks` and `yticks` can be used to specify where the tick marks are to be drawn + +* `plot(...; aspect_ratio=:equal)`: will keep $x$ and $y$ axis on same scale so that squares look square. + +* `plot(...; framestyle=:origin)`: The default `framestyle` places $x$-$y$ guides on the edges; this specification places them on the $x-y$ plane. + +* `plot(...; color="green")`: this argument can be used to adjust the color of the drawn figure (color can be a string,`"green"`, or a symbol, `:green`, among other specifications) + +* `plot(...; linewidth=5)`: this argument can be used to adjust the width of drawn lines + +* `plot(...; linestyle=:dash)`: will change the line style of the plotted lines to dashed lines. Also `:dot`, ... For plotting points with `scatter`, or `scatter!` the markers can be adjusted via - * `scatter(...; markersize=5)`: increase marker size - * `scatter(...; marker=:square)`: change the marker (uses a symbol, not a string to specify) +* `scatter(...; markersize=5)`: increase marker size + +* `scatter(...; marker=:square)`: change the marker (uses a symbol, not a string to specify) Of course, zero, one, or more of these can be used on any given call to `plot`, `plot!`, `scatter`, or `scatter!`. @@ -562,10 +629,55 @@ Of course, zero, one, or more of these can be used on any given call to `plot`, There are also several *shorthands* in `Plots` that allows several related attributes to be specified to a single argument that is disambiguated using the type of the value. A few used herein are: * `line`. For example, `line=(5, 0.25, "blue")` will specify `linewidth=5` (integer), `linecolor="blue"` (string or symbol), `linealpha=0.25` (floating point) + * `marker`. For example `marker=(:star, 5)` will specify `markerstyle=:star` (symbol) and `markersize=5` (integer). + * `fill`. For example `fill=(:blue, 0.25)` will specify `fillcolor=:blue` (string or symbol) and `fillalpha=0.25` (floating point). -#### Example: Bresenham's algorithm + +## Points, lines, polygons + +Two basic objects to graph are points and lines. Add to these polygons. + +A point in two-dimensional space has two coordinates, often denoted by $(x,y)$. In `Julia`, the same notation produces a `tuple`. Using square brackets, as in `[x,y]`, produces a vector. Vectors are are more commonly used in these notes, as we have seen there are algebraic operations defined for them. However, tuples have other advantages and are how `Plots` designates a point. + +The plot command `plot(xs, ys)` plots the points $(x_1,y_1), \dots, (x_n, y_n)$ and then connects adjacent points with lines. The command `scatter(xs, ys)` just plots the points. + +However, the points might be more naturally specified as coordinate pairs. If tuples are used to pair them off, then `Plots` will plot a vector of tuples as a sequence of points through `plot([(x1,y1), (x2, y2), ..., (xn, yn)])`: + +::: {#fig-scatter-plot-vector-of-tuples} +```{julia} +pts = [( 1, 0), ( 1/4, 1/4), (0, 1), (-1/4, 1/4), + (-1, 0), (-1/4, -1/4), (0, -1), ( 1/4, -1/4)] +scatter(pts; legend=false) +``` +Scatter plot produced from a vector of tuples +::: + +A line segment simply connects two points. While these can be specified as vectors of $x$ and $y$ values, again it may be more convenient to use coordinate pairs to specify the points. Continuing the above, we can connect adjacent points with line segments: + +::: {#fig-scatter-plot-vector-of-tuples-and-connecting-lines} +```{julia} +plot!(pts; line=(:gray, 0.5, :dash)) +``` + +Previous scatter plot with points connected with lines +::: + +This uses the shorthand notation of `Plots` to specify `linecolor=:gray, linealpha=0.5, linestyle=:dash`. To plot just a line segment, just specifying two points suffices. + +The four-pointed star is not closed off, as there isn't a value from the last point to the first point. A polygon closes itself off. The `Shape` function can take a vector of points or a pair of `xs` and `ys` to specify a polygon. When these are plotted, the arguments to `fill` describe the interior of the polygon, the arguments to `line` the boundary: + +::: {#fig-Shape-example-with-vector-of-tuples} +```{julia} +plot(Shape(pts); fill=(:gray, 0.25), line=(:black, 2), legend=false) +scatter!(pts) +``` + +Plot of a `Shape` using vector of tuples to specify vertices +::: + +##### Example: Bresenham's algorithm In plotting a primitive, like a line, some mapping of the mathematical object to a collection of pixels must be made. For the case of a line [Bresenhams's line algorithm](https://en.wikipedia.org/wiki/Bresenham%27s_line_algorithm) can be used. @@ -573,8 +685,10 @@ In plotting a primitive, like a line, some mapping of the mathematical object to In the simplest case, let's assume a few things: * we have a line with slope $-1 < m < 0$. + * the pixels have integer coordinates (e.g., the pixel $(1, -1)$ would cover the region $[1,2] \times [-1, -2]$ when lit.) -* we start at point $(x_0, y_0)$, $f(x_0) = y_0$, with integer coordinates and end a point $(x_1, y_1)$, also with integer coordinates. The pixel $(x_0,y_0)$ is lit. + +* we start at point $(x_0, y_0)$, $f(x_0) = y_0$, with integer coordinates and end at point $(x_1, y_1)$, also with integer coordinates. The pixel $(x_0,y_0)$ is lit. With these assumptions, we have an initial decision to make: @@ -600,13 +714,13 @@ A,B,C = -(y₁ - y₀), (x₁-x₀), -x₁*y₀ + x₀*y₁ f(x,y) = A*x + B*y + C -xs = [(x₀, y₀)] -for i ∈ 1:(x₁ - 1) +xs = [(x₀, y₀)] # initial point +for i in 1:(x₁ - 1) xᵢ, yᵢ = xs[end] xᵢ₊₁ = xᵢ + 1 Δ = f(xᵢ+1, yᵢ-1/2) > 0 ? 1 : 0 yᵢ₊₁ = yᵢ - Δ - push!(xs, (xᵢ₊₁, yᵢ₊₁)) + push!(xs, (xᵢ₊₁, yᵢ₊₁)) # add another point end xs @@ -618,11 +732,11 @@ We can visualize with the following: p = plot(f, x₀, x₁; legend=false, aspect_ratio=:equal, xticks=0:x₁, yticks = (floor(Int, f(x₁))-1):(1 + ceil(Int, f(x₀)))) col = RGBA(.64,.64,.64, 0.25) -for xy ∈ xs - x, y = xy +for (x, y) in xs scatter!([x], [y]; marker=(5,)) scatter!([x+1], [y - 1/2]; marker=(5,:star)) - plot!(Shape(x .+ [0, 1, 1, 0], y .+ [0, 0, -1, -1]); color=col) + pixel = Shape([x, x+1, x+1, x], [y, y, y-1, y-1]) + plot!(pixel; fill=(col,)) end p ``` @@ -631,39 +745,6 @@ We see a number of additional arguments used: different marker sizes and shapes Of course, generalizations for positive slope and slope with magnitude greater than $1$ are needed. As well, this basic algorithm could be optimized, especially if it is part of a lower-level drawing primitive. But this illustrates the considerations involved. -## Points, lines, polygons - -Two basic objects to graph are points and lines. Add to these polygons. - -A point in two-dimensional space has two coordinates, often denoted by $(x,y)$. In `Julia`, the same notation produces a `tuple`. Using square brackets, as in `[x,y]`, produces a vector. Vectors are are more commonly used in these notes, as we have seen there are algebraic operations defined for them. However, tuples have other advantages and are how `Plots` designates a point. - -The plot command `plot(xs, ys)` plots the points $(x_1,y_1), \dots, (x_n, y_n)$ and then connects adjacent points with lines. The command `scatter(xs, ys)` just plots the points. - -However, the points might be more naturally specified as coordinate pairs. If tuples are used to pair them off, then `Plots` will plot a vector of tuples as a sequence of points through `plot([(x1,y1), (x2, y2), ..., (xn, yn)])`: - -```{julia} -pts = [( 1, 0), ( 1/4, 1/4), (0, 1), (-1/4, 1/4), - (-1, 0), (-1/4, -1/4), (0, -1), ( 1/4, -1/4)] -scatter(pts; legend=false) -``` - -A line segment simply connects two points. While these can be specified as vectors of $x$ and $y$ values, again it may be more convenient to use coordinate pairs to specify the points. Continuing the above, we can connect adjacent points with line segments: - -```{julia} -plot!(pts; line=(:gray, 0.5, :dash)) -``` - -This uses the shorthand notation of `Plots` to specify `linecolor=:gray, linealpha=0.5, linestyle=:dash`. To plot just a line segment, just specifying two points suffices. - -The four-pointed star is not closed off, as there isn't a value from the last point to the first point. A polygon closes itself off. The `Shape` function can take a vector of points or a pair of `xs` and `ys` to specify a polygon. When these are plotted, the arguments to `fill` describe the interior of the polygon, the arguments to `line` the boundary: - - -```{julia} -plot(Shape(pts); fill=(:gray, 0.25), line=(:black, 2), legend=false) -scatter!(pts) -``` - - ## Graphs of parametric equations @@ -678,13 +759,16 @@ A different graph can be made to compare the two functions side-by-side. This is The most "famous" parametric graph is one that is likely already familiar, as it follows the parametrization of points on the unit circle by the angle made between the $x$ axis and the ray from the origin through the point. (If not familiar, this will soon be discussed in these notes.) - +::: {#fig-plot-cos-sin-parametrically-aspect-ratio-equal} ```{julia} f(x) = cos(x); g(x) = sin(x) ts = range(0, 2pi, length=100) plot(f.(ts), g.(ts), aspect_ratio=:equal) # make equal axes ``` +Parametric plot of circle +::: + Any point $(a,b)$ on this graph is represented by $(\cos(t), \sin(t))$ for some value of $t$, and in fact multiple values of $t$, since $t + 2k\pi$ will produce the same $(a,b)$ value as $t$ will. @@ -713,17 +797,20 @@ scatter!(f.(θs), g.(θs)) As with the plot of a univariate function, there is a convenience interface for these plots---just pass the two functions in: - +::: {#fig-parametric-plot-passing-two-functions} ```{julia} plot(f, g, 0, 2pi, aspect_ratio=:equal) ``` +Illustration of `Plots` recipe for parametric plots +::: + ##### Example Looking at growth. Comparing $x^2$ with $x^3$ can run into issues, as the scale gets big: - +::: {#fig-compare-growth-by-layers} ```{julia} x²(x) = x^2 x³(x) = x^3 @@ -731,14 +818,20 @@ plot(x², 0, 25) plot!(x³, 0, 25) ``` +Plotting layers for $x^2$ and $x^3$ to compare growth +::: + In the above, `x³` is already $25$ times larger on the scale of $[0,25]$ and this only gets worse if the viewing window were to get larger. However, the parametric graph is quite different: - +::: {#fig-compare-growth-by-a-parametric-plot} ```{julia} plot(x², x³, 0, 25) ``` -In this graph, as $x^3/x^2 = x$, as $x$ gets large, the ratio stays reasonable. +Plotting parametrically to compare $x^2$ and $x^3$ +::: + +In @fig-compare-growth-by-a-parametric-plot as $x$ gets large, the ratio stays reasonable, it being $x^3/x^2 = x$. ##### Example @@ -749,7 +842,7 @@ Parametric plots are useful to compare the ratio of values near a point. In the Plot $f(x) = x^3$ and $g(x) = x - \sin(x)$ around $x=0$: - +::: {#fig-compare-x-minus-sinx-to-x-cubed} ```{julia} #| hold: true f(x) = x^3 @@ -757,7 +850,10 @@ g(x) = x - sin(x) plot(f, g, -pi/2, pi/2) ``` -This graph is *nearly* a straight line. At the point $(0,0)=(f(0), g(0))$, we see that both functions are behaving in a similar manner, though the slope is not $1$, so they do not increase at exactly the same rate. +Parametric plot to compare $f(x) = x^3$ with $g(x) = x - \sin(x)$ near $0$ +::: + +The graph in @fig-compare-x-minus-sinx-to-x-cubed is *nearly* a straight line. At the point $(0,0)=(f(0), g(0))$, we see that both functions are behaving in a similar manner, though the slope is not $1$, so they do not increase at exactly the same rate. ##### Example: Etch A Sketch @@ -769,10 +865,13 @@ This graph is *nearly* a straight line. At the point $(0,0)=(f(0), g(0))$, we se Playing with the toy makes a few things become clear: - * Twisting just the left knob (the horizontal or $x$ motion) will move the pointer left or right, leaving a horizontal line. Parametrically, this would follow the equations $f(t) = \xi(t)$ for some $\xi$ and $g(t) = c$. - * Twisting just the right knob (the vertical or $y$ motion) will move the pointer up or down, leaving a vertical line. Parametrically, this would follow the equations $f(t) = c$ and $g(t) = \psi(t)$ for some $\psi$. - * Drawing a line with a slope different from $0$ or $\infty$ requires moving both knobs at the same time. A $45$$^\circ$ line with slope $m=1$ can be made by twisting both at the same rate, say through $f(t) = ct$, $g(t)=ct$. It doesn't matter how big $c$ is, just that it is the same for both $f$ and $g$. Creating a different slope is done by twisting at different rates, say $f(t)=ct$ and $g(t)=dt$. The slope of the resulting line will be $d/c$. - * Drawing a curve is done by twisting the two knobs with varying rates. +* Twisting just the left knob (the horizontal or $x$ motion) will move the pointer left or right, leaving a horizontal line. Parametrically, this would follow the equations $f(t) = \xi(t)$ for some $\xi$ and $g(t) = c$. + +* Twisting just the right knob (the vertical or $y$ motion) will move the pointer up or down, leaving a vertical line. Parametrically, this would follow the equations $f(t) = c$ and $g(t) = \psi(t)$ for some $\psi$. + +* Drawing a line with a slope different from $0$ or $\infty$ requires moving both knobs at the same time. A $45$$^\circ$ line with slope $m=1$ can be made by twisting both at the same rate, say through $f(t) = ct$, $g(t)=ct$. It doesn't matter how big $c$ is, just that it is the same for both $f$ and $g$. Creating a different slope is done by twisting at different rates, say $f(t)=ct$ and $g(t)=dt$. The slope of the resulting line will be $d/c$. + +* Drawing a curve is done by twisting the two knobs with varying rates. These all apply to parametric plots, as the Etch A Sketch trace is no more than a plot of $(f(t), g(t))$ over some range of values for $t$, where $f$ describes the movement in time of the left knob and $g$ the movement in time of the right. @@ -790,7 +889,7 @@ Parametric plots can describe a richer set of curves than can plots of functions Here is an example using a parameterization provided on the Wikipedia page where $R$ is the radius of the larger disc, $r$ the radius of the smaller disc and $\rho < r$ indicating the position of the pencil within the smaller disc. - +::: {#fig-spirograph-formula-parametric-plot} ```{julia} #| hold: true R, r, rho = 1, 1/4, 1/4 @@ -800,8 +899,10 @@ g(t) = (R-r) * sin(t) - rho * sin((R-r)/r * t) plot(f, g, 0, max((R-r)/r, r/(R-r))*2pi) ``` -In the above, one can fix $R=1$. Then different values for `r` and `rho` will produce different graphs. These graphs will be periodic if $(R-r)/r$ is a rational. (Nothing about these equations requires $\rho < r$.) +Plot of two circles moving about each other +::: +In the above, one can fix $R=1$. Then different values for `r` and `rho` will produce different graphs. These graphs will be periodic if $(R-r)/r$ is a rational. (Nothing about these equations requires $\rho < r$.) @@ -821,8 +922,8 @@ choices = ["`(-Inf, -1)` and `(0,1)`", "`(-Inf, -0.577)` and `(0.577, Inf)`", "`(-1, 0)` and `(1, Inf)`" ]; -answ=3; -radioq(choices, answ) +answer=3; +radioq(choices, answer) ``` ###### Question @@ -852,8 +953,8 @@ choices = ["`(-Inf, -3)` and `(0, 1)`", "`(-3, 0)` and `(1, Inf)`", "`(-Inf, -4.1)` and `(1.455, Inf)`" ]; -answ=2; -radioq(choices, answ) +answer=2; +radioq(choices, answer) ``` ###### Question @@ -912,8 +1013,8 @@ choices = [ "`f(x) = x <= 4 ? 35.0 : 35.0 + 10.0 * (x-4)`", "`f(x) = x <= 10 ? 35.0 : 35.0 + 4.0 * (x-10)`" ] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` Make a plot of the model. Graphically estimate how many bags of trash will cost 55 dollars. @@ -922,8 +1023,8 @@ Make a plot of the model. Graphically estimate how many bags of trash will cost ```{julia} #| hold: true #| echo: false -answ = 15 -numericq(answ, .5) +answer = 15 +numericq(answer, .5) ``` ###### Question @@ -966,8 +1067,8 @@ What is seen? choices = [L"It oscillates wildly, as the period is $T=2\pi/(500 \pi)$ so there are 250 oscillations.", "It should oscillate evenly, but instead doesn't oscillate very much near 0 and 1", L"Oddly, it looks exactly like the graph of $f(x) = \sin(2\pi x)$."] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` The algorithm to plot a function works to avoid aliasing issues. Does the graph generated by `plot(f, 0, 1)` look the same, as the one above? @@ -980,8 +1081,8 @@ choices = ["Yes", "No, but is still looks pretty bad, as fitting 250 periods into a too small number of pixels is a problem.", "No, the graph shows clearly all 250 periods." ] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question @@ -1027,8 +1128,8 @@ choices = [ "An ellipse", "A straight line" ] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question @@ -1065,8 +1166,8 @@ choices = [ "A straight line", "None of the above" ] -answ = 1 -radioq(choices, answ, keep_order=true) +answer = 1 +buttonq(choices, answer) ``` Make this plot for the following specific values of the parameters `R`, `r`, and `rho`. What shape best describes it? @@ -1088,8 +1189,8 @@ choices = [ "A straight line", "None of the above" ] -answ = 3 -radioq(choices, answ,keep_order=true) +answer = 3 +buttonq(choices, answer) ``` Make this plot for the specific values of the parameters `R`, `r`, and `rho`. What shape best describes it? @@ -1111,8 +1212,8 @@ choices = [ "A straight line", "None of the above" ] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +buttonq(choices, answer) ``` Make this plot for the specific values of the parameters `R`, `r`, and `rho`. What shape best describes it? @@ -1134,6 +1235,6 @@ choices = [ "A straight line", "None of the above" ] -answ = 5 -radioq(choices, answ, keep_order=true) +answer = 5 +buttonq(choices, answer) ``` diff --git a/quarto/precalc/polynomial.qmd b/quarto/precalc/polynomial.qmd index 8e0b27b..913cad7 100644 --- a/quarto/precalc/polynomial.qmd +++ b/quarto/precalc/polynomial.qmd @@ -1,7 +1,6 @@ # Polynomials - -Now that basic properties of functions have been discussed, we move to various types of related functions beginning with polynomial functions. +With the basic properties of functions have been discussed, we move to various types of related functions beginning with polynomial functions. {{< include ../_common_code.qmd >}} @@ -12,7 +11,7 @@ In this section we use the following add-on packages: ```{julia} using SymPy using Plots -plotly() +plotly(); ``` ```{julia} @@ -26,13 +25,13 @@ nothing -Polynomials are a particular class of expressions that are simple enough to have many properties that can be analyzed. In particular, the key concepts of calculus: limits, continuity, derivatives, and integrals are all relatively trivial for polynomial functions. However, polynomials are flexible enough that they can be used to approximate a wide variety of functions. Indeed, though we don't pursue this, we mention that `Julia`'s `ApproxFun` package exploits this to great advantage. +Polynomials are a particular class of expressions that are simple enough to have many properties that can be analyzed. In particular, the key concepts of calculus: limits, continuity, derivatives, and integrals are all relatively straightforward for polynomial functions. However, polynomials are flexible enough that they can be used to approximate a wide variety of functions. Indeed, though we don't pursue this, we mention that `Julia`'s `ApproxFun` package exploits this to great advantage. -Here we discuss some vocabulary and basic facts related to polynomials and show how the add-on `SymPy` package can be used to model polynomial expressions within `SymPy`. `SymPy` provides a Computer Algebra System (CAS) for `Julia`. In this case, by leveraging a mature `Python` package [SymPy](https://www.sympy.org/). Later we will discuss the `Polynomials` package for polynomials. +Here we discuss some vocabulary and basic facts related to polynomials and show how the add-on `SymPy` package can be used to model polynomial expressions within `SymPy`. `SymPy` provides a Computer Algebra System (CAS) for `Julia`. In this case, by leveraging a mature `Python` package [SymPy](https://www.sympy.org/). In the next section we will discuss the `Polynomials` package for representing polynomials. -For our purposes, a *monomial* is simply a non-negative integer power of $x$ (or some other indeterminate symbol) possibly multiplied by a scalar constant. For example, $5x^4$ is a monomial, as are constants, such as $-2$ (it being $-2x^0$) and the symbol $x$ itself (it begin $x^1$. In general, one may consider restrictions on where the constants can come from, and consider more than one symbol, but we won't pursue this here, restricting ourselves to the case of a single variable and real coefficients. +For our purposes, a *monomial* is simply a non-negative integer power of $x$ (or some other indeterminate symbol) possibly multiplied by a scalar constant. For example, $5x^4$ is a monomial, as are constants, such as $-2$ (it being $-2x^0$) and the symbol $x$ itself (it being $x^1$). In general, one may consider restrictions on where the constants can come from, and consider more than one symbol, but we won't pursue this here, restricting ourselves to the case of a single variable and real coefficients. A *polynomial* is a sum of monomials. After combining terms with same powers, a non-zero polynomial may be written uniquely as: @@ -42,6 +41,9 @@ $$ a_n x^n + a_{n-1}x^{n-1} + \cdots + a_1 x + a_0, \quad a_n \neq 0 $$ +The zero polynomial is just $0$. + +::: {#fig-various-even-degree-monic-monomials} ```{julia} #| hold: true #| echo: false @@ -61,13 +63,16 @@ end imgfile = tempname() * ".gif" gif(anim, imgfile, fps = 1) -caption = "Polynomials of varying even degrees over ``[-1,1]``." +caption = "" plotly() ImageFile(imgfile, caption) ``` -The numbers $a_0, a_1, \dots, a_n$ are the **coefficients** of the polynomial in the standard basis. With the identifications that $x=x^1$ and $1 = x^0$, the monomials above have their power match their coefficient's index, e.g., $a_ix^i$. Outside of the coefficient $a_n$, the other coefficients may be negative, positive, *or* $0$. Except for the zero polynomial, the largest power $n$ is called the [degree](https://en.wikipedia.org/wiki/Degree_of_a_polynomial). The degree of the [zero](http://tinyurl.com/he6eg6s) polynomial is typically not defined or defined to be $-1$, so as to make certain statements easier to express. The term $a_n$ is called the **leading coefficient**. When the leading coefficient is $1$, the polynomial is called a **monic polynomial**. The monomial $a_n x^n$ is the **leading term**. +Polynomials of varying even degrees over $[-1,1]$ +::: + +The numbers $a_0, a_1, \dots, a_n$ are the *coefficients* of the polynomial in the standard basis. With the identifications that $x=x^1$ and $1 = x^0$, the monomials above have their power match their coefficient's index, e.g., $a_ix^i$. Outside of the coefficient $a_n$, the other coefficients may be negative, positive, *or* $0$. Except for the zero polynomial, the largest power $n$ is called the [degree](https://en.wikipedia.org/wiki/Degree_of_a_polynomial). The degree of the [zero](http://tinyurl.com/he6eg6s) polynomial is typically not defined or defined to be $-1$, so as to make certain statements easier to express. The term $a_n$ is called the *leading coefficient*. When the leading coefficient is $1$, the polynomial is called a *monic polynomial*. The monomial $a_n x^n$ is the *leading term*. For example, the polynomial $-16x^2 - 32x + 100$ has degree $2$, leading coefficient $-16$ and leading term $-16x^2$. It is not monic, as the leading coefficient is not $1$. @@ -86,13 +91,14 @@ $$ a_1 x + a_0, \quad a_1 \neq 0, $$ -is often written as $mx + b$, which is the **slope-intercept** form. The slope of a line determines how steeply it rises. The value of $m$ can be found from two points through the well-known formula: +is often written as $mx + b$, which is the *slope-intercept* form. The slope of a line determines how steeply it rises. The value of $m$ can be found from two points through the well-known formula: $$ m = \frac{y_1 - y_0}{x_1 - x_0} = \frac{\text{rise}}{\text{run}} $$ +::: {#fig-graph-linear-polynomial-different-m} ```{julia} #| hold: true #| echo: false @@ -111,15 +117,17 @@ end imgfile = tempname() * ".gif" gif(anim, imgfile, fps = 1) -caption = "Graphs of y = mx for different values of m" +caption = "" plotly() ImageFile(imgfile, caption) ``` +Graphs of y = mx for different values of m +::: -The intercept, $b$, comes from the fact that when $x=0$ the expression is $b$. That is the graph of the function $f(x) = mx + b$ will have $(0,b)$ as a point on it. +The intercept, $b$, comes from the fact that when $x=0$ the expression is $b$. That is, the graph of the function $f(x) = mx + b$ will have $(0,b)$ as a point on it. -More generally, we have the **point-slope** form of a line, written as a polynomial through +More generally, we have the *point-slope* form of a line, written as a polynomial through $$ @@ -138,11 +146,11 @@ Thinking in terms of transformations, this looks like the function $f(x) = x$ (w The indeterminate value `x` (or some other symbol) in a polynomial, is like a variable in a function and unlike a variable in `Julia`. Variables in `Julia` are identifiers, just a means to look up a specific, already determined, value. Rather, the symbol `x` is not yet determined, it is essentially a place holder for a future value. Although we have seen that `Julia` makes it very easy to work with mathematical functions, it is not the case that base `Julia` makes working with expressions of algebraic symbols easy. This makes sense, `Julia` is primarily designed for technical computing, where numeric approaches rule the day. However, symbolic math can be used from within `Julia` through add-on packages. -Symbolic math programs include well-known ones like the commercial programs Mathematica and Maple. Mathematica powers the popular [WolframAlpha](www.wolframalpha.com) website, which turns "natural" language into the specifics of a programming language. The open-source [Sage](https://www.sagemath.org/) project is an alternative to these two commercial giants. It includes a wide-range of open-source math projects available within its umbrella framework. (`Julia` can even be run from within the free service [cloud.sagemath.com](https://cloud.sagemath.com/projects).) A more focused project for symbolic math, is the [SymPy](www.sympy.org) Python library. SymPy is also used within Sage. However, SymPy provides a self-contained library that can be used standalone within a Python session. +Symbolic math programs include well-known ones like the commercial programs Mathematica and Maple. Mathematica powers the popular [WolframAlpha](www.wolframalpha.com) website, which turns "natural" language into the specifics of a programming language. The open-source [Sage](https://www.sagemath.org/) project is an alternative to these two commercial giants. It includes a wide-range of open-source math projects available within its umbrella framework. A more focused project for symbolic math, is the [SymPy](www.sympy.org) Python library. SymPy is also used within Sage. However, SymPy provides a self-contained library that can be used standalone within a Python session. -The [Symbolics](https://github.com/JuliaSymbolics/Symbolics.jl) package for `Julia` provides a "fast and modern CAS for fast and modern language." It is described further in [Symbolics.jl](../alternatives/symbolics.qmd). +The [Symbolics](https://github.com/JuliaSymbolics/Symbolics.jl) package for `Julia` provides a "fast and modern CAS for fast and modern language." It is described further in [Symbolics.jl](../alternatives/symbolics.qmd). The [Giac.jl](https://github.com/s-celles/Giac.jl/) package is another alternative. -As `SymPy` has some features not yet implemented in `Symbolics`, we use `SymPy` in these notes. The `PyCall` and `PythonCall` packages are available to glue `Julia` to Python in a seamless manner. These allow the `Julia` package `SymPy` (or `SymPyPythonCall`) to provide functionality from SymPy within `Julia`. +As `SymPy` has some features not yet implemented in `Symbolics`, we use `SymPy` in these notes. The `PyCall` and `PythonCall` packages are available to glue `Julia` to Python in a seamless manner. These allow the `Julia` package `SymPy` (or `SymPyPythonCall`) to provide functionality from SymPy within `Julia`. (`Giac` calls a `C++` library and is an alternative that may not be easier to install.) :::{.callout-note} @@ -152,9 +160,8 @@ When `SymPy` is installed through the package manager, the underlying `Python` l ::: :::{.callout-note} -## Note -The [`Symbolics`](../alternatives/symbolics) package is a rapidly developing `Julia`-only package that provides symbolic math options. - +## Alternative +The [Symbolics](../alternatives/symbolics) package is a rapidly developing `Julia`-only package that provides symbolic math options. The [Giac package](https://github.com/s-celles/Giac.jl/) is another good alternative to `SymPy`. Both packages are described further when alternative packages are discussed. ::: --- @@ -167,7 +174,7 @@ To use `SymPy`, we create symbolic objects to be our indeterminate symbols. The @syms a, b, c, x::real, zs[1:10] ``` -The above shows that multiple symbols can be defined at once. The annotation `x::real` instructs `SymPy` to assume the `x` is real, as otherwise it assumes it is possibly complex. There are many other [assumptions](http://docs.sympy.org/dev/modules/core.html#module-sympy.core.assumptions) that can be made. The `@syms` macro documentation lists them. The `zs[1:10]` tensor notation creates a container with $10$ different symbols. The *macro* `@syms` does not need assignment, as the variable(s) are created behind the scenes by the macro. +The above shows that multiple symbols can be defined at once. The annotation `x::real` instructs `SymPy` to assume the `x` is real, as otherwise it assumes it is possibly complex. There are many other [assumptions](http://docs.sympy.org/dev/modules/core.html#module-sympy.core.assumptions) that can be made. The `@syms` macro documentation lists them. The `zs[1:10]` "tensor notation" creates a container with $10$ different symbols. The *macro* `@syms` does not need assignment, as the variable(s) are created behind the scenes by the macro. :::{.callout-note} @@ -179,9 +186,11 @@ Macros in `Julia` are just transformations of the syntax into other syntax. The The `SymPy` package does three basic things: - * It imports some of the functionality provided by `SymPy`, including the ability to create symbolic variables. - * It overloads many `Julia` functions to work seamlessly with symbolic expressions. This makes working with polynomials quite natural. - * It gives access to a wide range of SymPy's functionality through the `sympy` object. +* It imports some of the functionality provided by `SymPy`, including the ability to create symbolic variables. + +* It add methods for many `Julia` functions to work seamlessly with symbolic expressions. This makes working with polynomials quite natural. + +* It gives access to a wide range of SymPy's functionality through the `sympy` object. To illustrate, using the just defined `x`, here is how we can create the polynomial $-16x^2 + 100$: @@ -263,13 +272,6 @@ The result will always be of a symbolic type, even if the answer is just a numbe typeof(y) ``` -If there is just one free variable in an expression, the pair notation can be dropped: - - -```{julia} -p(4) # substitutes x=>4 -``` - ##### Example @@ -298,28 +300,25 @@ In the above, we substituted `2` in for `x` to get `y`: ```{julia} #| hold: true p = -16x^2 + 100 -y = p(2) +y = p(x => 2) ``` The value, $36$ is still symbolic, but clearly an integer. If we are just looking at the output, we can easily translate from the symbolic value to an integer, as they print similarly. However the conversion to an integer, or another type of number, does not happen automatically. If a number is needed to pass along to another `Julia` function, it may need to be converted. In general, conversions between different types are handled through various methods of `convert`. -For real numbers, an easy to call conversion is available through the `float` method: +For real numbers, an easy to call conversion is available through the generic `float` method, which converts a value to a floating point type:: ```{julia} float(y) ``` - -The use of the generic `float` method returns a floating point number. (The `.evalf()` method of `SymPy` objects uses `SymPy` to produce floating point versions of symbolic values. - `SymPy` objects have their own internal types. To preserve these on conversion to a related `Julia` value, the `N` function from `SymPy` is useful: ```{julia} #| hold: true p = -16x^2 + 100 -N(p(2)) +N(p(x=>2)) ``` Where `convert(T, x)` requires a specification of the type to convert `x` to, `N` attempts to match the data type used by SymPy to store the number. As such, the output type of `N` may vary (rational, a BigFloat, a float, etc.) Conversion by `N` will fail if the value to be converted contains free symbols, as would be expected. @@ -333,7 +332,7 @@ Evaluating a symbolic expression and returning a numeric value can be done by co ```{julia} p = 200 - 16x^2 -N(p(2)) +N(p(x=>2)) ``` This approach is direct, but can be slow *if* many such evaluations were needed (such as with a plot). An alternative is to turn the symbolic expression into a `Julia` function and then evaluate that as usual. @@ -358,26 +357,26 @@ pp = lambdify(p) pp(1,2,3) ``` -This evaluation matches `a` with `1`, `b` with`2`, and `x` with `3` as that is the order returned by the function call `free_symbols(p)`. To adjust that, a second `vars` argument can be given: +This evaluation matches `a` with `1`, `b` with`2`, and `x` with `3` as that is the order returned by the function call `free_symbols(p)`. Leaving the order to an underlying function is not a great idea, rather, explicitly passing the variables is recommended: ```{julia} #| hold: true -pp = lambdify(p, (x,a,b)) +pp = lambdify(p, (x,a,b)) # specify the variables pp(1,2,3) # computes 2*1^2 + 3 ``` -(We suggest using the pair notation when there is more than one variable.) - ## Graphical properties of polynomials -Consider the graph of the polynomial `x^5 - x + 1`: - +Consider the graph of the polynomial `x^5 - x + 1` in @fig-plot-polynomial-using-sympy-plot-recipe: +::: {#fig-plot-polynomial-using-sympy-plot-recipe} ```{julia} plot(x^5 - x + 1, -3/2, 3/2) ``` +Plot of $f(x) = x^5 - x + 1$ using the recipe for `SymPy` expression +::: (Plotting symbolic expressions with `Plots` is similar to plotting a function, in that the expression is passed in as the first argument. The expression must have only one free variable, as above, or an error will occur. This happens, as there is a `Plots` "recipe" for `SymPy` defined.) @@ -385,14 +384,16 @@ plot(x^5 - x + 1, -3/2, 3/2) This graph illustrates the key features of polynomial graphs: - * there may be values for `x` where the graph crosses the $x$ axis (real roots of the polynomial); - * there may be peaks and valleys (local maxima and local minima); - * except for constant polynomials, the ultimate behaviour for large values of $|x|$ is either both sides of the graph going to positive infinity, or negative infinity, or as in this graph one to the positive infinity and one to negative infinity. In particular, there is no *horizontal asymptote*. +* there may be values for `x` where the graph crosses the $x$ axis (real roots of the polynomial); + +* there may be peaks and valleys (local maxima and local minima); + +* except for constant polynomials, the ultimate behaviour for large values of $|x|$ is either both sides of the graph going to positive infinity, or negative infinity, or as in this graph one to the positive infinity and one to negative infinity. In particular, there is no *horizontal asymptote*. -To investigate this last point, let's consider the case of the monomial $x^n$. When $n$ is even, the following animation shows that larger values of $n$ have greater growth once outside of $[-1,1]$: - +To investigate this last point, let's consider the case of the monomial $x^n$. When $n$ is even, the animation in @fig-faster-growing-monomials-animation shows that larger values of $n$ have greater growth once outside of $[-1,1]$: +::: {#fig-faster-growing-monomials-animation} ```{julia} #| hold: true #| echo: false @@ -411,30 +412,36 @@ end imgfile = tempname() * ".gif" gif(anim, imgfile, fps = 1) -caption = L"Demonstration that $x^{10}$ grows faster than $x^8$, ... and $x^2$ grows faster than $x^0$ (which is constant)." +caption = L"" plotly() ImageFile(imgfile, caption) ``` +Demonstration that $x^{10}$ grows faster than $x^8$, $\dots$, and $x^2$ grows faster than $x^0$ (which is constant) +::: -Of course, this is expected, as, for example, $2^2 < 2^4 < 2^6 < \cdots$. The general shape of these terms is similar - $U$ shaped, and larger powers dominate the smaller powers as $|x|$ gets big. +Of course, this is expected, as, for example, $2^2 < 2^4 < 2^6 < \cdots$. The general shape of these terms is similar---$U$ shaped, and larger powers dominate the smaller powers as $|x|$ gets big. -For odd powers of $n$, the graph of the monomial $x^n$ is no longer $U$ shaped, but rather constantly increasing. This graph of $x^5$ is typical: - +For odd powers of $n$, the graph of the monomial $x^n$ is no longer $U$ shaped, but rather constantly increasing. This graph of $x^5$ is typical; for larger powers the shape is similar, but the growth is faster. +::: {#fig-show-odd-power-shape} ```{julia} plot(x^5, -2, 2) ``` -Again, for larger powers the shape is similar, but the growth is faster. +Plot of odd degree monomial showing the general shape for large $x$. +::: + ### Leading term dominates -To see the roots and/or the peaks and valleys of a polynomial requires a judicious choice of viewing window, as ultimately the leading term will dominate the graph. The following animation of the graph of $(x-5)(x-3)(x-2)(x-1)$ illustrates. Subsequent images show a widening of the plot window until the graph appears U-shaped. +To see the roots and/or the peaks and valleys of a polynomial requires a judicious choice of viewing window, as ultimately the leading term will dominate the graph. @fig-leading-term-dominates shows the graph of the fourth-degree polynomial $(x-5)(x-3)(x-2)(x-1)$ over different domains. Subsequent images show a widening of the plot window until the graph appears U-shaped. The leading term in the animation is $x^4$, of even degree, so the graphic is U-shaped, were the leading term of odd degree the left and right sides would each head off to different signs of infinity. +::: {#fig-leading-term-dominates} + ```{julia} #| hold: true #| echo: false @@ -457,14 +464,14 @@ anim = @animate for n in 1:6 end end -caption = "The previous graph is highlighted in red. Ultimately the leading term (\$x^4\$ here) dominates the graph." imgfile = tempname() * ".gif" gif(anim, imgfile, fps=1) plotly() -ImageFile(imgfile, caption) +ImageFile(imgfile, "") ``` -The leading term in the animation is $x^4$, of even degree, so the graphic is U-shaped, were the leading term of odd degree the left and right sides would each head off to different signs of infinity. +The previous graph is highlighted in red. Ultimately the leading term ($x^4$ here) dominates the graph. +::: To illustrate analytically why the leading term dominates, consider the polynomial $2x^5 - x + 1$ and then factor out the largest power, $x^5$, leaving a product: @@ -474,15 +481,15 @@ $$ x^5 \cdot (2 - \frac{1}{x^4} + \frac{1}{x^5}). $$ -For large $|x|$, the last two terms in the product on the right get close to $0$, so this expression is *basically* just $2x^5$ - the leading term. +For large $|x|$, the last two terms in the product on the right get close to $0$, so this expression is *basically* just $2x^5$---the leading term. --- -The following graphic illustrates the $4$ basic *overall* shapes that can result when plotting a polynomials as $x$ grows without bound: - +@fig-four-basic-polynomial-shapes illustrates the $4$ basic *overall* shapes that can result when plotting a polynomials as $x$ grows without bound: +::: {#fig-four-basic-polynomial-shapes} ```{julia} #| echo: false let @@ -501,12 +508,8 @@ plotly() nothing ``` -##### Example - -This graphic shows some of the above: - -[![you tube](https://img.youtube.com/vi/OFzqDatEvCo/3.jpg)](https://m.youtube.com/watch?v=OFzqDatEvCo) - +Figure illustrating the four basic shapes a polynomial may take for large values of $x$ or $-x$. +::: ##### Example @@ -536,7 +539,7 @@ This observation is the start of Descartes' rule of [signs](http://sepwww.stanfo ## Factoring polynomials -Among numerous others, there are two common ways of representing a non-zero polynomial: +Among others, there are two common ways of representing a non-zero polynomial: * expanded form, as in $a_n x^n + a_{n-1}x^{n-1} + \cdots + a_1 x + a_0,\quad a_n \neq 0$; or @@ -547,7 +550,7 @@ The former uses the *standard basis* to represent the polynomial $p$. The latter writes $p$ as a product of linear factors, though this is only possible in general if we consider complex roots. With real roots only, then the factors are either linear or quadratic, as will be discussed later. -There are values to each representation. One value of the expanded form is that polynomial addition and scalar multiplication is much easier than in factored form. For example, adding polynomials just requires matching up the monomials of similar powers. (These can be realized easily as vector operations.) For the factored form, polynomial multiplication is much easier than expanded form. For the factored form it is easy to read off *roots* of the polynomial (values of $x$ where $p$ is $0$), as a product is $0$ only if a term is $0$, so any zero must be a zero of a factor. Factored form has other technical advantages. For example, the polynomial $(x-1)^{1000}$ can be compactly represented using the factored form, but would require $1001$ coefficients to store in expanded form. (As well, due to floating point differences, the two would evaluate quite differently as one would require over a $1000$ operations to compute, the other just two.) +There are values to each representation. One value of the expanded form is that polynomial addition and scalar multiplication is much easier than in factored form. For example, adding polynomials just requires matching up the monomials of similar powers. For the factored form, polynomial multiplication is much easier than expanded form. For the factored form it is easy to read off *roots* of the polynomial (values of $x$ where $p$ is $0$), as a product is $0$ only if a term is $0$, so any zero must be a zero of a factor. Factored form has other technical advantages. For example, the polynomial $(x-1)^{1000}$ can be compactly represented using the factored form, but would require $1001$ coefficients to store in expanded form. (As well, due to floating point differences, the two would evaluate quite differently as one would require over a $1000$ operations to compute, the other just two.) Translating from factored form to expanded form can be done by carefully following the distributive law of multiplication. For example, with some care it can be shown that: @@ -594,10 +597,7 @@ The factoring $(x-\sqrt{2})\cdot(x + \sqrt{2})$ is not found, as $\sqrt{2}$ is n ### Polynomial functions and polynomials. -Our definition of a polynomial is in terms of algebraic expressions which are easily represented by `SymPy` objects, but not objects from base `Julia`. (Later we discuss the `Polynomials` package for representing polynomials. There is also the `AbstractAlbegra` package for a more algebraic treatment of polynomials.) - - -However, *polynomial functions* are easily represented by `Julia`, for example, +Our definition of a polynomial is in terms of algebraic expressions which are easily represented by `SymPy` objects, but not objects from base `Julia`. However, *polynomial functions* are easily represented by `Julia`, for example: ```{julia} @@ -607,22 +607,16 @@ f(x) = -16x^2 + 100 The distinction is subtle, the expression is turned into a function just by adding the "`f(x) =`" preface. But to `Julia` there is a big distinction. The function form never does any computation until after a value of $x$ is passed to it. Whereas symbolic expressions can be manipulated quite freely before any numeric values are specified. -It is easy to create a symbolic expression from a function - just evaluate the function on a symbolic value: +It is easy to create a symbolic expression from a function---just evaluate the function on a symbolic value: ```{julia} f(x) ``` -This is easy---but can also be confusing. The function object is `f`, the expression is `f(x)`---the function evaluated on a symbolic object. Moreover, as seen, the symbolic expression can be evaluated using the same syntax as a function call: +This is easy---but can also be confusing. The function object is `f`, the expression is `f(x)`---the function evaluated on a symbolic object. - -```{julia} -p = f(x) -p(2) -``` - -For many uses, the distinction is unnecessary to make, as the many functions will work with any callable expression. For `Plots` there is a recipe – either `plot(f, a, b)` or `plot(f(x), a, b)` will produce the same plot using the `Plots` package. +For many uses, the distinction is unnecessary to make, as the many functions will work with any callable expression. For `Plots` there is a recipe---either `plot(f, a, b)` or `plot(f(x), a, b)` will produce the same plot using the `Plots` package. ## Questions @@ -690,8 +684,8 @@ What is the leading term of $p$? #| hold: true #| echo: false choices = ["``3``", "``3x^2``", "``-2x``", "``5``"] -answ = 2 -radioq(choices, answ) +answer = 2 +buttonq(choices, answer) ``` ###### Question @@ -734,8 +728,8 @@ The linear polynomial $p = 2x + 3$ is written in which form: #| hold: true #| echo: false choices = ["point-slope form", "slope-intercept form", "general form"] -answ = 2 -radioq(choices, answ) +answer = 2 +buttonq(choices, answer) ``` ###### Question @@ -758,8 +752,8 @@ What command will return the value of the polynomial when $x=2$? #| hold: true #| echo: false choices = [q"p*2", q"p[2]", q"p_2", q"p(x=>2)"] -answ = 4 -radioq(choices, answ) +answer = 4 +buttonq(choices, answer) ``` ###### Question @@ -776,8 +770,8 @@ L"Be $U$-shaped, opening upward", L"Be $U$-shaped, opening downward", L"Overall, go upwards from $-\infty$ to $+\infty$", L"Overall, go downwards from $+\infty$ to $-\infty$"] -answ = 3 -radioq(choices, answ, keep_order=true) +answer = 3 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -794,8 +788,8 @@ L"Be $U$-shaped, opening upward", L"Be $U$-shaped, opening downward", L"Overall, go upwards from $-\infty$ to $+\infty$", L"Overall, go downwards from $+\infty$ to $-\infty$"] -answ = 1 -radioq(choices, answ, keep_order=true) +answer = 1 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -812,8 +806,8 @@ L"Be $U$-shaped, opening upward", L"Be $U$-shaped, opening downward", L"Overall, go upwards from $-\infty$ to $+\infty$", L"Overall, go downwards from $+\infty$ to $-\infty$"] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -859,8 +853,8 @@ choices = [q"x^3 - 3x^2 + 2x", q"x^3 - x^2 - 2x", q"x^3 + x^2 - 2x", q"x^3 + x^2 + 2x"] -answ = 2 -radioq(choices, 2) +answer = 2 +buttonq(choices, answer) ``` ###### Question @@ -877,6 +871,6 @@ q"-h^2 + 3hx - 3x^2", q"h^3 + 3h^2x + 3hx^2 + x^3 -x^3/h", q"x^3 - x^3/h", q"0"] -answ = 1 -radioq(choices, answ) +answer = 1 +buttonq(choices, answer) ``` diff --git a/quarto/precalc/polynomial_roots.qmd b/quarto/precalc/polynomial_roots.qmd index 2651786..03c45c9 100644 --- a/quarto/precalc/polynomial_roots.qmd +++ b/quarto/precalc/polynomial_roots.qmd @@ -1,6 +1,5 @@ # Roots of a polynomial - {{< include ../_common_code.qmd >}} In this section we use the following add on packages: @@ -25,15 +24,18 @@ nothing --- -The [roots](http://en.wikipedia.org/wiki/Properties_of_polynomial_roots) of a polynomial are the values of $x$ that when substituted into the expression yield $0$. For example, the polynomial $x^2 - x$ has two roots, $0$ and $1$. A simple graph verifies this: +The [roots](http://en.wikipedia.org/wiki/Properties_of_polynomial_roots) of a polynomial are the values of $x$ that when substituted into the polynomial yield $0$. For example, the polynomial $x^2 - x$ has two roots, $0$ and $1$. A simple graph verifies this: +::: {#fig-plot-parabola-highlight-zero-line} ```{julia} #| hold: true f(x) = x^2 - x plot(f, -2, 2) plot!(zero, -2, 2) ``` +Plot of $f(x) = x^2 - x$ over $[-2, 2]$ showing the graph crosses the $x$ axis twice +::: The graph crosses the $x$-axis at both $0$ and $1$. @@ -41,9 +43,11 @@ The graph crosses the $x$-axis at both $0$ and $1$. What is known about polynomial roots? Some simple questions might be: - * Will a polynomial always have a root? - * How many roots can there be? - * How large can the roots be? +* Will a polynomial always have a root? A *real* root? + +* How many roots can there be? + +* How large can the roots be? We look at such questions here. @@ -52,10 +56,12 @@ We look at such questions here. ### The factor theorem -We begin with a comment that ties together two concepts related to polynomials. It allows us to speak of roots or factors interchangeably: +We begin with a comment that ties together two concepts related to polynomials. - -> The [factor theorem](http://en.wikipedia.org/wiki/Factor_theorem) relates the *roots* of a polynomial with its *factors*: $r$ is a root of $p$ if *and* only if $(x-r)$ is a factor of the polynomial $p$. +::: {.theorem title="The factor theorem"} +The [factor theorem](http://en.wikipedia.org/wiki/Factor_theorem) states that +$r$ is a root of the polynomial $p$ if *and* only if $(x-r)$ is a factor of $p$. +::: @@ -88,7 +94,7 @@ From this, we see that $f(c) = r$. Hence, when $c$ is a root of $f(x)$, then it --- -The division algorithm for the case of linear term, $(x-c)$, can be carried out by the [synthetic division](http://en.wikipedia.org/wiki/Synthetic_division) algorithm. This algorithm produces $q(x)$ and $r$, a.k.a $f(c)$. The Wikipedia page describes the algorithm well. +The division algorithm for the case of a linear term, $(x-c)$, can be carried out by the [synthetic division](http://en.wikipedia.org/wiki/Synthetic_division) algorithm. This algorithm produces $q(x)$ and $r$, a.k.a $f(c)$. The Wikipedia page describes the algorithm well. The following is an example where $f(x) = x^4 + 2x^2 + 5$ and $g(x) = x-2$: @@ -101,7 +107,7 @@ The following is an example where $f(x) = x^4 + 2x^2 + 5$ and $g(x) = x-2$: 1 2 6 12 29 ``` -The polynomial $f(x)$ is coded in terms of its coefficients ($a_n$, $a_{n-1}$, $\dots$, $a_1$, $a_0$) and is written on the top row. The algorithm then proceeds from left to right. The number just produced on the bottom row is multiplied by $c$ and placed under the coefficient of $f(x)$. Then values are then added to produce the next number. The sequence produced above is `1 2 6 12 29`. The last value (`29`) is $r=f(c)$, the others encode the coefficients of `q(x)`, which for this problem is $q(x)=x^3 + 2x^2 + 6x + 12$. That is, we have written: +The polynomial $f(x)$ is coded in terms of its coefficients ($a_n$, $a_{n-1}$, $\dots$, $a_1$, $a_0$) and is written on the top row. The algorithm then proceeds from left to right. The number just produced on the bottom row is multiplied by $c$ and placed under the coefficient of $f(x)$. Then values are then added to produce the next number. The sequence produced above is `1 2 6 12 29`. The last value (`29`) is $r=f(c)$, the others encode the coefficients of `q(x)`, which for this problem is $q(x)=1x^3 + 2x^2 + 6x + 12$. That is, we have written: $$ @@ -171,48 +177,38 @@ This naive attempt to divide won't "just work" though: (x^4 + 2x^2 + 5) / (x-2) ``` -`SymPy` is fairly conservative in how it simplifies answers, and, as written, there is no compelling reason to change the expressions, though in our example we want it done. +`SymPy` is fairly conservative in how it simplifies answers, and, as written, there is no compelling reason to change the expression, though in this example we want it done. -For this task, `divrem` is available: +For this task, `divrem` is available:^[ +For those who have worked with SymPy within Python, `divrem` is the `div` method renamed, as `Julia`'s `div` method has the generic meaning of returning the quotient.] ```{julia} quotient, remainder = divrem(x^4 + 2x^2 + 5, x - 2) ``` -The answer is a tuple containing the quotient and remainder. The quotient itself could be found with `div` or `÷` and the remainder with `rem`. - - -:::{.callout-note} -## Note -For those who have worked with SymPy within Python, `divrem` is the `div` method renamed, as `Julia`'s `div` method has the generic meaning of returning the quotient. - -::: - -As well, the `apart` function could be used for this task. This function computes the [partial fraction](http://en.wikipedia.org/wiki/Partial_fraction_decomposition) decomposition of a ratio of polynomial functions. - - -```{julia} -apart((x^4 + 2x^2 + 5) / (x-2)) -``` - -The function `together` would combine such terms, as an "inverse" to `apart`. This isn't so much of interest at the moment, but will be when techniques of integration are looked at. +The answer is a tuple containing the quotient and remainder. The quotient itself could be found with `div` or `÷` and the remainder with `rem`.^[There is also the `apart` function that could be used for this task. This function computes the [partial fraction](http://en.wikipedia.org/wiki/Partial_fraction_decomposition) decomposition of a ratio of polynomial functions. The function `together` would combine such terms, as an "inverse" to `apart`. This isn't so much of interest at the moment, but will be when techniques of integration are looked at.] ### The rational root theorem -Factoring polynomials to find roots is a task that most all readers here will recognize, and, perhaps, remember not so fondly. One helpful trick to find possible roots *by hand* is the [rational root theorem](http://en.wikipedia.org/wiki/Rational_root_theorem): if a polynomial has integer coefficients with $a_0 \neq 0$, then any rational root, $p/q$, must have $p$ dividing the constant $a_0$ and $q$ dividing the leading term $a_n$. +Factoring polynomials to find roots is a task that most all readers here will recognize, and, perhaps, remember not so fondly. One helpful trick to find possible roots *by hand* is the rational-root theorem. + +::: {.theorem title="The rational root theorem"} +The [rational root theorem](http://en.wikipedia.org/wiki/Rational_root_theorem) states that +if a polynomial has integer coefficients with $a_0 \neq 0$, then any rational root, $p/q$, must have $p$ dividing the constant $a_0$ and $q$ dividing the leading term $a_n$. +::: -To glimpse why, suppose we have a polynomial with a rational root and integer coefficients. With this in mind, a polynomial with identical roots may be written as $(qx -p)(a_{n-1}x^{n-1}+\cdots a_1 x + a_0)$, where each coefficient is an integer. Multiplying through, we get that the polynomial is $qa_{n-1}x^n + \cdots + pa_0$. So $q$ is a factor of the leading coefficient and $p$ is a factor of the constant. +To glimpse why this is true, suppose we have a polynomial with a rational root and integer coefficients. With this in mind, a polynomial with identical roots may be written as $(qx -p)(a_{n-1}x^{n-1}+\cdots a_1 x + a_0)$, where each coefficient is an integer. Multiplying through, we get that the polynomial is $qa_{n-1}x^n + \cdots + pa_0$. So $q$ is a factor of the leading coefficient and $p$ is a factor of the constant. An immediate consequence is that if the polynomial with integer coefficients is monic, then any rational root must be an integer. -This gives a finite - though possibly large - set of values that can be checked to exhaust the possibility of a rational root. By hand this process can be tedious, though may be speeded up using synthetic division. This task is one of the mainstays of high school algebra where problems are chosen judiciously to avoid too many possibilities. +This gives a finite---though possibly large---set of values that can be checked to exhaust the possibility of a rational root. By hand this process can be tedious, though may be speeded up using synthetic division. This task is one of the mainstays of high school algebra where problems are chosen judiciously to avoid too many possibilities. However, one of the great triumphs of computer algebra is the ability to factor polynomials with integer (or rational) coefficients over the rational numbers. This is typically done by first factoring over modular numbers (akin to those on a clock face) and has nothing to do with the rational root test. @@ -221,7 +217,7 @@ However, one of the great triumphs of computer algebra is the ability to factor `SymPy` can quickly find such a factorization, even for quite large polynomials with rational or integer coefficients. -For example, factoring $p = 2x^4 + x^3 -19x^2 -9x +9$. This has *possible* rational roots of plus or minus $1$ or $2$ divided by $1$, $3$, or $9$ - $12$ possible answers for this modest question. By hand that can be a bit of work, but `factor` does it without fuss: +For example, factoring $p = 2x^4 + x^3 -19x^2 -9x +9$. This has *possible* rational roots of plus or minus $1$ or $2$ divided by $1$, $3$, or $9$---twelve possible answers for this modest question. By hand that can be a bit of work, but `factor` does it without fuss: ```{julia} @@ -235,14 +231,12 @@ factor(p) There is a basic fact about the roots of a polynomial of degree $n$. Before formally stating it, we consider the earlier observation that a polynomial of degree $n$ for large values of $x$ has a graph that looks like the leading term. However, except at $0$, monomials do not cross the $x$ axis, the roots must be the result of the interaction of lower order terms. Intuitively, since each term can contribute only one basic shape up or down, there can not be arbitrarily many roots. In fact, a consequence of the [Fundamental Theorem of Algebra](http://en.wikipedia.org/wiki/Fundamental_theorem_of_algebra) (Gauss) is: - -> A polynomial of degree $n$ with real or complex coefficients has at most $n$ real roots. - - +::: {.theorem title="The fundamental theorem of algebra"} +A polynomial of degree $n$ with real or complex coefficients has at most $n$ real roots. +::: This statement can be proved with the factor theorem and the division algorithm. - In fact the fundamental theorem states that there are exactly $n$ roots, though, in general, one must consider multiple roots and possible complex roots to get all $n$. (Consider $x^2$ to see why multiplicity must be accounted for and $x^2 + 1$ to see why complex values may be necessary.) @@ -265,16 +259,21 @@ $$ \frac{-b \pm \sqrt{b^2 - 4ac}}{2a}. $$ -The discriminant is defined as $b^2 - 4ac$. When this is negative, the square root requires the concept of complex numbers to be defined, and the formula shows the two complex roots are conjugates. When the discriminant is $0$, then the root has multiplicity two, e.g., the polynomial will factor as $a_2(x-r)^2$. Finally, when the discriminant is positive, there will be two distinct, real roots. This figure shows the $3$ cases, that are illustrated by $x^2 -1$, $x^2$ and $x^2 + 1$: +The discriminant is defined as $b^2 - 4ac$. When this is negative, the square root requires the concept of complex numbers to be defined, and the formula shows the two complex roots are conjugates. When the discriminant is $0$, then the root has multiplicity two, e.g., the polynomial will factor as $a_2(x-r)^2$. Finally, when the discriminant is positive, there will be two distinct, real roots. @fig-three-cases-of-quadratic shows the $3$ cases illustrated by $x^2 -1$, $x^2$ and $x^2 + 1$. +::: {#fig-three-cases-of-quadratic} ```{julia} +#| echo: false plot(x^2 - 1, -2, 2, legend=false) # two roots plot!(x^2, -2, 2) # one (double) root plot!(x^2 + 1, -2, 2) # no real root plot!(zero, -2, 2) ``` +Three simple quadratic functions, one with two real roots; one with one; one with none +::: + There are similar formulas for the [cubic](http://en.wikipedia.org/wiki/Cubic_function#General_formula_for_roots) and [quartic](http://en.wikipedia.org/wiki/Quartic_function#General_formula_for_roots) cases. (The [cubic formula](http://arxiv.org/pdf/math/0005026v1.pdf) was known to Cardano in $1545$, though through Tartagli, and the quartic was solved by Ferrari, Cardano's roommate.) @@ -283,8 +282,9 @@ In general, there is no such formula using radicals for $5$th degree polynomials The `factor` function of `SymPy` only finds factors of polynomials with integer or rational coefficients corresponding to rational roots. There are alternatives. +### The solve and solveset functions -Finding roots with `SymPy` can also be done through its `solve` function, a function which also has a more general usage, as it can solve simple expressions or more than one expression. Here we illustrate that `solve` can easily handle quadratic expressions: +Finding roots with `SymPy` can also be done through its `solve` function, a function which also has a more general usage, as it can solve simple expressions or more than one expression for unknown variables. Here we illustrate that `solve` can easily handle quadratic expressions: ```{julia} @@ -293,10 +293,9 @@ solve(x^2 + 2x - 3 ~ 0, x) The answer is a vector of values that when substituted in for the free variable `x` produce $0.$ -We use the `~` notation to define an equation to pass to `solve`. This convention is not necessary here, as `SymPy` will assume an expression passed to solve is an equation set to `0`, but is pedagogically useful. Equations do not have an equals sign, which is reserved for assignment. To solve a more complicated expression of the type $f(x) = g(x),$ one can solve $f(x) - g(x) = 0,$ use the `Eq` function, or use `f ~ g`. +We use the `~` notation to define an equation to pass to `solve`.^[The `solve` function applied to a simple case need not have an equation specified with `~` and need not have a variable to solve for specified. However, we try to be explicit in each example so that exactly what is being asked is as clear as can be.] Equations can not be specified with an equals sign, which is reserved for assignment. To solve a more complicated expression of the type $f(x) = g(x),$ one can solve $f(x) - g(x) = 0$ or use `f ~ g`. - -When the expression to solve has more than one free variable, the variable to solve for should be explicitly stated with a second argument. (The specification above is unnecessary.) For example, here we show that `solve` is aware of the quadratic formula: + For example, here we show that `solve` is aware of the quadratic formula: ```{julia} @@ -348,9 +347,9 @@ Third- and fourth-degree polynomials can be solved in general, with increasingly ```{julia} #| hold: true @syms a[0:3] -p = sum(a*x^(i-1) for (i,a) in enumerate(a)) +p = sum(aᵢ*x^(i-1) for (i,aᵢ) in enumerate(a)) rts = solve(p ~ 0, x) -rts[1] # there are three roots +first(rts) # there are three roots ``` Some fifth degree polynomials are solvable in terms of radicals, however, `solve` will not seem to have luck with this particular fifth degree polynomial: @@ -366,42 +365,39 @@ solve(x^5 - x + 1 ~ 0, x) ### The `roots` function -Related to `solve` is the specialized `roots` function for identifying roots, Unlike solve, it will identify multiplicities. - +Related to `solve` is the specialized `roots` function for identifying roots of a polynomial, Unlike solve, it will identify multiplicities, returning a dictionary as its output. For a polynomial with only one indeterminate the usage is straight forward: ```{julia} -roots((x-1)^2 * (x-2)^2) # solve doesn't identify multiplicities +roots((x-1)^2 * (x-2)^3, x) # solve doesn't identify multiplicities ``` -For a polynomial with symbolic coefficients, the difference between the symbol and the coefficients must be identified. `SymPy` has a `Poly` type to do so. The following call illustrates: +For a polynomial with symbolic coefficients, the difference between the symbol and the coefficients *must* be identified. This can be done with the `Poly` type of `SymPy`,^[The use of `Poly` would look like `q1 = sympy.Poly(p, x); roots(q1)`. The `Poly` function is not exported, so is called by qualifying it with the `sympy` object. This is common when using `SymPy`, as only a small handful of the many functions available are turned into `Julia` functions, the rest are used as would be done in Python. (This is similar, but different than qualifying by a `Julia` module when there are two conflicting names. An example will be the use of the name `roots` in both `SymPy` and `Polynomials` to refer to a function that finds the roots of a polynomial. If both functions were loaded, then the last line in the above example would need to be `SymPy.roots(q)` (note the capitalization.)] or by specifying the variable, as above (which is optional when there is only one free variable): ```{julia} -#| hold: true -@syms a b c +@syms a b c x p = a*x^2 + b*x + c -q1 = sympy.Poly(p, x) # identify `x` as indeterminate; alternatively p.as_poly(x) -roots(q1) +roots(p, x) ``` -:::{.callout-note} -## Note -The sympy `Poly` function must be found within the underlying `sympy` module, a Python object, hence is qualified as `sympy.Poly`. This is common when using `SymPy`, as only a small handful of the many functions available are turned into `Julia` functions, the rest are used as would be done in Python. (This is similar, but different than qualifying by a `Julia` module when there are two conflicting names. An example will be the use of the name `roots` in both `SymPy` and `Polynomials` to refer to a function that finds the roots of a polynomial. If both functions were loaded, then the last line in the above example would need to be `SymPy.roots(q)` (note the capitalization.) - -::: ### Numerically finding roots -The `solve` function can be used to get numeric approximations to the roots. It is as easy as calling `N` on the solutions: +The output of this `solve` call finds $1$ roots, but it is hidden behind a `CRootOf` wrapper: + +```{julia} +@syms x::real +rts = solve(x^5 - x + 1 ~ 0, x) +``` + +Such values can be identified with numeric approximations. It is as easy as calling `N` on the solutions, which are held in a vector: ```{julia} -#| hold: true -rts = solve(x^5 - x + 1 ~ 0, x) N.(rts) # note the `.(` to broadcast over all values in rts ``` @@ -416,13 +412,15 @@ ex = x^7 -3x^6 + 2x^5 -1x^3 + 2x^2 + 1x^1 - 2 solve(ex ~ 0, x) ``` -This finds two of the seven possible roots, the remainder of the real roots can be found numerically: +This identifies exactly two of the up-to-seven possible roots, the remainding real root can be found numerically: ```{julia} N.(solve(ex ~ 0, x)) ``` +This approach can also be used to find complex answers. + ### The solveset function @@ -441,14 +439,13 @@ The `p_rts` object, a `Set`, does not allow indexed access to its elements. For collect(p_rts) ``` -To get the numeric approximation, we can broadcast: +To get the numeric approximation, we can broadcast:^[There is no need to call `collect` before broadcasting, as broadcasting over a set falls back to broadcasting over the iteration of the set and in this case returns a vector.] ```{julia} N.(solveset(p ~ 0, x)) ``` -(There is no need to call `collect`---though you can---as broadcasting over a set falls back to broadcasting over the iteration of the set and in this case returns a vector.) ## Do numeric methods matter when you can just graph? @@ -456,46 +453,56 @@ N.(solveset(p ~ 0, x)) It may seem that certain practices related to roots of polynomials are unnecessary as we could just graph the equation and look for the roots. This feeling is perhaps motivated by the examples given in textbooks to be worked by hand, which necessarily focus on smallish solutions. But, in general, without some sense of where the roots are, an informative graph itself can be hard to produce. That is, technology doesn't displace thinking---it only supplements it. -For another example, consider the polynomial $(x-20)^5 - (x-20) + 1$. In this form we might think the roots are near $20$. However, were we presented with this polynomial in expanded form: $x^5 - 100x^4 + 4000x^3 - 80000x^2 + 799999x - 3199979$, we might be tempted to just graph it to find roots. A naive graph might be to plot over $[-10, 10]$: - +For another example, consider the polynomial $(x-20)^5 - (x-20) + 1$. In this form we might think the roots are near $20$. However, were we presented with this polynomial in expanded form: $x^5 - 100x^4 + 4000x^3 - 80000x^2 + 799999x - 3199979$, we might be tempted to just graph it to find roots.^[Or ask AI for which google gemini gave a wrong answer the first time.] A naive graph might be to plot over $[-10, 10]$: +::: {#fig-plot-expanded-polynomial-naively} ```{julia} p = x^5 - 100x^4 + 4000x^3 - 80000x^2 + 799999x - 3199979 plot(p, -10, 10) ``` +Simple plot of expanded polynomial over $[-10,10]$ domain showing this is a poor choice of viewing window +::: -This seems to indicate a root near $10$. But look at the scale of the $y$ axis. The value at $-10$ is around $-25,000,000$ so it is really hard to tell if $f$ is near $0$ when $x=10$, as the range is too large. +@fig-plot-expanded-polynomial-naively maybe seems to indicate a root near $10$. But look at the scale of the $y$ axis. The value at $-10$ is around $-25,000,000$ so it is really hard to tell if $f$ is near $0$ when $x=10$, as the range is too large. -A graph over $[10,20]$ is still unclear: - +A graph over $[10,20]$ is still unclear, as seen in @fig-plot-expanded-polynomial-a-bit-less-naively. +::: {#fig-plot-expanded-polynomial-a-bit-less-naively} ```{julia} plot(p, 10,20) ``` +Plot of expanded polynomial over $[10, 20]$. Even though there is a zero, it can't be identified from this graph. +::: + We see that what looked like a zero near $10$, was actually a number around $-100,000$. Continuing, a plot over $[15, 20]$ still isn't that useful. It isn't until we get close to $18$ that the large values of the polynomial allow a clear vision of the values near $0$. That being said, plotting anything bigger than $22$ quickly makes the large values hide those near $0$, and might make us think where the function dips back down there is a second or third zero, when only $1$ is the case. (We know that, as this is the same $x^5 - x + 1$ shifted to the right by $20$ units.) - +::: {#fig-plot-expanded-polynomial-a-good-frame} ```{julia} plot(p, 18, 22) ``` +Plot of expanded polynomial using a good domain +::: + Not that it can't be done, but graphically solving for a root here can require some judicious choice of viewing window. Even worse is the case where something might graphically look like a root, but in fact not be a root. Something like $(x-100)^2 + 0.1$ will demonstrate. -For another example, the following polynomial when plotted over $[-5,7]$ appears to have two real roots: - +For another example, the following polynomial when plotted over $[-5,7]$ appears to have two real roots in @fig-mignotte-poly-2-or-3-roots. +::: {#fig-mignotte-poly-2-or-3-roots} ```{julia} h = x^7 - 16129x^2 + 254x - 1 plot(h, -5, 7) ``` +Plot of polynomial that appears to have two real roots +::: -in fact there are three, two are *very* close together: +In fact there are three roots of this polynomial, two are *very* close together: ```{julia} @@ -522,9 +529,17 @@ A polynomial with real coefficients may or may not have real roots. The followin The study of polynomial roots is an old one. In $1637$ Descartes published a *simple* method to determine an upper bound on the number of *positive* real roots of a polynomial. - -> [Descartes' rule of signs](http://en.wikipedia.org/wiki/Descartes%27_rule_of_signs): if $p=a_n x^n + a_{n-1}x^{n-1} + \cdots + a_1x + a_0$ then the number of positive real roots is either equal to the number of sign differences between consecutive nonzero coefficients, or is less than it by an even number. Repeated roots are counted separately. - +::: {.theorem title="Descartes' rule of signs"} +[Descartes' rule of signs](http://en.wikipedia.org/wiki/Descartes%27_rule_of_signs) states that if +$$ +p=a_n x^n + a_{n-1}x^{n-1} + \cdots + a_1x + a_0, \quad a_n \neq 0 +$$ +then the number of positive real roots is either equal to the number +of sign differences between consecutive coefficients +(omitting zero coefficients), +or is less than this count by an even number. +Repeated roots are counted separately. +::: One method of proof (sketched at the end of this section) first shows that in synthetic division by $(x-c)$ with $c > 0$, we must have that any sign change in $q$ is related to a sign change in $p$ and there must be at least one more in $p$. This is then used to show that there can be only as many positive roots as sign changes. That the difference comes in pairs is related to complex roots of real polynomials always coming in pairs. @@ -533,13 +548,13 @@ One method of proof (sketched at the end of this section) first shows that in sy An immediate consequence, is that a polynomial whose coefficients are all non-negative will have no positive real roots. -Applying this to the polynomial $x^5 -x + 1$ we get That the coefficients have signs: `+ 0 0 0 - +` which collapses to the sign pattern `+`, `-`, `+`. This pattern has two changes of sign. The number of *positive* real roots is either $2$ or $0$. In fact there are $0$ for this case. +Applying this to the polynomial $x^5 -x + 1$ we get that the coefficients have signs: `+ 0 0 0 - +` which collapses to the sign pattern `+ - +`. This pattern has two changes of sign. The number of *positive* real roots is either $2$ or $0$. In fact there are $0$ for this case. -What about negative roots? Clearly, any negative root of $p$ is a positive root of $q(x) = p(-x)$, as the graph of $q$ is just that of $p$ flipped through the $y$ axis. But the coefficients of $q$ are the same as $p$, except for the odd-indexed coefficients ($a_1, a_3, \dots$) have a changed sign. Continuing with our example, for $q(x) = -x^5 + x + 1$ we get the new sign pattern `-`, `+`, `+` which yields one sign change. That is, there *must* be a negative real root, and indeed there is, $x \approx -1.1673$. +What about negative roots? Clearly, any negative root of $p$ is a positive root of $q(x) = p(-x)$, as the graph of $q$ is just that of $p$ flipped through the $y$ axis. But the coefficients of $q$ are the same as $p$, except for the odd-indexed coefficients ($a_1, a_3, \dots$) have a changed sign. Continuing with our example, for $q(x) = -x^5 + x + 1$ we get the new sign pattern `- + +` which yields one sign change. That is, there *must* be a negative real root, and indeed there is, $x \approx -1.1673$. -With this knowledge, we could have known that in an earlier example the graph of `p = x^7 - 16129x^2 + 254x - 1` – which indicated two positive real roots – was misleading, as there must be $1$ or $3$ by a count of the sign changes. +With this knowledge, we could have known that in an earlier example the graph of `p = x^7 - 16129x^2 + 254x - 1`---which suggested two positive real roots---was misleading, as there must be $1$ or $3$ by a count of the sign changes. For another example, if we looked at $f(x) = x^5 - 100x^4 + 4000x^3 - 80000x^2 + 799999x - 3199979$ again, we see that there could be $1$, $3$, or $5$ *positive* roots. However, changing the signs of the odd powers leaves all "-" signs, so there are $0$ negative roots. From the graph, we saw just $1$ real root, not $3$ or $5$. We can verify numerically with: @@ -553,7 +568,7 @@ N.(solve(j ~ 0, x)) ### Cauchy's bound on the magnitude of the real roots. -Descartes' rule gives a bound on how many real roots there may be. Cauchy provided a bound on how large they can be. Assume our polynomial is monic (if not, divide by $a_n$ to make it so, as this won't effect the roots). Then any real root is no larger in absolute value than $h = 1 + |a_0| + |a_1| + |a_2| + \cdots + |a_{n-1}|$, (this is expressed in different ways.) +Descartes' rule gives a bound on how many real roots there may be. Cauchy provided a bound on how large any real roots can be. Assume our polynomial is monic (if not, divide by $a_n$ to make it so, as this won't effect the roots). Then any real root is no larger in absolute value than $h = 1 + |a_0| + |a_1| + |a_2| + \cdots + |a_{n-1}|$, (this is expressed in different ways.) To see precisely [why](https://captainblack.wordpress.com/2009/03/08/cauchys-upper-bound-for-the-roots-of-a-polynomial/) this bound works, suppose $x$ is a root with $|x| > 1$ and let $h$ be the bound. Then since $x$ is a root, we can solve $a_0 + a_1x + \cdots + 1 \cdot x^n = 0$ for $x^n$ as: @@ -567,20 +582,23 @@ Which after taking absolute values of both sides, yields by the triangle inequal $$ -|x^n| \leq |a_0| + |a_1||x| + |a_2||x^2| + \cdots |a_{n-1}| |x^{n-1}| \leq (h-1) (1 + |x| + |x^2| + \cdots |x^{n-1}|). +\begin{align*} +\lvert x^n\rvert &\leq \lvert a_0\rvert + \lvert a_1\rvert\lvert x\rvert + \lvert a_2\rvert\lvert x^2\rvert + \cdots \lvert a_{n-1}\rvert \lvert x^{n-1}\rvert \\ +& \leq (h-1) (1 + \lvert x\rvert + \lvert x^2\rvert + \cdots \lvert x^{n-1}\rvert).\\ +\end{align*} $$ -The last sum can be computed using a formula for geometric sums, $(|x^n| - 1)/(|x|-1)$. Rearranging, gives the inequality: +The last sum can be computed using a formula for geometric sums, $(\lvert x^n\rvert - 1)/(\lvert x\rvert-1)$. Rearranging, gives the inequality: $$ -|x| - 1 \leq (h-1) \cdot (1 - \frac{1}{|x^n|} ) \leq (h-1) +\lvert x\rvert - 1 \leq (h-1) \cdot (1 - \frac{1}{\lvert x^n\rvert} ) \leq (h-1) $$ -from which it follows that $|x| \leq h$, as desired. +from which it follows that $\lvert x\rvert \leq h$, as desired. -For our polynomial $x^5 -x + 1$ we have the sum above is $3$. The lone real root is approximately $-1.1673$ which satisfies $|-1.1673| \leq 3$. +For our polynomial $x^5 -x + 1$ we have the sum above is $3$. The lone real root is approximately $-1.1673$ which satisfies $\lvert -1.1673\rvert \leq 3$. @@ -605,8 +623,8 @@ choices = [ "``6``", "``0``" ] -answ = 3 -radioq(choices, answ) +answer = 3 +buttonq(choices, answer) ``` ###### Question @@ -623,8 +641,8 @@ choices = [ "``x^2 - 2x + 2``", "``2``" ] -answ = 2 -radioq(choices, answ) +answer = 2 +buttonq(choices, answer) ``` ###### Question @@ -644,8 +662,8 @@ choices = [ "``x^3 + x^2 - 1``", "``-2x + 2``" ] -answ = 3 -radioq(choices, answ) +answer = 3 +buttonq(choices, answer) ``` ###### Question @@ -676,8 +694,8 @@ choices = [ "``x^5 + 2x^4 + 4x^3 + 8x^2 + 15x + 31``", "``x^4 +2x^3 + 4x^2 + 8x + 15``", "``31``"] -answ = 1 -radioq(choices, answ) +answer = 1 +buttonq(choices, answer) ``` What is $q(x)$? @@ -692,8 +710,8 @@ choices = [ "``x^5 + 2x^4 + 4x^3 + 8x^2 + 15x + 31``", "``x^4 +2x^3 + 4x^2 + 8x + 15``", "``31``"] -answ = 4 -radioq(choices, answ) +answer = 4 +buttonq(choices, answer) ``` What is $r$? @@ -708,8 +726,8 @@ choices = [ "``x^5 + 2x^4 + 4x^3 + 8x^2 + 15x + 31``", "``x^4 +2x^3 + 4x^2 + 8x + 15``", "``31``"] -answ = 5 -radioq(choices, answ) +answer = 5 +buttonq(choices, answer) ``` ###### Question @@ -728,8 +746,8 @@ choices = [ L" $2$ and $3$", L" $(x-2)$ and $(x-3)$", L" $(x+2)$ and $(x+3)$"] -answ = 2 -radioq(choices, answ) +answer = 2 +buttonq(choices, answer) ``` ###### Question @@ -785,8 +803,8 @@ q"[-0.434235, -0.434235, 0.188049, 0.188049, 0.578696, 4.91368]", q"[-0.434235, -0.434235, 0.188049, 0.188049]", q"[0.578696, 4.91368]", q"[-0.434235+0.613836im, -0.434235-0.613836im]"] -answ = 3 -radioq(choices, answ) +answer = 3 +buttonq(choices, answer) ``` ###### Question @@ -863,8 +881,8 @@ Let $f(x) = x^5 - 4x^4 + x^3 - 2x^2 + x$. What does Cauchy's bound say is the la ```{julia} #| hold: true #| echo: false -answ = 1 + 4 + 1 + 2 + 1 -numericq(answ) +answer = 1 + 4 + 1 + 2 + 1 +numericq(answer) ``` What is the largest magnitude of a real root? @@ -875,14 +893,14 @@ What is the largest magnitude of a real root? #| echo: false f(x) = x^5 - 4x^4 + x^3 - 2x^2 + x rts = find_zeros(f, -5..5) -answ = maximum(abs.(rts)) -numericq(answ) +answer = maximum(abs.(rts)) +numericq(answer) ``` ###### Question -As $1 + 2 + 3 + 4$ is $10$, Cauchy's bound says that the magnitude of the largest real root of $x^3 - ax^2 + bx - c$ is $10$ where $a,b,c$ is one of $2,3,4$. By considering all 6 such possible polynomials (such as $x^3 - 3x^2 + 2x - 4$) what is the largest magnitude or a root? +As $1 + 2 + 3 + 4$ is $10$, Cauchy's bound says that the magnitude of the largest real root of $x^3 - ax^2 + bx - c$ is $10$ where $a,b,c$ is one of $2,3,4$. By considering all 6 such possible polynomials (such as $x^3 - 3x^2 + 2x - 4$) what is the largest magnitude of a root? ```{julia} @@ -921,8 +939,8 @@ choices = [ "``2x^2``", "``x``", "``2x``"] -answ = 1 -radioq(choices, answ) +answer = 1 +buttonq(choices, answer) ``` * True or false, the $degree$ of $T_n(x)$ is $n$: (Look at the defining relation and reason this out). @@ -950,19 +968,22 @@ The Chebyshev polynomials have the property that in fact all $n$ roots are real, @syms x p = 16x^5 - 20x^3 + 5x rts = N.(solve(p)) -answ = maximum(norm.(rts)) -numericq(answ) +answer = maximum(norm.(rts)) +numericq(answer) ``` * Plotting `p` over the interval $[-2,2]$ does not help graphically identify the roots: - +::: {#fig-plot-polynomial-over-minus2-2-not-roots-identifiable} ```{julia} #| hold: true plot(16x^5 - 20x^3 + 5x, -2, 2) ``` +Plot of $p = 16x^5 - 20x^3 + 5x$ over $[-2, 2]$ +::: -Does graphing over $[-1,1]$ show clearly the $5$ roots? + +Does graphing $p$ over $[-1,1]$ show clearly the $5$ roots? ```{julia} @@ -986,19 +1007,19 @@ Let `var(p)` be the number of sign changes and `pos(p)` the number of positive r First: For a monic $p$ if $p_0 < 0$ then `var(p)` is odd and if $p_0 > 0$ then `var(p)` is even. -This is true for degree $n=1$ the two sign patterns under the assumption are `+-` ($p_0 < 0$) or `++` ($p_0 > 0$). If it is true for degree $n-1$, then the we can consider the sign pattern of such an $n$ degree polynomial having one of these patterns: `+...+-` or `+...--` (if $p_0 < 0$) or `+...++` or `+...-+` if ($p_0>0$). An induction step applied to all but the last sign for these four patterns leads to even, odd, even, odd as the number of sign changes. Incorporating the last sign leads to odd, odd, even, even as the number of sign changes. +This is true for degree $n=1$ the two sign patterns under the assumption are `+-` ($p_0 < 0$) or `++` ($p_0 > 0$). If it is true for degree $n-1$, then the we can consider the sign pattern of such an $n$ degree polynomial having one of these patterns: `+...+(0?)-` or `+...-(0?)-` (if $p_0 < 0$) or `+...+(0?)+` or `+...-(0?)+` if ($p_0>0$). The presence of `(0?)` is to show there may be zero, one, or more zeros in that position. An induction step applied to all but the last sign for these four patterns (that is the polynomial after dropping the constant and any of the `0?` terms) leads to even, odd, even, odd as the number of sign changes. Incorporating the last sign leads to odd, odd, even, even as the number of sign changes. Second: For a monic $p$ if $p_0 < 0$ then `pos(p)` is *odd*, if $p_0 > 0$ then `pos(p)` is even. -This is clearly true for **monic** degree $1$ polynomials: if $c$ is positive $p = x - c$ has one real root (an odd number) and $p = x + c$ has $0$ real roots (an even number). Now, suppose $p$ has degree $n$ and is monic. Then as $x$ goes to $\infty$, it must be $p$ goes to $\infty$. +This is clearly true for *monic* degree $1$ polynomials: if $c$ is positive $p = x - c$ has one positive, real root (an odd number) and $p = x + c$ has zero *positive*, real roots (an even number). Now, suppose $p$ has degree $n$ and is monic. Then as $x$ goes to $\infty$, it must be $p$ goes to $\infty$. -If $p_0 < 0$ then there must be a positive real root, say $r$, (Bolzano's intermediate value theorem). Dividing $p$ by $(x-r)$ to produce $q$ requires $q_0$ to be *positive* and of lower degree. By *induction* $q$ will have an even number of roots. Add in the root $r$ to see that $p$ will have an **odd** number of roots. +If $p_0 < 0$ then there must be a positive real root, say $r$, (Bolzano's intermediate value theorem). Dividing $p$ by $(x-r)$ to produce $q$ requires $q_0$ to be *positive* and of lower degree. By *induction* $q$ will have an even number of roots. Add in the root $r$ to see that $p$ will have an *odd* number of roots. -Now consider the case $p_0 > 0$. There are two possibilities either `pos(p)` is zero or positive. If `pos(p)` is $0$ then there are an even number of roots. If `pos(p)` is positive, then call $r$ one of the real positive roots. Again divide by $x-r$ to produce $p = (x-r) \cdot q$. Then $q_0$ must be *negative* for $p_0$ to be positive. By *induction* $q$ must have an odd number of roots, meaning $p$ must have an even numbers. +Now consider the case $p_0 > 0$. There are two possibilities either `pos(p)` is zero or positive. If `pos(p)` is $0$ then there are an even number of roots. If `pos(p)` is positive, then call $r$ one of the real positive roots. Again divide by $x-r$ to produce $p = (x-r) \cdot q$. Then $q_0$ must be *negative* for $p_0$ to be positive. By *induction* $q$ must have an odd number of roots, meaning $p$ must have an even number. So there is parity between `var(p)` and `pos(p)`: if $p$ is monic and $p_0 < 0$ then both `var(p)` and `pos(p)` are both odd; and if $p_0 > 0$ both `var(p)` and `pos(p)` are both even. @@ -1017,7 +1038,7 @@ As $p = (x-c)q$ we must have the leading term is $p_nx^n = x \cdot q_{n-1} x^{n- + - - - + - + + 0 ``` -But actually, we can fill in more, as the second row is formed by multiplying a positive $c$: +The pattern for `q` only has a leading `+` and ending `0`, the pattern shown is just an arbitrary pattern to make a general point. From any pattern for `q`, we can we can fill in more. As the second row is formed by multiplying a positive $c$ the sign in the second row is the same as the sign in the bottom row shifted over 1 to the left. ```{verbatim} @@ -1030,6 +1051,15 @@ But actually, we can fill in more, as the second row is formed by multiplying a What's more, using the fact that to get `0` the two summands must differ in sign and to have a `?` plus `+` yield a `-`, the `?` must be `-` (and reverse), the following must be the case for the signs of `p`: +```{verbatim} + + - ? ? ? ? ? ? - ++ + - - - + - + + + ----------------- + + - - - + - + + 0 +``` + +Going further, to get a `+` in the bottom row with a `-` in the middle row, the top row must be `+` and similarly if `-` and `+`. Filling these in we have this pattern determined. + ```{verbatim} + - ? ? + - + ? - + + - - - + - + + diff --git a/quarto/precalc/polynomials_package.qmd b/quarto/precalc/polynomials_package.qmd index ec878ea..7f15756 100644 --- a/quarto/precalc/polynomials_package.qmd +++ b/quarto/precalc/polynomials_package.qmd @@ -18,13 +18,13 @@ import SymPy # imported only: some functions, e.g. degree, need qualification --- -While `SymPy` can be used to represent polynomials, there are also native `Julia` packages available for this and related tasks. These packages include `Polynomials`, `MultivariatePolynomials`, and `AbstractAlgebra`, among many others. (A search on [juliahub.com](https://juliahub.com) found almost $100$ packages matching "polynomial".) We will look at the `Polynomials` package in the following, as it is straightforward to use and provides the features we are looking at for *univariate* polynomials. +While `SymPy` can be used to represent polynomials, there are also native `Julia` packages available for this and related tasks. These packages include `Polynomials`, `MultivariatePolynomials`, and `AbstractAlgebra`, among many others.^[A search on [juliahub.com](https://juliahub.com) found over $50$ packages matching "polynomial".] In this section, we look at the `Polynomials` package, as it is straightforward to use and provides the features we are looking at for *univariate* polynomials. ## Construction -The polynomial expression $p = a_0 + a_1\cdot x + a_2\cdot x^2 + \cdots + a_n\cdot x^n$ can be viewed mathematically as a vector of numbers with respect to some "basis", which for standard polynomials, as above, is just the set of monomials, $1, x, x^2, \dots, x^n$. With this viewpoint, the polynomial $p$ can be identified with the vector `[a0, a1, a2, ..., an]`. The `Polynomials` package provides a wrapper for such an identification through the `Polynomial` constructor. We have previously loaded this add-on package. +The polynomial expression $p = a_0 + a_1\cdot x + a_2\cdot x^2 + \cdots + a_n\cdot x^n$ can be viewed mathematically as a collection of numbers ($a_0$, $a_1$, $\dots$, $a_n$) with respect to some "basis", which for standard polynomials, as above, is just the set of monomials, $1, x, x^2, \dots, x^n$. With this viewpoint, the polynomial $p$ can be identified with the vector `[a0, a1, a2, ..., an]`. The `Polynomials` package provides a wrapper for such an identification through the `Polynomial` constructor. We have previously loaded this add-on package. To illustrate, the polynomial $p = 3 + 4x + 5x^2$ is constructed with @@ -34,7 +34,7 @@ To illustrate, the polynomial $p = 3 + 4x + 5x^2$ is constructed with p = Polynomial([3,4,5]) ``` -where the vector `[3,4,5]` represents the coefficients. The polynomial $q = 3 + 5x^2 + 7x^4$ has some coefficients that are $0$, these too must be indicated on construction, so we would have: +where the vector `[3,4,5]` represents the coefficients. The polynomial $q = 3 + 5x^2 + 7x^4$ differs from $p$ in that some coefficients that are $0$. These must be indicated on construction. To represent $q$, we have: ```{julia} @@ -87,7 +87,7 @@ x = variable(p) ``` This variable is a `Polynomial` object that prints as `x`. -These variables can be manipulated as any polynomial. We can then construct polynomials through expressions like: +These variables can be algebraically manipulated as any polynomial. For example, wWe could construct polynomials through expressions like: ```{julia} @@ -150,19 +150,23 @@ r + s ## Graphs -Polynomial objects have a plot recipe defined – plotting from the `Plots` package should be as easy as calling `plot`: - +Polynomial objects have a plot recipe defined---plotting from the `Plots` package should be as easy as calling `plot` passing the polynomial as the first argument: +::: {#fig-plot-polynomial-using-basic-recipe} ```{julia} -plot(r, legend=false) # suppress the legend +plot(r; legend=false) # suppress the legend ``` +Plot of polynomial `r` using the basic plot recipe +::: The choice of domain is heuristically identified; it can be manually adjusted, as with: - +::: {#fig-plot-polynomial-using-specified-interval-1point5-to-2point5} ```{julia} plot(r, 1.5, 2.5; legend=false) ``` +Plot of polynomial `r` over the interval $[3/2, 5/2]$ +::: ## Roots @@ -177,7 +181,7 @@ p = x^5 - x - 1 roots(p) ``` -A consequence of the fundamental theorem of algebra and the factor theorem is that any fifth degree polynomial with integer coefficients has $5$ roots, where possibly some are complex. For real coefficients, these complex values must come in conjugate pairs, which can be observed from the output. The lone real root is approximately `1.1673039782614187`. This value being a numeric approximation to the irrational root. +A consequence of the fundamental theorem of algebra and the factor theorem is that any fifth degree polynomial with integer coefficients has $5$ roots, where possibly some are complex. For real coefficients, these complex values must come in conjugate pairs, which can be observed from the output. The lone real root (with imaginary part that is `0.0im`) is approximately `1.16730397⋯`. This value being a numeric approximation to the irrational root. :::{.callout-note} @@ -196,17 +200,9 @@ p = (x-1)^5 roots(p) ``` -The `Polynomials` package has the `multroot` function to identify roots of polynomials when there are multiplicities expected. This function is not exported, so is called through: +Internally, the `Polynomials` package has a routine^[`Polynomials.Multroot.multroot`] to identify multiple roots and this is used on conversion to a `FactoredPolynomial`: -```{julia} -#| hold: true -x = variable() -p = (x-1)^5 -Polynomials.Multroot.multroot(p) -``` - -Converting to the `FactoredPolynomial` type also does this work: ```{julia} convert(FactoredPolynomial, p) @@ -248,15 +244,36 @@ The line is then given from the *point-slope* form by, say, $y= y_0 + m\cdot (x- A line, $y=mx+b$ can be a linear polynomial or a constant depending on $m$, so we could say $2$ points determine a polynomial of degree $1$ or less. Similarly, $3$ distinct points determine a degree $2$ polynomial or less, $\dots$, $n+1$ distinct points determine a degree $n$ or less polynomial. Finding a polynomial, $p$ that goes through $n+1$ points (i.e., $p(x_i)=y_i$ for each $i$) is called [polynomial interpolation](https://en.wikipedia.org/wiki/Polynomial_interpolation). The main theorem is: +::: {.theorem title="Polynomial interpolation theorem"} -> *Polynomial interpolation theorem*: There exists a unique polynomial of degree $n$ or less that interpolates the points $(x_0,y_0), (x_1,y_1), \dots, (x_n, y_n)$ when the $x_i$ are distinct. +For $n+1$ points $(x_0,y_0), (x_1,y_1), \dots, (x_n, y_n)$ with distinct $x_i$ there exists a *unique* polynomial. $p$, of degree $n$ or less that interpolates each point. That is, $p(x_i) = y_i$ for every $i$. +::: (Uniqueness follows as suppose $p$ and $q$ satisfy the above, then $(p-q)(x) = 0$ at each of the $x_i$ and is of degree $n$ or less, so must be the $0$ polynomial. Existence comes by construction. See the Lagrange basis in the questions.) -Knowing we can succeed, we approach the problem of $3$ points, say $(x_0, y_0)$, $(x_1,y_1)$, and $(x_2, y_2)$. There is a polynomial $p = a\cdot x^2 + b\cdot x + c$ with $p(x_i) = y_i$. This gives $3$ equations for the $3$ unknown values $a$, $b$, and $c$: + +Suppose three points on a polynomial are $(1,3)$, $(2,1)$, $(3,2)$. Find the degree $2$ (or less) polynomial that interpolates the points. + + Numerically, the `fit` function from the `Polynomials` package will return the interpolating polynomial. + + +```{julia} +p = fit(Polynomial, [1,2,3], [3,1,2]) +``` + +And note: + + +```{julia} +p(1) == 3, p(2) == 1, p(3) == 2 +``` + +We *can* find a general formula for the interpolating polynomial using `SymPy` for three points, say $(x_0, y_0)$, $(x_1,y_1)$, and $(x_2, y_2)$. The theorem says there is a polynomial $p = a\cdot x^2 + b\cdot x + c$, with any of $a$, $b$, or $c$ *possibly* $0$, satisfying $p(x_i) = y_i$. + +The interpolating condition leads to three equations: $$ @@ -268,69 +285,69 @@ a\cdot x_2^2 + b\cdot x_2 + c &= y_2\\ $$ -Solving this with `SymPy` is tractable. A generator is used below to create the $3$ equations; the `zip` function is a simple means to iterate over $2$ or more iterables simultaneously: +Solving this with `SymPy` is tractable. A generator is used below to create the three equations: ```{julia} SymPy.@syms a b c xs[0:2] ys[0:2] -eqs = tuple((a*xi^2 + b*xi + c ~ yi for (xi,yi) in zip(xs, ys))...) +eqs = [a*xi^2 + b*xi + c ~ yi for (xi,yi) in zip(xs, ys)] abc = SymPy.solve(eqs, (a,b,c)) ``` -As can be seen, the terms do get quite unwieldy when treated symbolically. Numerically, the `fit` function from the `Polynomials` package will return the interpolating polynomial. To compare, - - -```{julia} -fit(Polynomial, [1,2,3], [3,1,2]) -``` - -and we can compare that the two give the same answer with, for example: +As can be seen, the terms do get quite unwieldy when treated symbolically. +We can compare with the numeric fit. For example, the value of `b` numerically was `6.5` and symbolically is identical and can be found as follows: ```{julia} abc[b]((xs .=> [1,2,3])..., (ys .=> [3,1,2])...) ``` -(Ignore the tricky way of substituting in each value of `xs` and `ys` for the symbolic values in `x` and `y`.) +(Ignore the tricky way of substituting in each value of `xs` and `ys` for the symbolic values in `x` and `y`, it is shortcut to `xs₀=>1, xs₁=>2, xs₂=>3, ...`.) -##### Example Inverse quadratic interpolation +##### Example: Inverse quadratic interpolation -A related problem, that will arise when finding iterative means to solve for zeros of functions, is *inverse* quadratic interpolation. That is finding $q$ that goes through the points $(x_0,y_0), (x_1, y_1), \dots, (x_n, y_n)$ satisfying $q(y_i) = x_i$. (That is $x$ and $y$ are reversed, as with inverse functions.) For the envisioned task, where the inverse quadratic function intersects the $x$ axis is of interest, which is at the constant term of the polynomial (as it is like the $y$ intercept of typical polynomial). Let's see what that is in general by replicating the above steps (though now the assumption is the $y$ values are distinct): +A related problem, that will arise when finding iterative means to solve for zeros of functions, is *inverse* quadratic interpolation. That is finding $q$ that goes through the points $(x_0,y_0), (x_1, y_1), \dots, (x_n, y_n)$ satisfying $q(y_i) = x_i$. (That is $x$ and $y$ are reversed, as with inverse functions.) For the envisioned task, where the inverse quadratic function intersects the $x$ axis is of interest, which is at the constant term of the polynomial (as it is like the $y$ intercept of typical polynomial). +To find this for the points $(1,3)$, $(2,1)$, $(3,2)$ we flip around the order above and use a `y` variable to distinguish: ```{julia} -#| hold: true -SymPy.@syms a b c xs[0:2] ys[0:2] -eqs = tuple((a*yi^2 + b*yi + c ~ xi for (xi, yi) in zip(xs,ys))...) -abc = SymPy.solve(eqs, (a,b,c)) -abc[c] +ip = fit(Polynomial, [3,1,2], [1,2,3]; var= :y) ``` -We can graphically see the result for the specific values of `xs` and `ys` as follows: +We see that $-2$ (or `ip(0)`) is the $x$ intercept. +@fig-inverse-quadratic-interpolation shows the $x$ intercept of the inverse quadratic interpolating polynomial for the specific values of `xs` and `ys` as follows: + +::: {#fig-inverse-quadratic-interpolation} + ```{julia} #| hold: true #| echo: false -SymPy.@syms a b c xs[0:2] ys[0:2] -eqs = tuple((a*yi^2 + b*yi + c ~ xi for (xi, yi) in zip(xs,ys))...) -abc = SymPy.solve(eqs, (a,b,c)) -abc[c] +let + SymPy.@syms a b c xs[0:2] ys[0:2] + eqs = tuple((a*yi^2 + b*yi + c ~ xi for (xi, yi) in zip(xs,ys))...) + abc = SymPy.solve(eqs, (a,b,c)) -𝒙s, 𝒚s = [1,2,3], [3,1,2] -q = fit(Polynomial, 𝒚s, 𝒙s) # reverse -# plot -us = range(-1/4, 4, length=100) -vs = q.(us) -plot(vs, us, legend=false) -scatter!(𝒙s, 𝒚s) -plot!(zero) -x0 = abc[c]((xs .=> 𝒙s)..., (ys .=> 𝒚s)...) -scatter!([SymPy.N(x0)], [0], markershape=:star) + 𝒙s, 𝒚s = [1,2,3], [3,1,2] + q = fit(Polynomial, 𝒚s, 𝒙s; var=:y) # reverse + # plot + us = range(-1/8, 3.5, length=100) + vs = q.(us) + plot(vs, us; xticks=-4:3, legend=false) + scatter!(𝒙s, 𝒚s; marker=(3, :blue) ) + plot!(zero) + scatter!([(q(0), 0)], marker=(6,:star,:red)) +end ``` +Plot of inverse quadratic interpolating polynomial through three points with the $x$ intercept marked with a star + +::: + + ## Questions @@ -382,15 +399,16 @@ numericq(length(st)) ###### Question -Mathematically we say the $0$ polynomial has no degree. What convention does `Polynomials` use? (Look at `degree(zero(Polynomial))`.) +Mathematically we say the $0$ polynomial has no degree. What convention does `Polynomials` use? ```{julia} #| hold: true #| echo: false choices = ["`nothing`", "`-1`", "`0`", "`Inf`", "`-Inf`"] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +explanation = "Look at `degree(zero(Polynomial))`" +buttonq(choices, answer; explanation) ``` ###### Question @@ -424,7 +442,7 @@ Make graphs of both `q` and `sin`. Over which interval is the approximation (vis choices = ["``[0,1]``", "``[0,\\pi]``", "``[0,2\\pi]``"] -radioq(choices, 1, keep_order=true) +buttonq(choices, 1) ``` (This [blog post](https://www.nullhardware.com/blog/fixed-point-sine-and-cosine-for-embedded-systems/) shows how this approximation is valuable under some specific circumstances.) @@ -459,8 +477,10 @@ A basis used by [Lagrange](https://en.wikipedia.org/wiki/Lagrange_polynomial) is $$ -l_i(x) = \prod_{0 \leq j \leq n; j \ne i} \frac{x-x_j}{x_i - x_j} = -\frac{(x-x_0)\cdot(x-x_1)\cdot \cdots \cdot (x-x_{i-1}) \cdot (x-x_{i+1}) \cdot \cdots \cdot (x-x_n)}{(x_i-x_0)\cdot(x_i-x_1)\cdot \cdots \cdot (x_i-x_{i-1}) \cdot (x_i-x_{i+1}) \cdot \cdots \cdot (x_i-x_n)}. +\begin{align*} +l_i(x) &= \prod_{0 \leq j \leq n; j \ne i} \frac{x-x_j}{x_i - x_j} \\ +&= \frac{(x-x_0)\cdot(x-x_1)\cdot \cdots \cdot (x-x_{i-1}) \cdot (x-x_{i+1}) \cdot \cdots \cdot (x-x_n)}{(x_i-x_0)\cdot(x_i-x_1)\cdot \cdots \cdot (x_i-x_{i-1}) \cdot (x_i-x_{i+1}) \cdot \cdots \cdot (x_i-x_n)}. +\end{align*} $$ That is $l_i(x)$ is a product of terms like $(x-x_j)/(x_i-x_j)$ *except* when $j=i$. @@ -486,7 +506,7 @@ All terms like ``(x-x_j)/(x_0 - x_j)`` will be ``1`` when ``x=x_0`` and these ar """, "The term ``(x_0-x_0)`` will be ``0``, so the product will be zero" ] -radioq(choices, 1) +buttonq(choices, 1) ``` What is the value of $l_i(x_i)$? @@ -518,7 +538,7 @@ The term like ``(x-x_1)/(x_0 - x_1)`` will be ``0`` when ``x=x_1`` and so the pr """, "The term ``(x-x_1)/(x_0-x_1)`` is omitted from the product, so the answer is non-zero." ] -radioq(choices, 1) +buttonq(choices, 1) ``` What is the value of $l_i(x_j)$ *if* $i \ne j$? @@ -544,12 +564,44 @@ What is the value of $p(x_j)$? #| hold: true #| echo: false choices = ["``0``", "``1``", "``y_j``"] -radioq(choices, 3) +buttonq(choices, 3) ``` This last answer is why $p$ is called an *interpolating* polynomial and this question shows an alternative way to identify interpolating polynomials from solving a system of linear equations. +###### Question + +Symbolically, the inverse quadratic polynomial through three points can be identified symbolically using a pattern similar to that shown for polynomial interpolation. This shows the formula for the $x$ intercept: + + +```{julia} +#| hold: true +SymPy.@syms a b c xs[0:2] ys[0:2] +eqs = [a*yi^2 + b*yi + c ~ xi for (xi, yi) in zip(xs,ys)] +abc = SymPy.solve(eqs, (a,b,c)) +abc[c] +``` + +As with the example, plug in the following values for the $x$ and $y$ coordinates of the three points in the example above: + +```{julia} +#| echo: false +abc[c]((xs .=> [1,2,3])..., (ys .=> [3,1,2])...) # answer not shown +``` + +What value is identified? + +```{julia} +#| echo: false +val = -2 +numericq(val) +``` + + + + + ###### Question @@ -581,7 +633,7 @@ It is ``0\cdot T_0(x) + 1\cdot T_1(x) + 2\cdot T_2(x) + 3\cdot T_3(x) = 0`` raw""" It is ``0\cdot T_0(x) + 1\cdot T_1(x) + 2\cdot T_2(x) + 3\cdot T_3(x) = -2 - 8\cdot x + 4\cdot x^2 + 12\cdot x^3`` """] -radioq(choices, 3) +buttonq(choices, 3) ``` :::{.callout-note} diff --git a/quarto/precalc/rational_functions.qmd b/quarto/precalc/rational_functions.qmd index c049075..a2953e0 100644 --- a/quarto/precalc/rational_functions.qmd +++ b/quarto/precalc/rational_functions.qmd @@ -34,17 +34,21 @@ The rational numbers are simply ratios of integers, of the form $p/q$ for non-ze We know that polynomials have nice behaviors due to the following facts: - * Behaviors at $-\infty$, $\infty$ are known just from the leading term. - * There are possible wiggles up and down, the exact behavior depends on intermediate terms, but there can be no more than $n-1$ wiggles. - * The number of real zeros is no more than $n$, the degree of the polynomial. +* Behaviors at $-\infty$, $\infty$ are known just from the leading term. + +* There are possible wiggles up and down, the exact behavior depends on intermediate terms, but there can be no more than $n-1$ wiggles. + +* The number of real zeros is no more than $n$, the degree of the polynomial. Rational functions are not quite so nice: - * behavior at $-\infty$ and $\infty$ can be like a polynomial of any degree, including constants - * behaviour at any value x can blow up due to division by $0$ - rational functions, unlike polynomials, need not be always defined - * The function may or may not cross zero, even if the range includes every other point, as the graph of $f(x) =1/x$ will show. +* behavior at $-\infty$ and $\infty$ can be like a polynomial of any degree, including constants + +* behaviour at any value x can blow up due to division by $0$---rational functions, unlike polynomials, need not be always defined + +* The function may or may not cross zero, even if the range includes every other point, as the graph of $f(x) =1/x$ illustrates. Here, as with our discussion on polynomials, we are interested for now in just a few properties: @@ -65,28 +69,34 @@ $$ f(x) = \frac{(x-1)^2 \cdot (x-2)}{(x+3) \cdot (x-3)}. $$ +::: {#fig-plot-rational-f-over-minus10-10-bad-choice} ```{julia} f(x) = (x-1)^2 * (x-2) / ((x+3)*(x-3) ) plot(f, -10, 10) ``` -We would be hard pressed to answer any of the three questions above from the graph, though, on inspection, we might think the strange spikes have something to do with $x$ values where $q(x)=0$. +Basic plot of the rational function $f(x)$ over $[-10,10]$. As no extra effort was made to adjust the $y$-viewing window, the graphic is not to useful for identifying basic properties of the function. +::: + +We would be hard pressed to answer any of the three questions above from the @fig-plot-rational-f-over-minus10-10-bad-choice, though, on inspection, we might think the strange spikes have something to do with $x$ values where $q(x)=0$. The question of big or small $x$ is not answered well with this graph, as the spikes dominate the scale of the $y$-axis. Setting a much larger viewing window illuminates this question: - +::: {#fig-plot-rational-f-over-minus100-100-can-see-slant-asymptote} ```{julia} plot(f, -100, 100) ``` +Plot of rational function $f(x)$ over $[-100, 100]$ allowing the slant asymptote to be identified +::: We can see from this, that the function eventually looks like a slanted straight line. The *eventual* shape of the graph is something that can be determined just from the two leading terms. The spikes haven't vanished completely. It is just that with only a few hundred points to make the graph, there aren't any values near enough to the problem to make a large spike. The spikes happen because the function has a *vertical asymptote* at these values. Though not quite right, it is reasonable to think of the graph being made by selecting a few hundred points in the specified domain, computing the corresponding $y$ values, plotting the pairs, and finally connecting the points with straight line segments. Near a vertical asymptote the function values can be arbitrarily large in absolute values, though at the vertical asymptote the function is undefined. This graph doesn't show such detail. -The spikes will be related to the points where $q(x) = 0$, though not necessarily all of them – not all such points will produce a vertical asymptote. +The spikes will be related to the points where $q(x) = 0$, though not necessarily all of them. Where the function crosses $0$ is very hard to tell from these two graphs. As well, other finer features, such as local peaks or valleys, when present, can be hard to identify as the $y$-scale is set to accommodate the asymptotes. Working around the asymptotes requires some extra effort. Strategies are discussed herein. @@ -146,12 +156,14 @@ h = g(x) # a symbolic expression apart(h) ``` -This decomposition breaks the rational expression into two pieces: $x-4$ and $40/(3x+9) + 2/(3x-9)$. The first piece would have a graph that is the line with slope $1$ and $y$-intercept $4$. As $x$ goes to $\infty$, the second piece will clearly go towards $0,$ as this simple graph shows: - +This decomposition breaks the rational expression into two pieces: $x-4$ and $40/(3x+9) + 2/(3x-9)$. The first piece would have a graph that is the line with slope $1$ and $y$-intercept $4$. As $x$ goes to $\infty$, the second piece will clearly go towards $0,$ as @fig-plot-aparth-minus-x-minus-4-over-10-100 shows. +::: {#fig-plot-aparth-minus-x-minus-4-over-10-100} ```{julia} plot(apart(h) - (x - 4), 10, 100) ``` +Plot of just remainder term of the rational function $g(x)$ after reduction showing the decay towards $0$ as $x$ gets large +::: Similarly, a plot over $[-100, -10]$ would show decay towards $0$, though in that case from below. Combining these two facts then, it is now no surprise that the graph of the rational function $f(x)$ should approach a straight line, in this case $y=x-4$, as $x \rightarrow \pm \infty$. @@ -176,16 +188,19 @@ $$ The terms $(1 - 4/x + 5/x^2 - 2/x^3)$ and $(1 - 9/x^2)$ go towards $1$ as $x \rightarrow \pm \infty$, as each term with $x$ goes towards $0$. So the dominant terms comes from the ratio of the leading terms, $x^3$ and $x^2$. This ratio is $x$, so their will be an asymptote around a line with slope $1$. (The fact that the asymptote is $y=x-4$ takes a bit more work, as a division step is needed.) -Just by looking at the ratio of the two leading terms, the behaviour as $x \rightarrow \pm \infty$ can be discerned. If this ratio is of: +Just by looking at the ratio of the two leading terms, the behaviour as $x \rightarrow \pm \infty$ can be discerned. If this ratio is of the form: - * the form $c x^m$ with $m > 1$ then the shape will follow the polynomial growth of the monomial $c x^m$. - * the form $c x^m$ with $m=1$ then there will be a line with slope $c$ as a *slant asymptote*. - * the form $cx^0$ with $m=0$ (or just $c$) then there will be a *horizontal asymptote* $y=c$. - * the form $c/x^{m}$ with $m > 0$ then there will be a horizontal asymptote $y=0$, or the $y$ axis. +* $c x^m$ with $m > 1$ then the shape will follow the polynomial growth of the monomial $c x^m$; + +* $c x^m$ with $m=1$ then there will be a line with slope $c$ as a *slant asymptote*; + +* $cx^0$ with $m=0$ (or just $c$) then there will be a *horizontal asymptote* $y=c$; + +* $c/x^{m}$ with $m > 0$ then there will be a horizontal asymptote $y=0$, or the $y$ axis. -To expand on the first points where the degree of the numerator is greater than that of the denominator, we have from the division algorithm that if $a(x)$ is the numerator and $b(x)$ the denominator, then $a(x)/b(x) = q(x) + r(x)/b(x)$ where the degree of $b(x)$ is greater than the degree of $r(x)$, so the right-most term will have a horizontal asymptote of $0$. This says that the graph will eventually approach the graph of $q(x)$, giving more detail than just saying it follows the shape of the leading term of $q(x)$, at the expense of the work required to find $q(x)$. +To expand on the first point, where the degree of the numerator is greater than that of the denominator, we have from the division algorithm that if $a(x)$ is the numerator and $b(x)$ the denominator, then $a(x)/b(x) = q(x) + r(x)/b(x)$ where the degree of $b(x)$ is greater than the degree of $r(x)$, so the right-most term will have a horizontal asymptote of $0$. This says that the graph will eventually approach the graph of $q(x)$, giving more detail than just saying it follows the shape of the leading term of $q(x)$, at the expense of the work required to find $q(x)$. ### Examples @@ -217,8 +232,7 @@ Then we can see that the ratio of the leading terms is $x^5 / (5x^4) = (1/5)x$. ```{julia} #| hold: true p = (x^5 - 2x^4 + 3x^3 - 4x^2 + 5) / (5x^4 + 4x^3 + 3x^2 + 2x + 1) -quo, rem = divrem(numerator(p), denominator(p)) # or apart(p) -quo +div(numerator(p), denominator(p)) # or apart(p) ``` --- @@ -240,23 +254,23 @@ bottom = x-1 quo, rem = divrem(top, bottom) ``` -The graph has nothing in common with the graph of the quotient for small $x$ +@fig-asymptote-matches-for-large-x shows that for small $x$ ($\lvert x\rvert \approx 0$) the graph of the rational function and `quo` are not similar, but that they are when $x$ is larger. + +::: {#fig-asymptote-matches-for-large-x} ```{julia} -plot(top/bottom, -3, 3) -plot!(quo, -3, 3) +#| echo: false +p1 = plot(top/bottom, -3, 3; title="small x", label="p/q") +plot!(quo, -3, 3; line=(3, :red), label="quo") + +p2 = plot(top/bottom, 5, 10; title="larger x", label="p/q") +plot!(quo, 5, 10; label="quo") + +plot(p1, p2) ``` - -But the graphs do match for large $x$: - - -```{julia} -plot(top/bottom, 5, 10) -plot!(quo, 5, 10) -``` - ---- +The asymptotic nature is similar to the function for larger $x$ values. +::: Finally, consider this rational expression in factored form: @@ -272,7 +286,7 @@ By looking at the powers we can see that the leading term of the numerator will #### Partial fractions -The `apart` function was useful to express a rational function in terms of a polynomial plus additional rational functions whose horizontal asymptotes are $0$. This function computes the partial fraction [decomposition](https://en.wikipedia.org/wiki/Partial_fraction_decomposition) of a rational function. Outside of the initial polynomial, this decomposition is a reexpression of a rational function into a sum of rational functions, where the denominators are *irreducible*, or unable to be further factored (non-trivially) and the numerators have lower degree than the denominator. Hence the horizontal asymptotes of $0$. +The `apart` function is used to express a rational function in terms of a polynomial plus additional rational functions whose horizontal asymptotes are $0$. This function computes the partial fraction [decomposition](https://en.wikipedia.org/wiki/Partial_fraction_decomposition) of a rational function. Outside of the initial polynomial, this decomposition is a reexpression of a rational function into a sum of rational functions, where the denominators are *irreducible*, or unable to be factored further over the set of rational numbers and the numerators have lower degree than the denominator. Hence the horizontal asymptotes of $0$. To see another example we have: @@ -291,28 +305,35 @@ The denominator, $q$, has factors $x-3$ and $x^2 - x - 1$, each irreducible. The ### Vertical asymptotes -As just discussed, the graph of $1/x$ will have a horizontal asymptote. However it will also show a spike at $0$: - +As just discussed, the graph of $1/x$ will have a horizontal asymptote. However its graph (@fig-plot-1-over-x-over-minus1-1-showing-vertical-asymptote) also shows a spike at $0$. +::: {#fig-plot-1-over-x-over-minus1-1-showing-vertical-asymptote} ```{julia} plot(1/x, -1, 1) ``` +Plot of $f(x) = 1/x$ over $[-1,1]$. The vertical asymptote at $0$ dominates the display, preventing other features to be identified. +::: + Again, this spike is an artifact of the plotting algorithm. The $y$ values for $x$-values just smaller than $0$ are large negative values and the $x$ values just larger than $0$ produce large, positive $y$ values. -The two points with $x$ components closest to $0$ are connected with a line, though that is misleading. Here we deliberately use far fewer points to plot $1/x$ to show how this happens: +The two points with $x$ components closest to $0$ are connected with a line, though that is misleading. @fig-plot-of-1-over-x-few-points we deliberately use far fewer points to plot $1/x$ to show how this happens: +::: {#fig-plot-of-1-over-x-few-points} ```{julia} #| hold: true f(x) = 1/x xs = range(-1, 1, length=12) ys = f.(xs) -plot(xs, ys) +plot(xs, ys; legend=false) scatter!(xs, ys) ``` +Plot of $f(x) = 1/x$ with few points to illustrate how "spikes" can be artifacts of the plotting algorithm +::: + The line $x = 0$ is a *vertical asymptote* for the graph of $1/x$. As $x$ values get close to $0$ from the right, the $y$ values go towards $\infty$ and as the $x$ values get close to $0$ on the left, the $y$ values go towards $-\infty$. @@ -328,12 +349,12 @@ $$ where both $r(c)$ and $s(c)$ are non zero. Knowing $m$ and $n$ (the multiplicities of the root $c$) allows the following to be said: +* If $m < n$ then $x=c$ will be a vertical asymptote. - * If $m < n$ then $x=c$ will be a vertical asymptote. - * If $m \geq n$ then $x=c$ will not be vertical asymptote. (The value $c$ will be known as a removable singularity). In this case, the graph of $p(x)/q(x)$ and the graph of $(x-c)^{m-n}r(x)/s(x)$ will differ, though very slightly, as the latter will include a value for $x=c$, whereas $x=c$ is not in the domain of $p(x)/q(x)$. +* If $m \geq n$ then $x=c$ will not be vertical asymptote. (The value $c$ will be known as a removable singularity). In this case, the graph of $p(x)/q(x)$ and the graph of $(x-c)^{m-n}r(x)/s(x)$ will differ, though very slightly, as the latter will include a value for $x=c$, whereas $x=c$ is not in the domain of $p(x)/q(x)$. -Finding the multiplicity may or may not be hard, but there is a very kludgy quick check that is often correct. With `Julia`, if you have a rational function that has `f(c)` evaluate to `Inf` or `-Inf` then there will be a vertical asymptote. If the expression evaluates to `NaN`, more analysis is needed. (The value of `0/0` is `NaN`, where as `1/0` is `Inf`.) +Finding the multiplicity may or may not be hard, but there is a very kludgy quick check that is often correct. With `Julia`, if you have a rational function that has `f(c)` evaluate to `Inf` or `-Inf` then there will be a vertical asymptote. If the expression evaluates to `NaN`, more analysis is needed. (The value of `0/0` is `NaN`, whereas `1/0` is `Inf`.) For example, the function $f(x) = ((x-1)^2 \cdot (x-2)) / ((x+3) \cdot(x-3))$ has vertical asymptotes at $-3$ and $3$, as its graph illustrated. Without the graph we could see this as well: @@ -353,45 +374,62 @@ As seen in several graphs, the basic plotting algorithm does a poor job with ver Consider again the function $f(x) = ((x-1)^2 \cdot (x-2)) / ((x+3) \cdot(x-3))$. Without much work, we can see that $x=3$ and $x=-3$ will be vertical asymptotes and there will be a slant asymptote with slope $1$. How to graph this? +::: {#fig-rational-function-avoiding-spikes} +```{julia} +#| echo: false +f(x) = (x-1)^2 * (x-2) / ((x+3)*(x-3) ) +p1 = plot(f, -5, 5; legend=false, + title="no effort", xlabel="(a)") +p2 = plot(f, -2.9, 2.9; legend=false, + title = "restrict domain", xlabel="(b)") +p3 = plot(f, -20, 20; ylims=(-20, 20), + legend=false, + title ="set ylims", xlabel="(c)") +p4 = plot(rangeclamp(f, 30), -20, 20; legend=false, + title="rangeclamp", xlabel="(d)") -We can avoid the vertical asymptotes in our viewing window. For example we could look at the area between the vertical asymptotes, by plotting over $(-2.9, 2.9)$, say: +plot(p1, p2, p3, p4; layout = 4) + +``` + +Four ways to plot the rational function $f(x)$: (a) has no effort, (b) restricts the plot domain, (c) uses `ylims` to restrict the viewing window, (d) calls `rangeclamp` to break lines when there are larger values. +::: + + +@fig-rational-function-avoiding-spikes shows four graphics, produced along the lines of: + +```{julia} +#| eval: false +f(x) = (x-1)^2 * (x-2) / ((x+3)*(x-3)) +plot(f, -5, 5) +``` + +This no effort approach shows the reason extra effort should be taken. This extra effort can come in terms of limiting the $x$ viewing window by backing off from the boundaries $(-3, 3)$: ```{julia} -f(x) = (x-1)^2 * (x-2) / ((x+3)*(x-3) ) +#| eval: false plot(f, -2.9, 2.9) ``` -This backs off by $\delta = 0.1$. As we have that $3 - 2.9$ is $\delta$ and $1/\delta$ is 10, the $y$ axis won't get too large, and indeed it doesn't. - - -This graph doesn't show well the two zeros at $x=1$ and $x=2$, for that a narrower viewing window is needed. By successively panning throughout the interesting part of the graph, we can get a view of the function. - - -We can also clip the `y` axis. The `plot` function can be passed an argument `ylims=(lo, hi)` to limit which values are plotted. With this, we can have: - +Extra effort can come by limiting the $y$ viewing window through `ylims`: ```{julia} -#| hold: true -plot(f, -5, 5, ylims=(-20, 20)) +#| eval: false +plot(f, -20, 20, ylims=(-20, 20)) ``` This isn't ideal, as the large values are still computed, just the viewing window is clipped. This leaves the vertical asymptotes still effecting the graph. -There is another way, we could ask `Julia` to not plot $y$ values that get too large. This is not a big request. If instead of the value of `f(x)` - when it is large - we use `NaN` instead, then the connect-the-dots algorithm will skip those values. - - -This was discussed in an earlier section where the `rangeclamp` function was introduced to replace large values of `f(x)` (in absolute values) with `NaN`. +Finally, we show the use of `rangeclamp` from the `CalculusWithJulia` package which replaces the plotted values with `NaN` when they are large (in absolute value). This breaks the lines. ```{julia} -plot(rangeclamp(f, 30), -25, 25) # rangeclamp is in the CalculusWithJulia package +#| eval: false +plot(rangeclamp(f, 30), -20, 20) ``` -We can see the general shape of $3$ curves broken up by the vertical asymptotes. The two on the side heading off towards the line $x-4$ and the one in the middle. We still can't see the precise location of the zeros, but that wouldn't be the case with most graphs that show asymptotic behaviors. However, we can clearly tell where to "zoom in" were those of interest. - - ### Sign charts @@ -410,7 +448,7 @@ The usual recipe for construction follows these steps: * Identify "test points" within each implied interval (these are $(-\infty, -1)$, $(-1,0)$, $(0,1)$, and $(1, \infty)$ in the example) and check for the sign of $f(x)$ at these test points. Write in `-`, `+`, `0`, or `*`, as appropriate. The value comes from the fact that "continuous" functions may only change sign when they cross $0$ or are undefined. -With the computer, where it is convenient to draw a graph, it might be better to emphasize the sign on the graph of the function, but at times numeric values are preferred. The `sign_chart` function from `CalculusWithJulia` does this analysis by numerically identifying points where the function is $0$ or $\infty$ and indicating the sign as $x$ crosses over these points. +With the computer, where it is convenient to draw a graph, it might be better to emphasize the sign on the graph of the function, but at times numeric values are preferred. The `sign_chart` function from `CalculusWithJulia` does this analysis by numerically identifying points where the function is $0$ or $\infty$ and indicating any sign change as $x$ crosses over these points. ```{julia} @@ -419,7 +457,7 @@ f(x) = x^3 - x sign_chart(f, -3/2, 3/2) ``` -This format is a bit different from above, but shows to the left of $-1$ a minussign; between $-1$ and $0$ a plus sign; between $0$ and $1$ a minus sign; and between $1$ and $3/2$ a plus sign. +This format is a bit different from above, but shows from $-3/2$ to the left of $-1$ a minus sign; between $-1$ and $0$ a plus sign; between $0$ and $1$ a minus sign; and between $1$ and $3/2$ a plus sign. ## Pade approximate @@ -441,20 +479,26 @@ $$ \tan(x) \approx \frac{x - 1/15 \cdot x^3}{1 - 2/5 \cdot x^2} $$ -We can look graphically at these approximations: +@fig-pade-approximations show these approximations through a graph. +::: {#fig-pade-approximations} ```{julia} +#| echo: false sin_p(x) = (x - (7/60)*x^3) / (1 + (1/20)*x^2) tan_p(x) = (x - (1/15)*x^3) / (1 - (2/5)*x^2) -plot(sin, -pi, pi) -plot!(sin_p, -pi, pi) +p1 = plot(sin, -pi, pi; label="sin") +plot!(p1, sin_p, -pi, pi; label="pade approx") + +Δ = 0.1 +p2 = plot(tan, -pi/2 + Δ, pi/2 - Δ; label="tan") +plot!(p2, tan_p, -pi/2 + Δ, pi/2 - Δ; label="pade approx") + +plot(p1, p2) ``` -```{julia} -plot(tan, -pi/2 + 0.2, pi/2 - 0.2) -plot!(tan_p, -pi/2 + 0.2, pi/2 - 0.2) -``` +Small-degree Pade approximations for $\sin(x)$ and $\tan(x)$. +::: ## The `Polynomials` package for rational functions @@ -530,26 +574,32 @@ One difference is the rational number `3//1` also represents other expressions, Rational functions also have a plot recipe defined for them that attempts to ensure the basic features are identifiable. As previously discussed, a plot of a rational function can require some effort to avoid the values associated to vertical asymptotes taking up too many of the available vertical pixels in a graph. -For the polynomial `pq` above, we have from observation that $1$ and $2$ will be zeros and $x=3$ a vertical asymptote. We also can identify a slant asymptote with slope $1$. These are hinted at in this graph: +For the polynomial `pq` above, we have from observation that $1$ and $2$ will be zeros and $x=3$ a vertical asymptote. We also can identify a slant asymptote with slope $1$. These are hinted at in @fig-basic-plot-recipe-rational-function +::: {#fig-basic-plot-recipe-rational-function} ```{julia} plot(pq) ``` +Plot of rational function `pq` using the basic plot recipe. The vertical asymptote is clear, but the slant asymptote requires a trained eye. +::: + To better see the zeros, a plot over a narrower interval, say $[0,2.5]$, would be encouraged; to better see the slant asymptote, a plot over a wider interval, say $[-10,10]$, would be encouraged. -For one more example of the default plot recipe, we redo the graphing of the rational expression we earlier plotted with `rangeclamp`: - +For one more example of the default plot recipe, we redo the graphing of the rational expression we earlier plotted with `rangeclamp` in @fig-plot-rational-function-with-asymptotes-at-minus3-3-from-pq-recipe. +::: {#fig-plot-rational-function-with-asymptotes-at-minus3-3-from-pq-recipe} ```{julia} #| hold: true p,q = fromroots([1,1,2]), fromroots([-3,3]) -plot(p//q) +plot(p//q; legend=false) ``` +A plot of a rational function with vertical asymptotes at $x= \pm 3$. The plot recipe avoids showing artifacts from the vertical asymptotes. +::: -##### Example: transformations of polynomials; real roots +##### Example: Transformations of polynomials; real roots We have seen some basic transformations of functions such as shifts and scales. For a polynomial expression we can implement these as follows, taking advantage of polynomial evaluation: @@ -633,7 +683,18 @@ numerator(lowest_terms( (x + 1)^d * pq((a*x + b)/(x + 1)))) Now, why is this of any interest? -Mobius transforms are used to map regions into other regions. In this special case, the transform $\phi(x) = (ax + b)/(x + 1)$ takes the interval $[0,\infty]$ and sends it to $[a,b]$ ($0$ goes to $(a\cdot 0 + b)/(0+1) = b$, whereas $\infty$ goes to $ax/x \rightarrow a$). Using this, if $p(u) = 0$, with $q(x) = (x+1)^d p(\phi(x))$, then setting $u = \phi(x)$ we have $q(x) = (\phi^{-1}(u)+1)^d p(\phi(\phi^{-1}(u))) = (\phi^{-1}(u)+1)^d \cdot p(u) = (\phi^{-1}(u)+1)^d \cdot 0 = 0$. That is, a zero of $p$ in $[a,b]$ will appear as a zero of $q$ in $[0,\infty)$ at $\phi^{-1}(u)$. +Mobius transforms are used to map regions into other regions. In this special case, the transform $\phi(x) = (ax + b)/(x + 1)$ takes the interval $[0,\infty]$ and sends it to $[a,b]$ ($0$ goes to $(a\cdot 0 + b)/(0+1) = b$, whereas $\infty$ goes to $ax/x \rightarrow a$). Using this, if $p(u) = 0$, with $q(x) = (x+1)^d p(\phi(x))$, then setting $u = \phi(x)$ we have + +$$ +\begin{align*} +q(x) &= (x+1)^d p(\phi(x))\\ +&= (\phi^{-1}(u)+1)^d p(\phi(\phi^{-1}(u))) \\ +& = (\phi^{-1}(u)+1)^d \cdot p(u) \\ +& = (\phi^{-1}(u)+1)^d \cdot 0 = 0.\\ +\end{align*} +$$ + +That is, a zero of $p$ in $[a,b]$ will appear as a zero of $q$ in $[0,\infty)$ at $x = \phi^{-1}(u)$. The Descartes rule of signs applied to $q$ then will give a bound on the number of possible roots of $p$ in the interval $[a,b]$. In the example we did, the Mobius transform for $a=4, b=6$ is $15 - x - 11x^2 - 3x^3$ with $1$ sign change, so there must be exactly $1$ real root of $p=(x-1)(x-3)(x-5)$ in the interval $[4,6]$, as we can observe from the factored form of $p$. @@ -653,12 +714,14 @@ This observation, along with a detailed analysis provided by [Kobel, Rouillier, The basic algorithm, as presented next, is fairly simple to understand, and hints at the bisection algorithm to come. It is due to Akritas and Collins. Suppose you know the only possible positive real roots are between $0$ and $M$ *and* no roots are repeated. Find the transformed polynomial over $[0,M]$: - * If there are no sign changes, then there are no roots of $p$ in $[0,M]$. - * If there is one sign change, then there is a single root of $p$ in $[0,M]$. The interval $[0,M]$ is said to isolate the root (and the actual root can then be found by other means) - * If there is more than one sign change, divide the interval in two ($[0,M/2]$ and $[M/2,M]$, say) and apply the same consideration to each. +* If there are no sign changes, then there are no roots of $p$ in $[0,M]$. + +* If there is one sign change, then there is a single root of $p$ in $[0,M]$. The interval $[0,M]$ is said to isolate the root (and the actual root can then be found by other means) + +* If there is more than one sign change, divide the interval in two ($[0,M/2]$ and $[M/2,M]$, say) and apply the same consideration to each. -Eventually, **mathematically** this will find isolating intervals for each positive real root. (The negative ones can be similarly isolated.) +Eventually, *mathematically* this will find isolating intervals for each positive real root. (The negative ones can be similarly isolated.) Applying these steps to $p$ with an initial interval, say $[0,9]$, we would have: @@ -686,7 +749,7 @@ The `ANewDsc` function takes a collection of coefficients representing a polynom ```{julia} -p₀ = fromroots([1,3,5]) +p₀ = fromroots([1, 3, 5]) st = ANewDsc(coeffs(p₀)) ``` @@ -701,7 +764,7 @@ More challenging problems can be readily handled by this package. The following ```{julia} -s = Polynomial([0,1]) # also just variable(Polynomial{Int}) +s = variable(:s) u = -1 + 254*s - 16129*s^2 + s^15 ``` @@ -719,6 +782,7 @@ and refine_roots(st) ``` +The `Hecke.jl` package has a `roots` function that handles the task of root isolation more performantly than `ANewDsc`. The SymPy package (`sympy.real_roots`) can accurately identify the three roots but it can take a **very** long time. The `Polynomials.roots` function from the `Polynomials` package identifies the cluster as complex valued. Though the implementation in `RealPolynomialRoots` doesn't handle such large polynomials, the authors of the algorithm have implementations that can quickly solve polynomials with degrees as high as $10,000$. @@ -737,8 +801,9 @@ The rational expression $(x^3 - 2x + 3) / (x^2 - x + 1)$ would have choices = [L"A horizontal asymptote $y=0$", L"A horizontal asymptote $y=1$", L"A slant asymptote with slope $m=1$"] -answ = 3 -radioq(choices, answ) +answer = 3 +explanation = L"The ratio of the two leading terms is $x$" +buttonq(choices, answer; explanation) ``` ###### Question @@ -753,8 +818,9 @@ The rational expression $(x^2 - x + 1)/ (x^3 - 2x + 3)$ would have choices = [L"A horizontal asymptote $y=0$", L"A horizontal asymptote $y=1$", L"A slant asymptote with slope $m=1$"] -answ = 1 -radioq(choices, answ) +answer = 1 +explanation = L"The ratio of the two leading terms is $1/x$" +buttonq(choices, answer; explanation) ``` ###### Question @@ -769,8 +835,9 @@ The rational expression $(x^2 - x + 1)/ (x^2 - 3x + 3)$ would have choices = [L"A horizontal asymptote $y=0$", L"A horizontal asymptote $y=1$", L"A slant asymptote with slope $m=1$"] -answ = 2 -radioq(choices, answ) +answer = 2 +explanation = L"The ratio of the two leading terms can be seen to be $1$" +buttonq(choices, answer; explanation) ``` ###### Question @@ -792,8 +859,9 @@ would have choices = [L"A horizontal asymptote $y=0$", L"A horizontal asymptote $y=1$", L"A slant asymptote with slope $m=1$"] -answ = 2 -radioq(choices, answ) +answer = 2 +explanation = L"the ratio of the leading terms is $1$" +buttonq(choices, answer; explanation) ``` ###### Question @@ -816,8 +884,9 @@ choices = [L"A vertical asymptote $x=1$", L"A slant asymptote with slope $m=1$", L"A vertical asymptote $x=5$" ] -answ = 3 -radioq(choices, answ) +answer = 3 +explanation = L"the ratio of the leading terms is $1$, not $x$ and there are vertical asymptotes at $x=4,5,6$" +buttonq(choices, answer; explanation) ``` ###### Question @@ -841,8 +910,9 @@ choices = [ "``y = (1/3)x``", "``y = (1/3)x - (1/3)``" ] -answ = 3 -radioq(choices, answ) +answer = 3 +explanation = "Try `div(x^3 - 3x^2+ 2x, 3x^2 - 6x +2)`" +buttonq(choices, answer) ``` ###### Question @@ -871,8 +941,8 @@ Is the following common conception true: "The graph of a function never crosses #| echo: false choices = ["No, the graph clearly crosses the drawn asymptote", "Yes, this is true"] -answ = 1 -radioq(choices, answ) +answer = 1 +buttonq(choices, answer) ``` (The wikipedia page indicates that the term "asymptote" was introduced by Apollonius of Perga in his work on conic sections, but in contrast to its modern meaning, he used it to mean any line that does not intersect the given curve. It can sometimes take a while to change perception.) @@ -883,18 +953,17 @@ radioq(choices, answ) Consider the two graphs of $f(x) = 1/x$ over $[10,20]$ and $[100, 200]$: - +::: {#fig-plot-1-over-x-10-to-20-100-to-200} ```{julia} #| hold: true #| echo: false -plot(x -> 1/x, 10, 20) +p1 = plot(x -> 1/x, 10, 20) +p2 = plot(x -> 1/x, 100, 200) +plot(p1, p2) ``` -```{julia} -#| hold: true -#| echo: false -plot(x -> 1/x, 100, 200) -``` +The graph of $f(x) = 1/x$ over different viewing windows +::: The two shapes are basically identical and do not look like straight lines. How does this reconcile with the fact that $f(x)=1/x$ has a horizontal asymptote $y=0$? @@ -906,8 +975,8 @@ choices = ["The horizontal asymptote is not a straight line.", L"The $y$-axis scale shows that indeed the $y$ values are getting close to $0$.", L"The graph is always decreasing, hence it will eventually reach $-\infty$." ] -answ = 2 -radioq(choices, answ) +answer = 2 +buttonq(choices, answer) ``` ###### Question @@ -955,8 +1024,8 @@ choices = ["between ``0`` and ``8`` hours", "between ``8`` and ``16`` hours", "between ``16`` and ``24`` hours", "after one day"] -answ = 1 -radioq(choices, answ) +answer = 1 +buttonq(choices, answer) ``` This graph has @@ -969,8 +1038,8 @@ choices = [L"a slant asymptote with slope $50$", L"a horizontal asymptote $y=20$", L"a horizontal asymptote $y=0$", L"a vertical asymptote with $x = 20^{1/3}$"] -answ = 3 -radioq(choices, answ) +answer = 3 +buttonq(choices, answer) ``` ###### Question @@ -996,6 +1065,6 @@ L"The $\sin(x)$ oscillates, but the rational function eventually follows $7/60 \ L"The $\sin(x)$ oscillates, but the rational function has a slant asymptote", L"The $\sin(x)$ oscillates, but the rational function has a non-zero horizontal asymptote", L"The $\sin(x)$ oscillates, but the rational function has a horizontal asymptote of $0$"] -answ = 2 -radioq(choices, answ) +answer = 2 +buttonq(choices, answer) ``` diff --git a/quarto/precalc/transformations.qmd b/quarto/precalc/transformations.qmd index 313f89e..c70df7e 100644 --- a/quarto/precalc/transformations.qmd +++ b/quarto/precalc/transformations.qmd @@ -1,22 +1,14 @@ -# Function manipulations +# Function transformations {{< include ../_common_code.qmd >}} -In this section we will use these add-on packages: +In this section we will use this add-on package: ```{julia} -using CalculusWithJulia using Plots -plotly() -``` - -```{julia} -#| echo: false -#| results: "hidden" -using DataFrames -nothing +plotly(); ``` --- @@ -38,25 +30,21 @@ $$ We have given meaning to a new function $f+g$ by defining what is does to $x$ with the rule on the right hand side. Similarly, we can define operations for subtraction, multiplication, addition, and powers. -These mathematical concepts aren't defined for functions in base `Julia`, though they could be if desired, by a commands such as: - +These mathematical concepts aren't defined for function instances in base `Julia`. For example, a common mistake in computing $\sin^2(1)$ would be to try that syntax exactly: ```{julia} -import Base: + -f::Function + g::Function = x -> f(x) + g(x) +#| error: true +sin^2(1) ``` -This adds a method to the generic `+` function for functions. The type annotations `::Function` ensure this applies only to functions. To see that it would work, we could do odd-looking things like: +The error message might seem cryptic but it comes from there not being a power operation (`^`) for `sin` defined. Similarly, out of the box there are not operations for `+`, `-`, `*`, or `/` defined for functions.^[There *could* be such operations defined. For example, if `+` is imported from `Base`, then this command `f::Function + g::Function = x -> f(x) + g(x)` would define such a operation. However, this definition in general is kind of limiting, as functions in mathematics and Julia can be much more varied than just the univariate functions we have defined addition for. Further, users shouldn't be modifying base methods on types they don't control, as that can lead to really unexpected and undesirable behaviours. This is called *type piracy*.] +To define such a function, we would specify what it does to each value of `x`, along the lines of: ```{julia} -ss = sin + sqrt -ss(4) +fplusg(x) = f(x) + g(x) ``` -Doing this works, as Julia treats functions as first class objects, lending itself to [higher](https://en.wikipedia.org/wiki/Higher-order_programming) order programming. However, this definition in general is kind of limiting, as functions in mathematics and Julia can be much more varied than just the univariate functions we have defined addition for. Further, users shouldn't be modifying base methods on types they don't control, as that can lead to really unexpected and undesirable behaviours. This is called *type piracy*. We won't pursue this possibility further. Rather we will define new function by what they do to their values, such as `h(x) = f(x) + g(x)`. - - ### Composition of functions @@ -82,26 +70,29 @@ $$ Though they may be *typographically* similar don't be fooled, the following graph shows that the two functions aren't even close except for $x$ near $0$ (for example, one composition is always non-negative, whereas the other is not): - +::: {#fig-plot-sin-xsquared-sinsquared-x} ```{julia} #| hold: true f(x) = x^2 g(x) = sin(x) -fg = f ∘ g # typed as f \circ[tab] g -gf = g ∘ f # typed as g \circ[tab] f +fg(x) = f(g(x)) +gf(x) = g(f(x)) plot(fg, -2, 2, label="f∘g") plot!(gf, label="g∘f") ``` -:::{.callout-note} -## Note -Unlike how the basic arithmetic operations are treated, `Julia` defines the infix Unicode operator `\circ[tab]` to represent composition of functions, mirroring mathematical notation. This infix operations takes in two functions and returns a composed function. It can be useful and will mirror standard mathematical usage up to issues with precedence rules. - +Plot of $f(x) = \sin^2(x)$ and $g(x) = \sin(x^2)$, both written as compositions, illustrating the order of composition is important ::: Starting with two functions and composing them requires nothing more than a solid grasp of knowing the rules of function evaluation. If $f(x)$ is defined by some rule involving $x$, then $f(g(x))$ just replaces each $x$ in the rule with a $g(x)$. +::: {.callout-note} +## Infix operator +Composition of two functions does have an infix operator, `∘`, entered as `\circ[tab]`. This mirrors the mathematical usage of this syntax, though the order of operations are such that calling the composed function on a value requires an extra set of parentheses: `(f∘g)(x)`, as the expresssion `f∘g(x)` evaluates `g(x)` before the composition. +::: + + So if $f(x) = x^2 + 2x - 1$ and $g(x) = e^x - x$ then $f \circ g$ would be (before any simplification) @@ -123,10 +114,11 @@ $$ Here we look at a few compositions: +* The function $h(x) = \sqrt{1 - x^2}$ can be seen as $f\circ g$ with $f(x) = \sqrt{x}$ and $g(x) = 1-x^2$. - * The function $h(x) = \sqrt{1 - x^2}$ can be seen as $f\circ g$ with $f(x) = \sqrt{x}$ and $g(x) = 1-x^2$. - * The function $h(x) = \sin(x/3 + x^2)$ can be viewed as $f\circ g$ with $f(x) = \sin(x)$ and $g(x) = x/3 + x^2$. - * The function $h(x) = e^{-1/2 \cdot x^2}$ can be viewed as $f\circ g$ with $f(x) = e^{-x}$ and $g(x) = (1/2) \cdot x^2$. +* The function $h(x) = \sin(x/3 + x^2)$ can be viewed as $f\circ g$ with $f(x) = \sin(x)$ and $g(x) = x/3 + x^2$. + +* The function $h(x) = e^{-1/2 \cdot x^2}$ can be viewed as $f\circ g$ with $f(x) = e^{-x}$ and $g(x) = (1/2) \cdot x^2$. Decomposing a function into a composition of functions is not unique, other compositions could have been given above. For example, the last function is also $f(x) = e^{-x/2}$ composed with $g(x) = x^2$. @@ -144,19 +136,19 @@ The real value of composition is to break down more complicated things into a se It is very useful to mentally categorize functions within families. The difference between $f(x) = \cos(x)$ and $g(x) = 12\cos(2(x - \pi/4))$ is not that much---both are cosine functions, one is just a simple enough transformation of the other. As such, we expect bounded, oscillatory behaviour with the details of how large and how fast the oscillations are to depend on the specifics of the function. Similarly, both these functions $f(x) = 2^x$ and $g(x)=e^x$ behave like exponential growth, the difference being only in the rate of growth. There are families of functions that are qualitatively similar, but quantitatively different, linked together by a few basic transformations. -There is a set of operations of functions, which does not really change the type of function. Rather, it basically moves and stretches how the functions are graphed. We discuss these four main transformations of $f$: +There is a set of operations of functions, which does not really change the type of function. Rather, it basically moves and stretches how the functions are graphed. We discuss the four main transformations of $f$ from @tbl-four-main-transformations. +::: {#tbl-four-main-transformations .striped .hover tbl-colwidths="[25,75]"} -```{julia} -#| echo: false +| Transformation | Description | +|:--------------------|:---------------| +| *vertical shifts* | The function $h(x) = k + f(x)$ will have the same graph as $f$ shifted **up** $k$ units.| +| *horizontal shifts* | The function $h(x) = f(x - k)$ will have the same graph as $f$ shifted **over** right by $k$ units. | +| *stretching* | The function $h(x) = kf(x)$ will have the same graph as $f$ **stretch**ed by a factor $k$ of in the $y$ direction. | +| *scaling* |The function $h(x) = f(kx)$ will have the same graph as $f$ **scale**d horizontally by a factor of $1$ over $k$ | -nms = ["*vertical shifts*","*horizontal shifts*","*stretching*","*scaling*"] -acts = [L"The function $h(x) = k + f(x)$ will have the same graph as $f$ shifted up by $k$ units.", -L"The function $h(x) = f(x - k)$ will have the same graph as $f$ shifted right by $k$ units.", -L"The function $h(x) = kf(x)$ will have the same graph as $f$ stretched by a factor of $k$ in the $y$ direction.", -L"The function $h(x) = f(kx)$ will have the same graph as $f$ compressed horizontally by a factor of $1$ over $k$."] -table(DataFrame(Transformation=nms, Description=acts)) -``` +: Four main transformations of a function. +::: The functions $h$ are derived from $f$ in a predictable way. To implement these transformations within `Julia`, we define operators (functions which transform one function into another). As these return functions, the function bodies are anonymous functions. The basic definitions are similar, save for the `x -> ...` part that signals the creation of an anonymous function to return: @@ -175,44 +167,58 @@ To illustrate, let's define a hat-shaped function as follows: f(x) = max(0, 1 - abs(x)) ``` -A plot over the interval $[-2,2]$ is shown here: +A plot over the interval $[-2,2]$ is shown in @fig-plot-hat-function: +::: {#fig-plot-hat-function} ```{julia} -plot(f, -2,2) +plot(f, -2, 2; aspect_ratio=:equal) ``` +Plot of $f(x) = \max(0, 1 - \lvert x\rvert)$ over $[-2,2]$ +::: The same graph of $f$ and its image shifted up by $2$ units would be given by: - +::: {#fig-plot-hat-function-also-up-2} ```{julia} -plot(f, -2, 2, label="f") -plot!(up(f, 2), label="up") +plot(f, -2, 2; aspect_ratio=:equal, label="f") +plot!(up(f, 2); label="up") ``` -A graph of $f$ and its shift over by $2$ units would be given by: +Plot of $f(x) = \max(0, 1 - \lvert x\rvert)$ and its transformation shifted up by $2$ units +::: +A graph of $f$ and its shift over by $2$ units would be generated by: +::: {#fig-plot-hat-function-shifted-over-2} ```{julia} -plot(f, -2, 4, label="f") -plot!(over(f, 2), label="over") +plot(f, -2, 4; aspect_ratio=:equal, label="f") +plot!(over(f, 2); label="over") +``` +Plot of $f(x) = \max(0, 1 - \lvert x\rvert)$ and its transformation shifted over by $2$ units +::: + +A graph of $f$ and it being stretched by $2$ units would be generated by: + +::: {#fig-plot-hat-function-stretched-by-2} +```{julia} +plot(f, -2, 2; aspect_ratio=:equal, label="f") +plot!(stretch(f, 2); label="stretch") ``` -A graph of $f$ and it being stretched by $2$ units would be given by: +Plot of $f(x) = \max(0, 1 - \lvert x\rvert)$ and its transformation stretched in #y# direction by $2$ units +::: +Finally, a graph of $f$ and it being scaled by $2$ would be generated by: + +::: {{{{{{{#fig-plot-hat-function-scaled-by-2} ```{julia} -plot(f, -2, 2, label="f") -plot!(stretch(f, 2), label="stretch") -``` - -Finally, a graph of $f$ and it being scaled by $2$ would be given by: - - -```{julia} -plot(f, -2, 2, label="f") -plot!(scale(f, 2), label="scale") +plot(f, -2, 2; aspect_ratio=:equal, label="f") +plot!(scale(f, 2); label="scale") ``` +Plot of $f(x) = \max(0, 1 - \lvert x\rvert)$ and its transformation scaled in $x$ by $2$ units +::: Scaling by $2$ shrinks the non-zero domain, scaling by $1/2$ would stretch it. If this is not intuitive, the definition `x-> f(x/c)` could have been used, which would have opposite behaviour for scaling. @@ -220,37 +226,69 @@ Scaling by $2$ shrinks the non-zero domain, scaling by $1/2$ would stretch it. I --- -More exciting is what happens if we compose these operations. - - -A shift right by $2$ and up by $1$ is achieved through - +More exciting is what happens if we compose these operations. Before doing so, let's note how the syntax of composition would be: ```{julia} -plot(f, -2, 4, label="f") -plot!(up(over(f,2), 1), label="over and up") +#| eval: false +up(over(f, 2), 1) ``` -Shifting and scaling can be confusing. Here we graph `scale(over(f,2),1/3)`: - +The `1` is the value for `up` and can get lost without careful parsing. It might be better to see something like this instead: ```{julia} -plot(f, -1,9, label="f") -plot!(scale(over(f,2), 1/3), label="over and scale") +#| eval: false +f |> over(2) |> up(1) ``` -This graph is over by $6$ with a width of $3$ on each side of the center. Mathematically, we have $h(x) = f((1/3)\cdot x - 2)$ +Meaning, take `f`, shift it over, `2` and then up `1`. To make this happen, we need to pass in a function and return a function from both `over` and `up`. We define another set of functions using multiple dispatch to sort out which is which: + +```{julia} +up(k) = f -> up(f, k) +over(k) = f -> over(f, k) +stretch(k) = Base.Fix2(stretch, k) +scale(k) = Base.Fix2(scale, k) +``` + +We did this two ways, the last two using `Fix2` to fix the second argument, leaving a function of just the first argument. + + +Now, a shift right by $2$ and up by $1$ is achieved through + +::: {#fig-plot-hat-over-up-composed-call} +```{julia} +plot(f, -2, 4; aspect_ratio=:equal, label="f") +plot!(f |> over(2) |> up(1); label="over and up") +``` + +Plot of $f(x) = \max(0, 1 - \lvert x\rvert)$ and it transform after moving over by $2$ and up by $1$ using chaining notation +::: + +Shifting and scaling can be confusing. Here we graph the action of `over` by $2$ and then `scale` by $1/3$: + +::: {#fig-plot-hat-over-scale-composed} +```{julia} +plot(f, -1, 9; aspect_ratio=:equal, label="f") +plot!(f |> over(2) |> scale(1/3); label="over and scale") +``` + +Plot of $f(x)$ and it transform being over $2$ and then scaled by $1/3$ +::: + +@fig-plot-hat-over-scale-composed show $f(x)$ after being moved over by $2$ and *then$ scaled by $1/3$. It's center moves to $6$ and instead of stretching from $6-1$ to $6+1$ it stretches from $6-3$ to $6+3$. Mathematically, we have $h(x) = f((1/3)\cdot x - 2)$ Compare this to the same operations in opposite order: - +::: {#fig-plot-hat-scale-over-composed} ```{julia} -plot(f, -1, 5, label="f") -plot!(over(scale(f, 1/3), 2), label="scale and over") +plot(f, -1, 5; aspect_ratio=:equal, label="f") +plot!(f |> scale(1/3) |> over(2); label="scale and over") ``` -This graph first scales the symmetric graph, stretching from $-3$ to $3$, then shifts over right by $2$. The resulting function is $f((1/3)\cdot (x-2))$. +Plot of $f(x)$ and it transform being first scaled by $1/3$ and then shifted over $2$ +::: + +@fig-plot-hat-scale-over-composed shows the transform of $f(x)$ first scaled, stretching from $-3$ to $3$, then shifted over right by $2$. The resulting function is $f((1/3)\cdot (x-2))$. As a last example, following up on the last example, a common transformation mathematically is @@ -260,19 +298,36 @@ $$ h(x) = \frac{1}{a}f(\frac{x - b}{a}). $$ -We can view this as a composition of "scale" by $1/a$, then "over" by $b$, and finally "stretch" by $1/a$: - +We can view this as a composition of "scale" by $1/a$, then "over" by $b$, and finally "stretch" by $1/a$, as seen in @fig-plot-hat-function-wavelet-transform: +::: {#fig-plot-hat-function-wavelet-transform} ```{julia} #| hold: true a = 2; b = 5 -h(x) = stretch(over(scale(f, 1/a), b), 1/a)(x) -plot(f, -1, 8, label="f"; xticks=-1:8) -plot!(h, label="h") +plot(f, -1, 8; aspect_ratio=:equal, label="f", xticks=-1:8) +plot!(f |> scale(1/a) |> over(b) |> stretch(1/a); label="h") ``` -(This transformation keeps the same amount of area in the triangles, can you tell from the graph?) +Plot showing transform $\frac{1}{a}f(\frac{x - b}{a})$ +::: +This transformation keeps the same amount of area in the triangles, can you tell from the graph? + +##### Example: a growth model in fisheries + + +The von Bertalanffy growth [equation](https://en.wikipedia.org/wiki/Von_Bertalanffy_function) is $L(t) =L_\infty \cdot (1 - e^{-k\cdot(t-t_0)})$. This family of functions can be viewed as a transformation of the exponential function $f(t)=e^t$. Part is a scaling and shifting (the $e^{-k \cdot (t - t_0)}$) along with some shifting and stretching. The various parameters have physical importance which can be measured: $L_\infty$ is a carrying capacity for the species or organism, and $k$ is a rate of growth. These parameters may be estimated from data by finding the "closest" curve to a given data set. For this set of parameters^[From [Campbell and Phillips](https://doi.org/10.1093/icesjms/34.2.295) shows steady but decelerating growth of a whelk population.] + +::: {#fig-von-Bertalanffy-plot} +```{julia} +L, k = 53.1120, 0.0335 # from https://doi.org/10.1093/icesjms/34.2.295 +t0 = 1968 +u = exp |> scale(-k) |> over(t0) |> stretch(-1) |> up(1) |> stretch(L) +plot(u, t0, t0+200; legend=false) +``` + +Plot of von-Bertalanffy model applied to a whelk population +::: ##### Example @@ -308,12 +363,15 @@ c = 80 Putting this together, we have our graph is "scaled" by $d$, "over" by $c$, "stretched" by $b$ and "up" by $a$. Here we plot it over slightly more than one year so that we can see that the shortest day of light is in late December ($x \approx -10$ or $x \approx 355$). - +::: {#fig-plot-newyork-daylight-sinusoidal-model} ```{julia} -newyork(t) = up(stretch(over(scale(sin, d), c), b), a)(t) +newyork = sin |> scale(d) |> over(c) |> stretch(b) |> up(a) plot(newyork, -20, 385) ``` +Plot of a sinusoidal model for length of daylight in New York City +::: + To test, if we match up with the model powering [dateandtime.info](http://dateandtime.info/citysunrisesunset.php?id=5128581) we note that it predicts "$12$h $10$m $38$s" on September $23$th, $2015$. This is day $266$ (`Date(2015, 9, 23) - Date(2015, 1, 1) + Day(1)`). Our model prediction has a difference of @@ -325,43 +383,28 @@ delta = (newyork(266) - datetime) * 60 This is off by a fair amount---almost $8$ minutes. Clearly a trigonometric model, based on the assumption of circular motion of the earth around the sun, is not accurate enough for precise work, but it does help one understand how summer days are longer than winter days and how the length of a day changes fastest at the spring and fall equinoxes. -##### Example: the pipeline operator +##### Example: Representing data visually -In the last example, we described our sequence as scale, over, stretch, and up, but code this in reverse order, as the composition $f \circ g$ is done from right to left. A more convenient notation would be to have syntax that allows the composition of $g$ then $f$ to be written $x \rightarrow g \rightarrow f$. `Julia` provides the [pipeline](https://docs.julialang.org/en/v1/manual/functions/#Function-composition-and-piping) operator for chaining function calls together. +The `Plots.jl` package also uses transformations to display different shapes on a graphic. This next example shows how scale can be used to display a third piece of information to augment the two pieces shown through location. +Suppose we have a data set like @tbl-palmer-penguins-data:^[Which comes from the "Palmer Penguins" data set] -For example, if $g(x) = \sqrt{x}$ and $f(x) =\sin(x)$ we could call $f(g(x))$ through: - - -```{julia} -#| hold: true -g(x) = sqrt(x) -f(x) = sin(x) -pi/2 |> g |> f -``` - -The output of the preceding expression is passed as the input to the next. This notation is especially convenient when the enclosing function is not the main focus. (Some programming languages have more developed [fluent interfaces](https://en.wikipedia.org/wiki/Fluent_interface) for chaining function calls. Julia has more powerful chaining macros provided in packages, such as `DataPipes.jl` or `Chain.jl`.) - -##### Example: a growth model in fisheries - - -The von Bertalanffy growth [equation](https://en.wikipedia.org/wiki/Von_Bertalanffy_function) is $L(t) =L_\infty \cdot (1 - e^{k\cdot(t-t_0)})$. This family of functions can be viewed as a transformation of the exponential function $f(t)=e^t$. Part is a scaling and shifting (the $e^{k \cdot (t - t_0)}$) along with some shifting and stretching. The various parameters have physical importance which can be measured: $L_\infty$ is a carrying capacity for the species or organism, and $k$ is a rate of growth. These parameters may be estimated from data by finding the "closest" curve to a given data set. - -##### Example: Representing data visually. - -Suppose we have a data set like the following:^[Which comes from the "Palmer Penguins" data set] - +::: {#tbl-palmer-penguins-data .striped .hover} |flipper length | bill length | body mass | gender | species | -|---------------|-------------|-----------|--------|:--------| +|:--------------|:------------|-----------|--------|:--------| | 38.8 | 18.3 | 3701 | male | Adelie | | 48.8 | 18.4 | 3733 | male | Chinstrap | | 47.5 | 15.0 | 5076 | male | Gentoo | -We might want to plot on an $x$-$y$ axis flipper length versus bill length but also indicate body size with a large size marker for bigger sizes. +: Data set on sampled penguin attributes +::: -We could do so by transforming a marker: scaling by size, then shifting it to an `x-y` position; then plotting. Something like this: +We might want to plot on an $x$-$y$ axis flipper length versus bill length but also indicate body size with a larger-sized marker for bigger sizes. +We could do so by transforming a marker: scaling by size, then shifting it to an `x-y` position; then plotting. @fig-plot-palmer-penguin-shapes illustrates. + +::: {#fig-plot-palmer-penguin-shapes} ```{julia} flipper = [38.8, 48.8, 47.5] bill = [18.3, 18.4, 15.0] @@ -369,7 +412,7 @@ bodymass = [3701, 4733, 5076] shape = Shape(:star5) p = plot(; legend=false) -for (x,y,sz) in zip(flipper, bill, bodymass) +for (x, y, sz) in zip(flipper, bill, bodymass) sz = (sz - 2000) ÷ 1000 new_shape = Plots.translate(Plots.scale(shape, sz, sz), x, y); @@ -379,36 +422,18 @@ end p ``` +Plot of penguin data using size of marker to represent flipper length +::: + While some of the commands in this example are unfamiliar and won't be explained further, the use of `translate` and `scale` for shapes is very similar to how transformations for functions are being described (Though this `translate` function combines `up` and `over`; and this `scale` function allows different values depending on direction.) In the above, the function names are qualified, as they are not exported by the `Plots.jl` package. More variables from the data set could be encoded through colors, different shapes etc. allowing very data-rich graphics. ### Operators - -The functions `up`, `over`, etc. are operators that take a function as an argument and return a function. The use of operators fits in with the template `action(f, args...)`. The action is what we are doing, such as `plot`, `over`, and others to come. The function `f` here is just an object that we are performing the action on. For example, a plot takes a function and renders a graph using the additional arguments to select the domain to view, etc. +In computer science a *higher order function* is one that either takes a function as input, returns a function as its output, or both. We prefer the less standard, but more mathematical sounding *operator* to describe higher-order functions like `up` and `over`. The use of operators fits in with the template `action(f, args...)`. The action is what we are doing, such as `plot`, `over`, and others to come. The function `f` here is just an object that we are performing the action on. For example, a plot takes a function and renders a graph using the additional arguments to select the domain to view, etc. -Creating operators that return functions involves the use of anonymous functions, using these operators is relatively straightforward. Two basic patterns are - - - * Storing the returned function, then calling it: - - -```{julia} -#| eval: false -l(x) = action1(f, args...)(x) -l(10) -``` - - * Composing two operators: - - -```{julia} -#| eval: false -action2( action1(f, args..), other_args...) -``` - -Composition like the above is convenient, but can get confusing if more than one composition is involved. +Creating operators that return functions typically involves the use of anonymous functions, using these operators is relatively straightforward. In this next example, we create two new operators that will be prototypes for two foundational operator in calculus. ##### Example: two operators @@ -420,22 +445,31 @@ Composition like the above is convenient, but can get confusing if more than one ```{julia} D(f::Function) = k -> f(k) - f(k-1) ``` +As written, `D` takes a function, `f`, and then returns a function that finds the difference of `f` at `k` and `k-1`. -To see that it works, we take a typical function +To see how `D` works, we take a typical function: ```{julia} f(k) = 1 + k^2 ``` -and check: - +Then we have: ```{julia} D(f)(3), f(3) - f(3-1) ``` -That the two are the same value is no coincidence. (Again, pause for a second to make sure you understand why `D(f)(3)` makes sense. If this is unclear, you could name the function `D(f)` and then call this with a value of `3`.) +That the two are the same value is by design. + +The calling syntax `D(f)(3)` is a bit awkward and is read as two steps: the first finds `D(f)` the second calls this function at `3`. The first could have been stored and then called, as with: + +```{julia} +df = D(f) +df(3) +``` + + Now we want a function to cumulatively *sum* the values $S(f)(k) = f(1) + f(2) + \cdots + f(k-1) + f(k)$, as a function of $k$. Adding up $k$ terms is easy to do with a generator and the function `sum`: @@ -457,7 +491,8 @@ So one function adds, the other subtracts. Addition and subtraction are somehow ```{julia} k = 10 # some arbitrary value k >= 1 -D(S(f))(k), f(k) +λ = D(S(f)) +λ(k), f(k) ``` Any positive integer value of `k` will give the same answer (up to overflow). This says the difference of the accumulation process is just the last value to accumulate. @@ -467,10 +502,11 @@ Adding after subtracting also leaves the function alone, save for a vestige of $ ```{julia} -S(D(f))(15), f(15) - f(0) +γ = S(D(f)) +γ(15), f(15) - f(0) ``` -That is the accumulation of differences is just the difference of the end values. +That is, the accumulation of differences is just the difference of the end values. These two operations are discrete versions of the two main operations of calculus---the derivative and the integral. This relationship will be known as the "fundamental theorem of calculus." @@ -489,8 +525,8 @@ If $f(x) = 1/x$ and $g(x) = x-2$, what is $g(f(x))$? #| hold: true #| echo: false choices=["``1/(x-2)``", "``1/x - 2``", "``x - 2``", "``-2``"] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question @@ -504,8 +540,8 @@ If $f(x) = e^{-x}$ and $g(x) = x^2$ and $h(x) = x-3$, what is $f \circ g \circ h #| echo: false choices=["``e^{-x^2 - 3}``", "``(e^x -3)^2``", "``e^{-(x-3)^2}``", "``e^x+x^2+x-3``"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` ###### Question @@ -520,8 +556,8 @@ If $h(x) = (f \circ g)(x) = \sin^2(x)$ which is a possibility for $f$ and $g$: choices = [raw"``f(x)=x^2; \quad g(x) = \sin^2(x)``", raw"``f(x)=x^2; \quad g(x) = \sin(x)``", raw"``f(x)=\sin(x); \quad g(x) = x^2``"] -answ = 2 -radioq(choices, answ) +answer = 2 +radioq(choices, answer) ``` ###### Question @@ -538,7 +574,7 @@ choices = [ raw"``h(x) = 6 + \sin(x + 4)``", raw"``h(x) = 6 + \sin(x-4)``", raw"``h(x) = 6\sin(x-4)``"] -answ = 3 +answer = 3 radioq(choices, 3) ``` @@ -554,10 +590,46 @@ Let $h(x) = 4x^2$ and $f(x) = x^2$. Which is **not** true: choices = [L"The graph of $h(x)$ is the graph of $f(x)$ stretched by a factor of ``4``", L"The graph of $h(x)$ is the graph of $f(x)$ scaled by a factor of ``2``", L"The graph of $h(x)$ is the graph of $f(x)$ shifted up by ``4`` units"] -answ = 3 -radioq(choices, answ) +answer = 3 +radioq(choices, answer) ``` +###### Question + +Consider the function + +$$ +g(x) = f(\frac{x - a}{b}) +$$ + +This is + +```{julia} +#| echo: false +choices = ["`over` by `a` and then `scale` by `1/b`", + "`scale` by `1/b` and then `over` by `a`"] +answer = 2 +buttonq(choices, answer) +``` + +Consider the function + +$$ +g(x) = f(\frac{x}{b} - a) +$$ + +This is + +```{julia} +#| echo: false +choices = ["`over` by `a` and then `scale` by `1/b`", + "`scale` by `1/b` and then `over` by `a`"] +answer = 1 +buttonq(choices, answer) +``` + + + ###### Question @@ -579,15 +651,17 @@ radioq(choices, answ) This is the graph of a transformed sine curve. - +::: {#fig-transformed-sine-graph-question} ```{julia} #| hold: true #| echo: false f(x) = 2*sin(pi*x) p = plot(f, -2,2) ``` +Plot of a transformation of the sine function +::: -What is the period of the graph? +What is the period of the graph in @fig-transformed-sine-graph-question? ```{julia} @@ -597,7 +671,7 @@ val = 2 numericq(val) ``` -What is the amplitude of the graph? +What is the amplitude of the graph in @fig-transformed-sine-graph-question? ```{julia} @@ -607,7 +681,7 @@ val = 2 numericq(val) ``` -What is the form of the function graphed? +What is the form of the function graphed in @fig-transformed-sine-graph-question? ```{julia} @@ -619,8 +693,8 @@ raw"``\sin(2x)``", raw"``\sin(\pi x)``", raw"``2 \sin(\pi x)``" ] -answ = 4 -radioq(choices, answ) +answer = 4 +radioq(choices, answer) ``` ###### Question @@ -647,8 +721,8 @@ choices = [ q"D(S(f))(n) = f(n)", q"S(D(f))(n) = f(n) - f(0)" ] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +radioq(choices, answer, keep_order=true) ``` ###### Question @@ -671,6 +745,6 @@ choices = [ q"D(S(f))(n) = f(n)", q"S(D(f))(n) = f(n) - f(0)" ] -answ = 1 -radioq(choices, answ, keep_order=true) +answer = 1 +radioq(choices, answer, keep_order=true) ``` diff --git a/quarto/precalc/trig_functions.qmd b/quarto/precalc/trig_functions.qmd index c6f3f08..16de81e 100644 --- a/quarto/precalc/trig_functions.qmd +++ b/quarto/precalc/trig_functions.qmd @@ -30,6 +30,7 @@ We measure angles in radians, where $360$ degrees is $2\pi$ radians. By proporti For a right triangle with angles $\theta$, $\pi/2 - \theta$, and $\pi/2$ ($0 < \theta < \pi/2$) we call the side opposite $\theta$ the "opposite" side, the shorter adjacent side the "adjacent" side and the longer adjacent side the hypotenuse. +::: {#fig-right-triangle-soh-cah-toa} ```{julia} #| hide: true @@ -67,10 +68,12 @@ plotly() nothing ``` -With these, the basic definitions for the primary trigonometric functions are +Labeling of a right triangle lengths in relation to a designated angle, $\theta$. +::: -::: {.callout-note icon=false} -## Trigonometric definitions +With the labelings in @fig-right-triangle-soh-cah-toa, the basic definitions for the primary trigonometric functions are + +::: {.definition title="Trigonometric definitions from a right triangle"} $$ \begin{align*} \sin(\theta) &= \frac{\text{opposite}}{\text{hypotenuse}} &\quad(\text{the sine function})\\ @@ -78,20 +81,16 @@ $$ \tan(\theta) &= \frac{\text{opposite}}{\text{adjacent}} &\quad(\text{the tangent function}) \end{align*} $$ -::: -:::{.callout-note} -## Note Many students remember these through [SOH-CAH-TOA](http://mathworld.wolfram.com/SOHCAHTOA.html). - ::: Some algebra shows that $\tan(\theta) = \sin(\theta)/\cos(\theta)$. There are also $3$ reciprocal functions, the cosecant, secant and cotangent. -These definitions in terms of sides only apply for $0 \leq \theta \leq \pi/2$. More generally, if we relate any angle taken in the counter clockwise direction for the $x$-axis with a point $(x,y)$ on the *unit* circle, then we can extend these definitions - the point $(x,y)$ is also $(\cos(\theta), \sin(\theta))$. - +These definitions in terms of sides only apply for $0 \leq \theta \leq \pi/2$. More generally, if we relate any angle taken in the counter clockwise direction for the $x$-axis with a point $(x,y)$ on the *unit* circle, then we can extend these definitions - the point $(x,y)$ is also $(\cos(\theta), \sin(\theta))$, cf. @fig-trig-values-along-unit-circle. +::: {#fig-trig-values-along-unit-circle} ```{julia} #| hold: true #| echo: false @@ -136,38 +135,46 @@ end imgfile = tempname() * ".gif" gif(anim, imgfile, fps = 1) -caption = "An angle in radian measure corresponds to a point on the unit circle, whose coordinates define the sine and cosine of the angle. That is ``(x,y) = (\\cos(\\theta), \\sin(\\theta))``." +caption = "" plotly() ImageFile(imgfile, caption) ``` +An angle in radian measure corresponds to a point on the unit circle, whose coordinates define the sine and cosine of the angle. That is $(x,y) = (\cos(\theta), \sin(\theta))$. +::: + ### The trigonometric functions in Julia Julia has the $6$ basic trigonometric functions defined through the functions `sin`, `cos`, `tan`, `csc`, `sec`, and `cot`. -Two right triangles - the one with equal, $\pi/4$, angles; and the one with angles $\pi/6$ and $\pi/3$ can have the ratio of their sides computed from basic geometry. In particular, this leads to the following values, which are usually committed to memory: +Two right triangles - the one with equal, $\pi/4$, angles; and the one with angles $\pi/6$ and $\pi/3$ can have the ratio of their sides computed from basic geometry. In particular, this leads to @tbl-basic-sin-cosine-values, values which are usually committed to memory: -$$ -\begin{align*} -\sin(0) &= 0, \quad \sin(\pi/6) = \frac{1}{2}, \quad \sin(\pi/4) = \frac{\sqrt{2}}{2}, \quad\sin(\pi/3) = \frac{\sqrt{3}}{2},\text{ and } \sin(\pi/2) = 1\\ -\cos(0) &= 1, \quad \cos(\pi/6) = \frac{\sqrt{3}}{2}, \quad \cos(\pi/4) = \frac{\sqrt{2}}{2}, \quad\cos(\pi/3) = \frac{1}{2},\text{ and } \cos(\pi/2) = 0. -\end{align*} -$$ +::: {#tbl-basic-sin-cosine-values .striped .hover} + +| $\theta$ | $\sin(\theta)$ | $\cos(\theta)$ | +|:---------:|:--------------:|:--------------:| +| $0$ | $0$ | $1$ | +| $\pi/6$ | $1/2$ | $\sqrt{3}/2$ | +| $\pi/4$ | $\sqrt{2}/2$ | $\sqrt{2}/2$ | +| $\pi/3$ | $\sqrt{3}/2$ | $1/2$ | +| $\pi/2$ | $1$ | $0$ | + +: Table of sine and cosine values that can be derived from different triangles +::: -Using the circle definition allows these basic values to inform us of values throughout the unit circle. +Using the circle definition allows these basic values to inform us of values throughout the unit circle: -These all follow from the definition involving the unit circle: +* If the angle $\theta$ corresponds to a point $(x,y)$ on the unit circle, then the angle $-\theta$ corresponds to $(x, -y)$. So $\sin(\theta) = - \sin(-\theta)$ (an odd function), but $\cos(\theta) = \cos(-\theta)$ (an even function). +* If the angle $\theta$ corresponds to a point $(x,y)$ on the unit circle, then rotating by $\pi$ moves the points to $(-x, -y)$. So $x = \cos(\theta) = - \cos(\theta + \pi)$, and $y = \sin(\theta) = -\sin(\theta + \pi)$. - * If the angle $\theta$ corresponds to a point $(x,y)$ on the unit circle, then the angle $-\theta$ corresponds to $(x, -y)$. So $\sin(\theta) = - \sin(-\theta)$ (an odd function), but $\cos(\theta) = \cos(-\theta)$ (an even function). - * If the angle $\theta$ corresponds to a point $(x,y)$ on the unit circle, then rotating by $\pi$ moves the points to $(-x, -y)$. So $\cos(\theta) = x = - \cos(\theta + \pi)$, and $\sin(\theta) = y = -\sin(\theta + \pi)$. - * If the angle $\theta$ corresponds to a point $(x,y)$ on the unit circle, then rotating by $\pi/2$ moves the points to $(-y, x)$. So $\cos(\theta) = x = \sin(\theta + \pi/2)$. +* If the angle $\theta$ corresponds to a point $(x,y)$ on the unit circle, then rotating by $\pi/2$ moves the points to $(-y, x)$. So $x = \cos(\theta) = \sin(\theta + \pi/2)$. The fact that $x^2 + y^2 = 1$ for the unit circle leads to the "Pythagorean identity" for trigonometric functions: @@ -232,7 +239,7 @@ opposite = adjacent * tan(theta) Having some means to compute an angle and then a tangent of that angle handy is not a given, so the linked to article provides a few other methods taking advantage of similar triangles. -You can also measure distance with your [thumb](http://www.vendian.org/mncharity/dir3/bodyruler_angle/) or fist. How? The fist takes up about $10$ degrees of view when held straight out. So, pacing off backwards until the fist completely occludes the tree will give the distance of the adjacent side of a right triangle. If that distance is $30$ paces what is the height of the tree? Well, we need some facts. Suppose your pace is $3$ feet. Then the adjacent length is $90$ feet. The multiplier is the tangent of $10$ degrees, or: +You can also measure distance with your [thumb](http://www.vendian.org/mncharity/dir3/bodyruler_angle/) or fist. How? The fist takes up about $10$ degrees of view when held straight out. So, pacing off backwards until the fist completely occludes the tree can give the distance of the adjacent side of a right triangle. If that distance is $30$ paces what is the height of the tree? Well, we need some facts. Suppose your pace is $3$ feet. Then the adjacent length is $90$ feet. The multiplier is the tangent of $10$ degrees, or: ```{julia} @@ -259,40 +266,52 @@ This could be reversed. If you know the height of something a distance away that ### Basic properties -The sine function is defined for all real $\theta$ and has a range of $[-1,1]$. Clearly as $\theta$ winds around the $x$-axis, the position of the $y$ coordinate begins to repeat itself. We say the sine function is *periodic* with period $2\pi$. A graph will illustrate: +The sine function is defined for all real $\theta$ and has a range of $[-1,1]$. Clearly as $\theta$ winds around the $x$-axis, the position of the $y$ coordinate begins to repeat itself. We say the sine function is *periodic* with period $2\pi$. @fig-sin-over-0-4-pi illustrates. The graph shows two periods. The wavy aspect of the graph is why this function is used to model periodic motions, such as the amount of sunlight in a day, or the alternating current powering a computer. +::: {#fig-sin-over-0-4-pi} ```{julia} +#| echo: false plot(sin, 0, 4pi) ``` -The graph shows two periods. The wavy aspect of the graph is why this function is used to model periodic motions, such as the amount of sunlight in a day, or the alternating current powering a computer. +Graph of $f(x) = \sin(x)$ over $[0, 4\pi]$ +::: -From this graph---or considering when the $y$ coordinate is $0$---we see that the sine function has zeros at any integer multiple of $\pi$, or $k\pi$, $k$ in $\dots,-2,-1, 0, 1, 2, \dots$. + +From the graph of $\sin(x)$---or considering when the $y$ coordinate is $0$ on the unit circle---we see that the sine function has zeros at any integer multiple of $\pi$, or $k\pi$, $k$ in $\dots,-2,-1, 0, 1, 2, \dots$. -The cosine function is similar, in that it has the same domain and range, but is "out of phase" with the sine curve. A graph of both shows the two are related: - +The cosine function is similar, in that it has the same domain and range, but is "out of phase" with the sine curve. @fig-sin-cosine-graph illustrates. +::: {#fig-sin-cosine-graph} ```{julia} +#| echo: false plot(sin, 0, 4pi, label="sin") plot!(cos, 0, 4pi, label="cos") ``` +: Graph of $\sin(x)$ and $\cos(x)$ over $[0, 4\pi]$. The cosine graph lags the sine graph by $\pi/2$, or $\cos(x) = \sin(x + \pi/2)$. +::: + The cosine function is just a shift of the sine function (or vice versa). We see that the zeros of the cosine function happen at points of the form $\pi/2 + k\pi$, $k$ in $\dots,-2,-1, 0, 1, 2, \dots.$ -The tangent function does not have all $\theta$ for its domain, rather those points where division by $0$ occurs are excluded. These occur when the cosine is $0$, or, again, at $\pi/2 + k\pi$, $k$ in $\dots,-2,-1, 0, 1, 2, \dots.$ The range of the tangent function will be all real $y$. +The tangent function does not have all $\theta$ for its domain, rather those points where division by $0$ occurs are excluded; when the cosine is $0$, or, again, at $\pi/2 + k\pi$, $k$ in $\dots,-2,-1, 0, 1, 2, \dots.$ The range of the tangent function will be all real $y$. -The tangent function is also periodic, but not with period $2\pi$, but rather just $\pi$. A graph will show this. Here we avoid the vertical asymptotes using `rangeclamp`: - +The tangent function is also periodic, but not with period $2\pi$, but rather just $\pi$. @fig-graph-of-tangent-minus10-10 illustrates. +::: {#fig-graph-of-tangent-minus10-10} ```{julia} +#| echo: false plot(rangeclamp(tan), -10, 10, label="tan") ``` +Graph of $f(x) = \tan(x)$ over $[-10, 10]$ showing the periodic nature of the function. The vertical asymptotes were avoided by plotting `rangeclamp(tan)`. +::: + ##### Example sums of sines @@ -303,34 +322,43 @@ $$ g(x) = a + b \sin((2\pi n)x) $$ -That is a graph of $g$ will be the sine curve shifted up by $a$ units, scaled vertically by $b$ units and has a period of $1/n$. We see a simple plot here where we can verify the transformation: +That is a graph of $g$ will be the sine curve shifted up by $a$ units, scaled vertically by $b$ units and has a period of $1/n$. +@fig-plot-of-2sin-2pi-n-x verifies the transformation: + +::: {#fig-plot-of-2sin-2pi-n-x} ```{julia} g(x; b=1, n=1) = b*sin(2pi*n*x) g1(x) = 1 + g(x, b=2, n=3) -plot(g1, 0, 1) +plot(g1, 0, 1; xticks = (0:1/3:1, ["0", "1/3", "2/3", "1"])) ``` +Plot of $g(x) = a + b \sin((2\pi n)x)$ with $n=1/3$ so the period of the sinusoical function is $1/3 +::: -We can consider the sum of such functions, for example +@fig-sum-two-gs shows the sum of two such functions. Though still periodic, we can see with this simple example that sums of different sine functions can have somewhat complicated graphs. +::: {#fig-sum-two-gs} + ```{julia} g2(x) = 1 + g(x, b=2, n=3) + g(x, b=4, n=5) plot(g2, 0, 1) ``` -Though still periodic, we can see with this simple example that sums of different sine functions can have somewhat complicated graphs. +Graph of sum of two sine functions, one with period $1/3$ one with period $1/5$. The resulting function has period $1$. +::: + Sine functions can be viewed as the `x` position of a point traveling around a circle so `g(x, b=2, n=3)` is the `x` position of point traveling around a circle of radius $2$ that completes a circuit in $1/3$ units of time. -The superposition of the two sine functions that `g2` represents could be viewed as the position of a circle moving around a point that is moving around another circle. The following graphic, with $b_1=1/3, n_1=3, b_2=1/4$, and $n_2=4$, shows an example that produces the related cosine sum (moving right along the $x$ axis), the sine sum (moving down along the $y$ axis, *and* the trace of the position of the point generating these two plots. +The superposition of the two sine functions that `g2` represents could be viewed as the position of a circle moving around a point that is moving around another circle. @fig-superposition-of-sines-cosines, with $b_1=1/3, n_1=3, b_2=1/4$, and $n_2=4$, shows an example that produces the related cosine sum (moving right along the $x$ axis), the sine sum (moving down along the $y$ axis, *and* the trace of the position of the point generating these two plots. +::: {#fig-superposition-of-sines-cosines} ```{julia} -#| hold: true #| echo: false #| cache: true gr() @@ -390,11 +418,14 @@ end imgfile = tempname() * ".gif" gif(anim, imgfile, fps = 5) -caption = "Superposition of sines and cosines represented by an epicycle" +caption = "" plotly() ImageFile(imgfile, caption) ``` +Superposition of sines and cosines represented by an epicycle. +::: + As can be seen, even a somewhat simple combination can produce complicated graphs (a fact known to [Ptolemy](https://en.wikipedia.org/wiki/Deferent_and_epicycle)) . How complicated can such a graph get? This won't be answered here, but for fun enjoy this video produced by the same technique using more moving parts from the [`Javis.jl`](https://github.com/Wikunia/Javis.jl/blob/master/examples/fourier.jl) package: @@ -431,6 +462,7 @@ More generally, suppose we have two angles $\alpha$ and $\beta$, can we represen Suppose both $\alpha$ and $\beta$ are positive with $\alpha + \beta \leq \pi/2$. Then using right triangle geometry we can associate the sine and cosine of $\alpha + \beta$ with distances in this figure: +::: {#fig-sin-cos-alpha-plus-beta-and-beta} ```{julia} #| echo: false gr() @@ -534,25 +566,25 @@ plot!(Shape([F,B]), fill=(:black, 0.35)) annotate!(map(s ->getindex(txtpoints,s), collect(keys(txtpoints)))) -p1 +plot(p1, p2) ``` -Another right triangle with hypotenuse of length $1$ can be made by isolating the angle $\beta$, as below: +The left figure labels the sides of a right triangle with angle $\alpha + \beta$, the right figure labels the sides of a right triangle with angle $\beta$. +::: -```{julia} -#| echo: false -p2 -``` +In @fig-cos-alpha-beta-sin-alpha-beta we make two more right triangles one with hypotenuse $\cos(\beta)$ and one with hypotenuse $\sin(\beta)$; each having an angle $\alpha$, the latter using some geometry, for which we can apply right-triangle trigonometry to find the length of their respective sides. - -We can make two more right triangles one with hypotenuse $\cos(\beta)$ and one with hypotenuse $\sin(\beta)$; each having an angle $\alpha$, the latter using some geometry, for which we can apply right-triangle trigonometry to find the length of their sides. +::: {#fig-cos-alpha-beta-sin-alpha-beta} ```{julia} #| echo: false plot(p3, p4) ``` -From the left figure and the initial triangle, by comparing the lengths along the $x$ direction, we can see the decomposition: +Two triangles with angle $\alpha$ and hypotenuses $\cos(\beta)$ and $\sin(\beta)$. +::: + +From the left side of #fig-cos-alpha-beta-sin-alpha-beta and the initial triangle, by comparing the lengths along the $x$ direction, we can see the decomposition: $$ \cos(\alpha)\cos(\beta) = \cos(\alpha + \beta) + \sin(\alpha)\sin(\beta) @@ -564,10 +596,9 @@ $$ \sin(\alpha+\beta) = \sin(\alpha)\cos(\beta) + \cos(\alpha)\sin(\beta) $$ -These lead to: +All combined, these lead to: -::: {.callout-note icon=false} -## The *sum* formulas for sine and cosine +::: {.relationship title="The sum formulas for sine and cosine"} $$ \begin{align*} @@ -575,12 +606,12 @@ $$ \cos(\alpha + \beta) &= \cos(\alpha)\cos(\beta) - \sin(\alpha)\sin(\beta) \end{align*} $$ + ::: Taking $\alpha = \beta$ we immediately get -::: {.callout-note icon=false} -## The "double-angle" formulas +::: {.relationship title="The double-angle formulas"} $$ \begin{align*} \sin(2\alpha) &= 2\sin(\alpha)\cos(\alpha)\\ @@ -589,17 +620,16 @@ $$ $$ ::: -The latter looks like the Pythagorean identify, but has a minus sign. In fact, the Pythagorean identify is often used to rewrite this, for example $\cos(2\alpha) = 2\cos^2(\alpha) - 1$ or $1 - 2\sin^2(\alpha)$. +The latter looks like the Pythagorean identify, but has a minus sign. In fact, the Pythagorean identify is often used to rewrite this formula, for example $\cos(2\alpha) = 2\cos^2(\alpha) - 1$ or $1 - 2\sin^2(\alpha)$. -Applying the above with $\alpha = \beta/2$, we get that $\cos(\beta) = 2\cos^2(\beta/2) -1$. Similarly, using the Pythagorean identity a formula for sine can be done; when rearranged these yield the "half-angle" formulas: +Applying the above with $\alpha = \beta/2$, we get that $\cos(\beta) = 2\cos^2(\beta/2) -1$. Similarly, using the Pythagorean identity a formula for sine can be identified; when rearranged these yield the "half-angle" formulas: -::: {.callout-note icon=false} -## The "half-angle" formula +::: {.relationship title="The half-angle formulas"} $$ \begin{align*} -\sin^2(\frac{\beta}{2}) &= \frac{1 - \cos(\beta)}{2}\\ -\cos^2(\frac{\beta}{2}) &= \frac{1 + \cos(\beta)}{2} +\sin^2\left(\frac{\beta}{2}\right) &= \frac{1 - \cos(\beta)}{2}\\ +\cos^2\left(\frac{\beta}{2}\right) &= \frac{1 + \cos(\beta)}{2} \end{align*} $$ ::: @@ -633,6 +663,13 @@ $$ That is the angle for a multiple of $n+1$ can be expressed in terms of the angle with a multiple of $n$ and $n-1$. This can be used recursively to find expressions for $\cos(n\theta)$ in terms of polynomials in $\cos(\theta)$. +For example, + +```{julia} +@syms θ +sympy.expand_trig(cos(5 * θ)) +``` + ## Inverse trigonometric functions @@ -640,29 +677,31 @@ That is the angle for a multiple of $n+1$ can be expressed in terms of the angle The trigonometric functions are all periodic. In particular they are not monotonic over their entire domain. This means there is no *inverse* function applicable. However, by restricting the domain to where the functions are monotonic, inverse functions can be defined: - * For $\sin(x)$, the restricted domain of $[-\pi/2, \pi/2]$ allows for the arcsine function to be defined. In `Julia` this is implemented with `asin`. - * For $\cos(x)$, the restricted domain of $[0,\pi]$ allows for the arccosine function to be defined. In `Julia` this is implemented with `acos`. - * For $\tan(x)$, the restricted domain of $(-\pi/2, \pi/2)$ allows for the arctangent function to be defined. In `Julia` this is implemented with `atan`. + +* For $\sin(x)$, the restricted domain of $[-\pi/2, \pi/2]$ allows for the arcsine function to be defined. In `Julia` this is implemented with `asin`. + +* For $\cos(x)$, the restricted domain of $[0,\pi]$ allows for the arccosine function to be defined. In `Julia` this is implemented with `acos`. + +* For $\tan(x)$, the restricted domain of $(-\pi/2, \pi/2)$ allows for the arctangent function to be defined. In `Julia` this is implemented with `atan`. + + For example, the arcsine function is defined for $-1 \leq x \leq 1$ and has a range of $-\pi/2$ to $\pi/2$: - +::: {#fig-arcsin-arctan} ```{julia} -plot(asin, -1, 1) +#| echo: false +p1 = plot(asin, -1, 1; legend=false, title="arcsin") +p2 = plot(atan, -15, 15; legend=false, title="arctan") +plot(p1, p2) ``` -The arctangent has domain of all real $x$. It has shape given by: +The function $f(x) = \arcsin(x)$ has domain $[-1,1]$, whereas the function $f(x) = \arctan(x)$ has domain $(-\infty, \infty)$. +::: -```{julia} -plot(atan, -10, 10) -``` - -The horizontal asymptotes are $y=\pi/2$ and $y=-\pi/2$. - - -### Implications of a restricted domain +##### Example: Implications of a restricted domain Notice that $\sin(\arcsin(x)) = x$ for any $x$ in $[-1,1]$, but, of course, not for all $x$, as the output of the sine function can't be arbitrarily large. @@ -805,7 +844,7 @@ $$ Both $\theta_0$ and $\theta_1$ are measured with respect to the coordinate system that looks like the $x-y$ plane. The red coordinate system is used to identify the angle of incidence for the second bending. Some right-triangle geometry relates the new angle $\theta'_1$ with $\theta_1$ through $\theta'_1 = \alpha - \theta_1$. With this new angle of incidence, the angle of refraction, $\theta'_2$, satisfies: $$ -n1 \sin(\theta'_1) = n2 \sin(\theta'_2) + n_1 \sin(\theta'_1) = n_2 \sin(\theta'_2) $$ Or @@ -868,9 +907,9 @@ $$ d = \pi + 2i - 4 \arcsin(\frac{1}{n} \sin(i)). $$ -Graphing this for incident angles between $0$ and $\pi/2$ we have: - +@fig-plot-of-deflection-for-different-incident-angles shows the deflection for incident angles between $0$ and $\pi/2$ +::: {#fig-plot-of-deflection-for-different-incident-angles} ```{julia} #| hold: true n = 4/3 @@ -878,6 +917,9 @@ d(i) = pi + 2i - 4 * asin(sin(i)/n) plot(d, 0, pi/2) ``` +Plot of deflection for different incident angles +::: + Descartes was interested in the minimum value of this graph, as it relates to where the light concentrates. This is roughly at $1$ radian or about $57$ degrees: @@ -915,9 +957,9 @@ A few things become clear from the above two representations: * Using the initial definition, we see that the zeros of $T_n(x)$ all occur within $[-1,1]$ and happen when $n\arccos(x) = k\pi + \pi/2$, or $x=\cos((2k+1)/n \cdot \pi/2)$ for $k=0, 1, \dots, n-1$. -Other properties of this polynomial family are not at all obvious. One is that amongst all polynomials of degree $n$ with roots in $[-1,1]$, $T_n(x)$ will be the smallest in magnitude (after we divide by the leading coefficient to make all polynomials considered to be monic). We check this for one case. Take $n=4$, then we have: $T_4(x) = 8x^4 - 8x^2 + 1$. Compare this with $q(x) = (x+3/5)(x+1/5)(x-1/5)(x-3/5)$ (evenly spaced zeros): - +Other properties of this polynomial family are not at all obvious. One is that amongst all polynomials of degree $n$ with roots in $[-1,1]$, $T_n(x)$ will be the smallest in magnitude (after we divide by the leading coefficient to make all polynomials considered to be monic). We check this for one case. Take $n=4$, then we have: $T_4(x) = 8x^4 - 8x^2 + 1$. We compare this polynomial with $q(x) = (x+3/5)(x+1/5)(x-1/5)(x-3/5)$ (evenly spaced zeros) in @fig-plot-T4-q-showing-chebyshev-minimal. +::: {#fig-plot-T4-q-showing-chebyshev-minimal} ```{julia} T4(x) = (8x^4 - 8x^2 + 1) / 8 q(x) = (x+3/5)*(x+1/5)*(x-1/5)*(x-3/5) @@ -925,14 +967,17 @@ plot(abs ∘ T4, -1,1, label="|T₄|") plot!(abs ∘ q, -1,1, label="|q|") ``` -We will return to this family of polynomials in the section on Orthogonal Polynomials. +The monic Chebyshev polynomial is has the smallest maximum value of $[-1,1]$ of all monic polynomials of the same degree +::: + +We will return to this family of polynomials in the section on orthogonal polynomials. ## Hyperbolic trigonometric functions Related to the trigonometric functions are the hyperbolic trigonometric functions. Instead of associating a point $(x,y)$ on the unit circle with an angle $\theta,$ we associate a point $(x,y)$ on the unit *hyperbola* ($x^2 - y^2 = 1$). We define the hyperbolic sine ($\sinh$) and hyperbolic cosine ($\cosh$) through $(\cosh(\theta), \sinh(\theta)) = (x,y)$. - +::: {#fig-hyperbolic-trig-functions-from-unit-hyperbola} ```{julia} #| echo: false let @@ -940,7 +985,7 @@ let # y^2 = x^2 - 1 top(x) = sqrt(x^2 - 1) - p = plot(; legend=false, aspect_ratio=:equal) + p = plot(; legend=false, framestyle=:origin, aspect_ratio=:equal) x₀ = 2 xs = range(1, x₀, length=100) @@ -976,6 +1021,8 @@ let p end ``` +Figure showing the definitions of $\cosh(x)$ and $\sinh(x)$ using the unit hyperbola $x^2 - y^2 = 1$ +::: These values are more commonly expressed using the exponential function as: @@ -1004,12 +1051,12 @@ What is bigger $\sin(1.23456)$ or $\cos(6.54321)$? ```{julia} -#| hold: true #| echo: false a = sin(1.23456) > cos(6.54321) choices = [raw"``\sin(1.23456)``", raw"``\cos(6.54321)``"] -answ = a ? 1 : 2 -radioq(choices, answ, keep_order=true) +answer = a ? 1 : 2 +explanation = "Compare with `sin(1.23456) > cos(6.54321)`" +buttonq(choices, answer; explanation) ``` ###### Question @@ -1019,13 +1066,13 @@ Let $x=\pi/4$. What is bigger $\cos(x)$ or $x$? ```{julia} -#| hold: true #| echo: false x = pi/4 a = cos(x) > x choices = [raw"``\cos(x)``", "``x``"] -answ = a ? 1 : 2 -radioq(choices, answ, keep_order=true) +answer = a ? 1 : 2 +explanation = "Compare with `cos(pi/4) > pi/4`" +radioq(choices, answer; explanation) ``` ###### Question @@ -1041,8 +1088,8 @@ choices = [ raw"``\cos(x) = \sin(x - \pi/2)``", raw"``\cos(x) = \sin(x + \pi/2)``", raw"``\cos(x) = \pi/2 \cdot \sin(x)``"] -answ = 2 -radioq(choices, answ) +answer = 2 +buttonq(choices, answer) ``` ###### Question @@ -1058,8 +1105,9 @@ choices = [ L"The values $k\pi$ for $k$ in $\dots, -2, -1, 0, 1, 2, \dots$", L"The values $\pi/2 + k\pi$ for $k$ in $\dots, -2, -1, 0, 1, 2, \dots$", L"The values $2k\pi$ for $k$ in $\dots, -2, -1, 0, 1, 2, \dots$"] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +explanation = "The secant is the reciprocal of the cosine function" +buttonq(choices, answer; explanation) ``` ###### Question @@ -1117,37 +1165,40 @@ numericq(val) The sine function is an *odd* function. - * The hyperbolic sine is: +* The hyperbolic sine is: ```{julia} #| hold: true #| echo: false choices = ["odd", "even", "neither"] -answ = 1 -radioq(choices, answ, keep_order=true) +answer = 1 +explanation = "subsitute `-x` into the exponential formula to see" +buttonq(choices, answer; explanation) ``` - * The hyperbolic cosine is: +* The hyperbolic cosine is: ```{julia} #| hold: true #| echo: false choices = ["odd", "even", "neither"] -answ = 2 -radioq(choices, answ, keep_order=true) +answer = 2 +explanation = L"The value of $\cosh(-x)$ is the $y$ position of the point $(x,y)$ refelected through the $y$ axis, so is unchanged." +buttonq(choices, answer; explanation) ``` - * The hyperbolic tangent is: +* The hyperbolic tangent is: ```{julia} #| hold: true #| echo: false choices = ["odd", "even", "neither"] -answ = 1 -radioq(choices, answ, keep_order=true) +answer = 1 +explanation = "A ratio of an odd function and an even function is *odd*" +buttonq(choices, answer; explanation) ``` ###### Question diff --git a/quarto/references.bib b/quarto/references.bib index a324c0b..f395fc5 100644 --- a/quarto/references.bib +++ b/quarto/references.bib @@ -8,6 +8,17 @@ note = {Published in 1991 by Wellesley-Cambridge Press, the book is a useful resource for educators and self-learners alike.} } +@misc{Apostol, + author = {Tom M. Apostol}, + title = {Calculus}, + published = {1967}, + publisher = {John Wiley and Sons}, + url = {https://simeioseismathimatikwn.wordpress.com/wp-content/uploads/2013/03/apostol-calculusi.pdf}, + note = {} +} + + + @misc{Knill, author = {Oliver Knill}, diff --git a/quarto/styles.css b/quarto/styles.css new file mode 100644 index 0000000..e69de29 From f2de1cdefc240ee1da8a68e7f20ef8b30e7cf8d4 Mon Sep 17 00:00:00 2001 From: jverzani Date: Tue, 11 Aug 2026 17:45:57 -0400 Subject: [PATCH 5/7] fix typos --- quarto/ODEs/differential_equations.qmd | 2 +- quarto/ODEs/euler.qmd | 2 +- quarto/alternatives/giac.qmd | 10 +- quarto/alternatives/makie_plotting.qmd | 6 +- quarto/alternatives/symbolics.qmd | 2 +- quarto/basics/numbers_types-II.html | 2804 ----------------- quarto/basics/numbers_types-II.qmd | 2 +- quarto/basics/numbers_types.qmd | 2 +- quarto/basics/vectors.qmd | 8 +- quarto/derivatives/curve_sketching.qmd | 2 +- .../derivatives/first_second_derivatives.qmd | 2 +- .../differentiable_vector_calculus/test.html | 641 ---- quarto/differentiable_vector_calculus/test.jl | 827 ----- .../differentiable_vector_calculus/test.qmd | 98 - quarto/integrals/area_between_curves.qmd | 2 +- quarto/integrals/center_of_mass.qmd | 4 +- quarto/integrals/substitution.qmd | 2 +- quarto/integrals/surface_area.qmd | 2 +- quarto/precalc/julia_overview.qmd | 2 +- quarto/precalc/transformations.qmd | 2 +- quarto/precalc/trig_functions.qmd | 4 +- 21 files changed, 28 insertions(+), 4398 deletions(-) delete mode 100644 quarto/basics/numbers_types-II.html delete mode 100644 quarto/differentiable_vector_calculus/test.html delete mode 100644 quarto/differentiable_vector_calculus/test.jl delete mode 100644 quarto/differentiable_vector_calculus/test.qmd diff --git a/quarto/ODEs/differential_equations.qmd b/quarto/ODEs/differential_equations.qmd index 5b5ff45..d98970f 100644 --- a/quarto/ODEs/differential_equations.qmd +++ b/quarto/ODEs/differential_equations.qmd @@ -260,7 +260,7 @@ This very clearly shows the sharp dependence on the value of $b$; below some lev The function `recovered` is of two variables returning a single value. In subsequent sections we will see a few $3$-dimensional plots that are common for such functions, here we skip ahead and show how to visualize multiple function plots at once using "`z`" values in a graph. -::: {#fig-recoverd-over-various-k-values} +::: {#fig-recovered-over-various-k-values} ```{julia} #| hold: true k, ks = 0.1, 0.2:0.1:0.9 # first `k` and then the rest diff --git a/quarto/ODEs/euler.qmd b/quarto/ODEs/euler.qmd index 682ec9e..d94a9fe 100644 --- a/quarto/ODEs/euler.qmd +++ b/quarto/ODEs/euler.qmd @@ -184,7 +184,7 @@ for i in 1:n end ``` -So how did we do? @fig-euler-yp-yx-n-5 shows the graph of the exact answer and the stiched-together answer. +So how did we do? @fig-euler-yp-yx-n-5 shows the graph of the exact answer and the stitched-together answer. ::: {#fig-euler-yp-yx-n-5} diff --git a/quarto/alternatives/giac.qmd b/quarto/alternatives/giac.qmd index b2b3dcd..42b8cae 100644 --- a/quarto/alternatives/giac.qmd +++ b/quarto/alternatives/giac.qmd @@ -9,7 +9,7 @@ format: # Using the Giac symbolic math engine within Julia -This page describes the use of symbolic math for some topics of Calculus within `Julia` utilizing the [Giac](http://www-fourier.ujf-grenoble.fr/~parisse/giac.html) library. Giac is accessed through `Giac.jl`. Giac is a computer algebra system (CAS) implemented in C++; `Giac.jl` is an interface. See this [pdf documentation](https://www-fourier.univ-grenoble-alpes.fr/~parisse/giac/cascmd_en.pdf) for a comprehensive set of topics or the `Julia` documention for the different commands discussed. +This page describes the use of symbolic math for some topics of Calculus within `Julia` utilizing the [Giac](http://www-fourier.ujf-grenoble.fr/~parisse/giac.html) library. Giac is accessed through `Giac.jl`. Giac is a computer algebra system (CAS) implemented in C++; `Giac.jl` is an interface. See this [pdf documentation](https://www-fourier.univ-grenoble-alpes.fr/~parisse/giac/cascmd_en.pdf) for a comprehensive set of topics or the `Julia` documentation for the different commands discussed. There are other possible choices for symbolic math within the Julia ecosystem: @@ -1680,7 +1680,7 @@ This new object can be manipulated as any other symbolic expression. ## Parametric description of functions -Vectors can be used to describe curves parameterically (where the $x$ and $y$ motions are modeled by a third variable, typically $t$ for time). We describe this in a bit more detail later when discussing univariate, vector-valued functions. +Vectors can be used to describe curves parametrically (where the $x$ and $y$ motions are modeled by a third variable, typically $t$ for time). We describe this in a bit more detail later when discussing univariate, vector-valued functions. Consider a person on an advanced ferris wheel with coordinates $x(t)$ and $y(t)$ describing position. The wheel of radius $r$ is centered $R$ units above and circles at radius $\omega$, the car has radius $r_0$ and rotates $6$ times faster. @@ -2176,7 +2176,7 @@ integrate.(spiral, t, 0, pi) ##### Line integrals -A line integral might be generically written over a curve $C$ or with a paramterization, $r(t)$, of $C$, yielding +A line integral might be generically written over a curve $C$ or with a parameterization, $r(t)$, of $C$, yielding $$ I = \int_C f(\vec{x}) ds = \int_a^b f(r(t)) dt @@ -2379,8 +2379,8 @@ For example, to integrate $F(x,y) = x^2 \cdot y^3$ over the triangular region fo ```{julia} @giac_var x y F = x^2 * y^3 -Iy = integrate(F, y, 0, 1-x) # integrating int_{x=0}^1 int_{y=0}^{1-x} F(x,y) dy dx -integrate(Iy, x, 0, 1) +I_y = integrate(F, y, 0, 1-x) # integrating int_{x=0}^1 int_{y=0}^{1-x} F(x,y) dy dx +integrate(I_y, x, 0, 1) ``` (Integrating is made easy, but *not* the task of identifying valid endpoints to describe the region integrated over.) diff --git a/quarto/alternatives/makie_plotting.qmd b/quarto/alternatives/makie_plotting.qmd index 50669d9..14435dd 100644 --- a/quarto/alternatives/makie_plotting.qmd +++ b/quarto/alternatives/makie_plotting.qmd @@ -510,7 +510,7 @@ arrows3d!(Point3.(us), Point3.(dus)) current_figure() ``` -Plot of tangent lines in both two and three dimenstions +Plot of tangent lines in both two and three dimensions ::: @@ -851,9 +851,9 @@ The manual construction of a figure and an axis object will be further discussed ### Three dimensional contour plots -The `contour` function can also plot $3$-dimensional contour plots. Concentric spheres, contours of $x^2 + y^2 + z^2 = c$ for $c > 0$ are presented in @fig-makie-countour-three-d. +The `contour` function can also plot $3$-dimensional contour plots. Concentric spheres, contours of $x^2 + y^2 + z^2 = c$ for $c > 0$ are presented in @fig-makie-contour-three-d. -::: {#fig-makie-countour-three-d} +::: {#fig-makie-contour-three-d} ```{julia} f(x,y,z) = x^2 + y^2 + z^2 xs = ys = zs = range(-3, 3, length=100) diff --git a/quarto/alternatives/symbolics.qmd b/quarto/alternatives/symbolics.qmd index 26b45bd..0393228 100644 --- a/quarto/alternatives/symbolics.qmd +++ b/quarto/alternatives/symbolics.qmd @@ -762,7 +762,7 @@ Symbolics.jacobian(eqs, [x,y]) ## Integration -The `SymbolicIntegration` package provides two means to integration *univariate* functions using either the Risch alogorithm or a rules-based approach. +The `SymbolicIntegration` package provides two means to integration *univariate* functions using either the Risch algorithm or a rules-based approach. ```{julia} using SymbolicIntegration, Symbolics diff --git a/quarto/basics/numbers_types-II.html b/quarto/basics/numbers_types-II.html deleted file mode 100644 index 86c4b3c..0000000 --- a/quarto/basics/numbers_types-II.html +++ /dev/null @@ -1,2804 +0,0 @@ - - - - - - - - - -numbers_types-ii - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- -
-
-

Number systems

-
- - - -
- - - - -
- - - -
- - -

In mathematics there are many different number systems in common use. For example by the end of pre-calculus, all of the following have been introduced:

-
    -

  • The integers, \(\{\dots, -3, -2, -1, 0, 1, 2, 3, \dots\}\);

    • -

  • The rational numbers, \(\{p/q: p, q \text{ are integers}, q \neq 0\}\);

    • -

  • The real numbers, \(\{x: -\infty < x < \infty\}\);

    • -

  • The complex numbers, \(\{a + bi: a,b \text{ are real numbers and } i^2=-1\}\).

    • -
-

On top of these, we have special subsets, such as the natural numbers \(\{1, 2, \dots\}\) (sometimes including \(0\)), the even numbers, the odd numbers, the positive numbers, the non-negative numbers, etc.

-

Mathematically, these number systems are naturally nested within each other as integers are rational numbers which are real numbers, which can be viewed as part of the complex numbers.

-

Calculators typically have just one type of number—floating point values. These model the real numbers.

-

Julia, on the other hand, has a rich type system, and within that has several different number types. There are types that model each of the four main systems above, and within each type, specializations for how these values are stored.

-

For now, let’s consider the number \(1\). It can be viewed as either an integer, rational, real, or complex number. To construct “\(1\)” in each type within Julia we have these different styles:

-
-
1, 1.0, 1//1, 1 + 0im
-
-
(1, 1.0, 1//1, 1 + 0im)
-
-
-

The basic number types in Julia are Int, Float64, Rational and Complex, though in fact there are many more, and the last two aren’t even concrete types. This distinction is important, as the type of number dictates how it will be displayed, how it will be stored, and how precisely the stored value can be expected to be to the mathematical value it models.

-

Though there are explicit constructors for these types, these notes avoid them unless necessary, as Julia’s parser can distinguish these types through an easy to understand syntax:

-
    -

  • integers have no decimal point;

    • -

  • floating point numbers have a decimal point (or are written with scientific notation);

    • -

  • rationals are constructed from integers using the double division operator, //; and

    • -

  • complex numbers are formed by including a term with the imaginary unit, im.

    • -
-
-
-
- -
-
-NoteWarning -
-
-
-

Heads up, the difference between 1 and 1.0 is subtle. Even more so, as 1. will parse as 1.0. This means some expressions, such as 2.*3, are ambiguous, as the . might be part of the 2 (as in 2. * 3) or the operation * (as in 2 .* 3).

-
-
-

The key distinction is between integers and floating points. While floating point values include integers, and so can be used exclusively on the calculator, the difference is that an integer is guaranteed to be an exact value, whereas a floating point value, while often an exact representation of a number is also often just an approximate value. This can be an advantage—floating point values can model a much wider range of numbers.

-

In nearly all cases the differences are not noticeable. To see why take, for instance, this simple calculation involving mixed types.

-
-
1 + 1.25 + 3//2
-
-
3.75
-
-
-

The sum of an integer, a floating point number and rational number returns a floating point number without a complaint.

-
-

Promotion

-

This is because behind the scenes, Julia will often “promote” the two numbers to a common type. In particular, before adding mixed-type numbers, the two are promoted to a common type by promote. In the example, first when computing 1 + 1.25 the integer 1 will be promoted to a floating point value, 1.0, and then the two values are added. Similarly, with 2.25 + 3//2, where the fraction is promoted to the floating point value 1.5 and afterwards addition is carried out.

-

We can see the promotion here:

-
-
promote(1, 1.25)
-
-
(1.0, 1.25)
-
-
-

and

-
-
promote(2.25, 3//2)
-
-
(2.25, 1.5)
-
-
-
-
-

Integers

-

Integers are often used casually, as they come about from parsing. As with a calculator, floating point numbers could be used for integers, but in Julia—and other languages—it proves useful to have numbers known to have exact values. Integers are needed for indexing and counting.

-

Except on older machines, the default integer is stored with 64 bits, though there are many available types for integers. With \(64\) bits, the range of integers that can be represented is \(-9223372036854775808=-(2^{63})\) to \(9223372036854775807 = 2^{63}-1\).

-
-
-

Floating point numbers

-

Floating point numbers are a model for the real numbers. With the same size storage, the integers provide exact numbers evenly spaced between the smallest and largest integer. Floating point values are exact for some values but as there are infinitely many real numbers are only approximations except in special cases. This leads to some differences between math done by hand and math done on the computer.

-
-

Float64

-

Float64 is the most common type of floating point number, as it is the most supported by the underlying hardware. Julia has other floating point types, notably Float32 and BigFloat for certain uses, but our focus here is on 64-bit floating point numbers.

-

The double-precision model for floating point numbers has three parts: a sign, an exponent (for a base of \(2\)), and a significand (in base \(2\)) representing numbers as \(\pm a \cdot 2^n\). The 64 bits are apportioned as follows: \(1\) is for the sign, \(11\) for the exponent, \(52\) for the significand.

-

The \(52\) bits of the significand are used to represent \(1.a_1a_2a_3\cdots a_{52}\) in base \(2\) or \(1 + a_12^{-1} + a_22^{-2} + a_32^{-3} + \cdots a_{52}2^{-52} = b/2^{52}\) for some integer \(b\). This means the significand represents a rational number.

-

The following shows the bits in the significand for a given number written in the form above:1

-
-
bitstring(1 + 1/2 + 1/4 + 0/8 + 1/16 + 1/32 + 0/64)[13:end] # 1101100…
-
-
"1101100000000000000000000000000000000000000000000000"
-
-
-

The 11 bits for the exponent covers a range from \(-1023\) to \(1024\) which in base \(10\) is around \(10^{-308}\) to \(10^{308}\).

-

Together these can represent exactly any rational number of the form \(\pm a \cdot 2^b\) where \(a\) is a sum of powers of \(1/2\) and \(b\) is an integer with \(1.0 \leq a \leq 1 + (1/2^1) + (1/2^2) + \cdots + (1/2^{52})\) and \(-1023 \leq b \leq 1024\).

-

Figure 1 shows the possible positive values were there only \(2\) bits for the exponent (for \(-1, 0, 1, 2\)) and \(2\) bits for the significand (\(1 + 0/4 + 0/2\), \(1 + 1/4 + 0/2\), \(1 + 0/4 + 1/2\), \(1 + 1/4 + 1/2\)). The main takeaway is that numbers get less concentrated the farther they get from \(0\).

-
-
-
-
-
-
- -
-
-
-
-
-Figure 1: Figure showing concentration of floating point values. The vertical ticks represent representable floating point values (were there only 2 bits (not 52) for the mantissa) and 2 bits (not 11) for the exponent. This leaves a range from a range of \(-1/2\) to not quite \(8\) being representable without using subnormal numbers. When the intervals double in length (from \([2^{i},2^{i+1}]\) to \([2^{i+1}, 2^{i+2}]\)) there are the same number of representable floating point values, so the concentration of representable values halves. With more bits there is a higher concentration, but the discrete nature is always present and leads to necessary approximations for modeling most all real numbers. -
-
-
-

In addition, there are special bit patterns recognized as 0.0 and even -0.0, which is a distinct number. There are also patterns for \(+\infty\) (Inf) and \(-\infty\) (-Inf). There are also patterns for NaN, or “not a number”, a value that is the result of some mathematical operations, such as 0.0 / 0.0. Finally, there are subnormal numbers representing even smaller numbers near \(0\) than described above, which are as small as \(2^{-1023} \approx 1.11 \cdot 10^{-308}\).

-
-
-

Scientific notation

-

Floating point numbers smaller than \(10^{-4}\) or bigger or equal to \(10^6\) (in absolute value) are displayed in scientific notation. Internally, most floating point numbers are stored in base \(2\) scientific notation as \(a \cdot 2^b\) with \(a=1.xxx\dots\). But when displayed, numbers are represented in base \(10\) and when scientific notation is used the numbers are normalized in the from \(a \cdot 10^b\) where \(1.0 \leq a < 10\).

-

The significand and exponent are separated by the character e—which is not the same as the constant \(e\)—rather denotes a 64-bit number separated into a significand and an exponent by a formatting character. (Float32 uses an f as a separator.)

-

Consider these two numbers one close to \(0\) one far from \(0\):

-
-
0.0000000123456789, 123456789.0
-
-
(1.23456789e-8, 1.23456789e8)
-
-
-

Their display is subtly different, as only a minus sign after e distinguishes them.

-

The parser will read in numbers with an e in the proper format as though they are scientific notation:

-
-
1e8
-
-
1.0e8
-
-
-

The above creates the same value as 10.0^8, but not 1e^8 which will error unless a value for e has been assigned.

-
-
-

Inexactness and consequences

-

For numbers not representable in floating point, some rounding must go on to fit the number into a representable floating point value. As such, some computed values are not quite what they would be mathematically:

-
-
sqrt(2) * sqrt(2) - 2, sin(1pi)
-
-
(4.440892098500626e-16, 1.2246467991473532e-16)
-
-
-

These values are very small numbers, but not exactly \(0\), as they are mathematically.

-

More surprisingly, simple fractions may also lead to mathematically different results:

-
-
1/10 + 2/10 - 3/10
-
-
5.551115123125783e-17
-
-
-

This, of course, is due to none of these fractions being of the form \(a\cdot 2^b\) for integers \(a, b\).

-

Another surprise: floating point addition is not necessarily associative. That is the property \(a + (b+c) = (a+b) + c\) may not hold exactly. For example:

-
-
l2r = (1/10 + 2/10) + 3/10
-r2l = 1/10 + (2/10 + 3/10)
-l2r - r2l
-
-
1.1102230246251565e-16
-
-
-

One other surprise. Mathematically, for real numbers, subtraction of similar-sized numbers is not exceptional, for example \(1 - \cos(x)\) is positive if \(0 < x < \pi/2\), say. This will not be the case for floating point values. If \(x\) is close enough to \(0\), then \(\cos(x)\) and \(1\) will be so close, that they will be represented by the same floating point value, 1.0, so the difference will be zero:

-
-
1.0 - cos(1e-8)
-
-
0.0
-
-
-
-
-
-

Rational numbers

-

Rational numbers can be used when the exactness of the number is more important than the speed or wider range of values offered by floating point numbers. In Julia a rational number is comprised of a numerator and a denominator, each an integer of the same type, and reduced to lowest terms. The operations of addition, subtraction, multiplication, and division will keep their answers as rational numbers. As well, raising a rational number to an integer value will produce a rational number.

-

As mentioned, these are constructed using double slashes:

-
-
1//2, 2//1, 6//4
-
-
(1//2, 2//1, 3//2)
-
-
-

Rational numbers are exact, so the following are identical to their mathematical counterparts:

-
-
1//10 + 2//10 == 3//10
-
-
true
-
-
-

and associativity:

-
-
(1//10 + 2//10) + 3//10 == 1//10 + (2//10 + 3//10)
-
-
true
-
-
-

Here we see that the type is preserved under the basic operations:

-
-
(1//2 + 1//3 * 1//4 / 1//5) ^ 6
-
-
1771561//2985984
-
-
-

For powers, a non-integer exponent is converted to floating point, so this operation is defined, though will always return a floating point value:

-
-
(1//2)^(1//2)   # the first parentheses are necessary as `^` will be evaluated before `//`.
-
-
0.7071067811865476
-
-
-
-
-

Complex numbers

-

Complex numbers in Julia are stored as two numbers, a real and imaginary part, each some type of Real number. The special constant im is used to represent \(i=\sqrt{-1}\). This makes the construction of complex numbers fairly standard:

-
-
1 + 2im, 3 + 4.0im
-
-
(1 + 2im, 3.0 + 4.0im)
-
-
-

(These two aren’t exactly the same, the 3 is promoted from an integer to a float to match the 4.0. Each of the components must be of the same type of number.)

-

Mathematically, complex numbers are needed so that certain equations can be satisfied. For example \(x^2 = -2\) has solutions \(-\sqrt{2}i\) and \(\sqrt{2}i\) over the complex numbers. Finding this in Julia requires some attention, as we have both sqrt(-2) and sqrt(-2.0) throwing a DomainError, as the sqrt function expects non-negative real arguments. However first creating a complex number and then taking a square root does work:

-
-
sqrt(-2 + 0im)
-
-
0.0 + 1.4142135623730951im
-
-
-

For complex arguments, the sqrt function will return complex values (even if the answer is a real number).

-

This means, if you wanted to perform the quadratic equation for any real inputs, your computations might involve something like the following:

-
-
a,b,c = 1,2,3  ## x^2 + 2x + 3
-discr = b^2 - 4a*c
-(-b + sqrt(discr + 0im))/(2a), (-b - sqrt(discr + 0im))/(2a)
-
-
(-1.0 + 1.4142135623730951im, -1.0 - 1.4142135623730951im)
-
-
-

When learning calculus, the only common usage of complex numbers arises when solving polynomial equations for roots, or zeros, though they are very important for subsequent work using the concepts of calculus.

-
-
-

Irrational numbers

-

Julia has a a few mathematical constants that are stored with a special type Irrational. One such value is pi. There are others in the Base.MathConstants module, and an external package IrrationalConstants.jl.

-

Irrational values may have special methods defined for them which can lead to subtle differences, such as:

-
-
sin(pi), sin(2pi)
-
-
(0.0, -2.4492935982947064e-16)
-
-
-

In computing the product 2pi first the two values are promoted to Float64 and then multiplied, leaving a floating-point approximation of \(2\pi\) for sin to evaluate.

-
-
-

Other types of data: strings and symbols

-

For text, Julia has a String type. When double quotes are used to specify a string, the parser creates this type:

-
-
x = "The quick brown fox jumped over the lazy dog"
-typeof(x)
-
-
String
-
-
-

Values can be inserted into a string through interpolation using a dollar sign.

-
-
animal = "lion"
-x = "The quick brown $(animal) jumped over the lazy dog"
-
-
"The quick brown lion jumped over the lazy dog"
-
-
-

The use of parentheses allows more complicated expressions; it isn’t always necessary.

-

Longer strings can be produced using triple quotes:

-
-
lincoln = """
-Four score and seven years ago our fathers brought forth, upon this continent, a new nation, conceived in Liberty, and dedicated to the proposition that all men are created equal.
-"""
-
-
"Four score and seven years ago our fathers brought forth, upon this continent, a new nation, conceived in Liberty, and dedicated to the proposition that all men are created equal.\n"
-
-
-

Strings are comprised of characters which can be produced directly using single quotes:

-
-
'c'
-
-
'c': ASCII/Unicode U+0063 (category Ll: Letter, lowercase)
-
-
-

We won’t use characters in these notes.

-

Finally, Julia has symbols which are interned strings which are used as identifiers. Symbols are used for advanced programming techniques; we will only see them as shortcuts to specify plotting arguments.

-
-
-

Questions

-
-
Question
-

The number created by pi/2 is?

-
-
- -
-
-
-
- -
-
-Select an item -
- -
-
- -
-
- -
-
- -
-
- -
-
- -
-
-
-
- - - - -
-
-
-
-
Question
-

The number created by 2/2 is?

-
-
- -
-
-
-
- -
-
-Select an item -
- -
-
- -
-
- -
-
- -
-
- -
-
- -
-
-
-
- - - - -
-
-
-
-
Question
-

The number created by 2//2 is?

-
-
- -
-
-
-
- -
-
-Select an item -
- -
-
- -
-
- -
-
- -
-
- -
-
- -
-
-
-
- - - - -
-
-
-
-
Question
-

The number created by 1 + 1//2 + 1/3 is?

-
-
- -
-
-
-
- -
-
-Select an item -
- -
-
- -
-
- -
-
- -
-
- -
-
- -
-
-
-
- - - - -
-
-
-
-
Question
-

The number created by 2^3 is?

-
-
- -
-
-
-
- -
-
-Select an item -
- -
-
- -
-
- -
-
- -
-
- -
-
- -
-
-
-
- - - - -
-
-
-
-
Question
-

The number created by sqrt(im) is?

-
-
- -
-
-
-
- -
-
-Select an item -
- -
-
- -
-
- -
-
- -
-
- -
-
- -
-
-
-
- - - - -
-
-
-
-
Question
-

The number created by 2^(-1) is?

-
-
- -
-
-
-
- -
-
-Select an item -
- -
-
- -
-
- -
-
- -
-
- -
-
- -
-
-
-
- - - - -
-
-
-
-
Question
-

The “number” created by 1/0 is?

-
-
- -
-
-
-
- -
-
-Select an item -
- -
-
- -
-
- -
-
- -
-
- -
-
- -
-
-
-
- - - - -
-
-
-
-
Question
-

Is (2 + 6) + 7 equal to 2 + (6 + 7)?

-
-
- -
-
-
-
- -
-
-Select an item -
- -
-
- -
-
- -
-
-
-
- - - - -
-
-
-
-
Question
-

Is (2/10 + 6/10) + 7/10 equal to 2/10 + (6/10 + 7/10)?

-
-
- -
-
-
-
- -
-
-Select an item -
- -
-
- -
-
- -
-
-
-
- - - - -
-
-
-
-
Question
-

The following should compute 2^(-1), which if entered directly will return 0.5. Does it?

-
-
a, b = 2, -1
-a^b
-
-
-
- -
-
-
-
- -
-
-Select an item -
- -
-
- -
-
- -
-
-
-
- - - - -
-
-

(This shows the special casing that is done when powers use literal numbers.)

-
-
-
Question
-

In NewScientist we learn “For the first time, physicists have measured changes in an atom to the level of zeptoseconds, or trillionths of a billionth of a second—the smallest division of time yet observed.”

-

That is

-
-
1e-9 / 1e12
-
-
1.0000000000000001e-21
-
-
-

Finding the value through division introduces a floating point deviation. Which of the following values will directly represent a zeptosecond?

-
-
- -
-
-
-
- -
-
- - -
- - -
-
-
- -
- - - - -
-
-
-
-
Question
-

Signed integers are stored on a computer in a special manner. We will see with 8 bit integers, formed by Int8. Eight bit means only 8 0’s or 1’s are used to store a given number. This is a useful format for storing many small integers but for this example, useful as we can more easily track the values.

-

The first bit is a sign bit. Based on these two outputs, can you guess how that works:

-
-
bitstring(Int8(-1)), bitstring(Int8(1))
-
-
("11111111", "00000001")
-
-
-
-
- -
-
-
-
- -
-
- - -
- - -
-
-
-
- - - - -
-
-

Positive numbers and negative numbers are stored a bit differently. Positive numbers just use binary: \(a_0 \cdot 2^0 + a_1 \cdot 2^1 + a_2 \cdot 2^2 + \cdots a_7 \cdot 2^7\). The number \(27\) is \(1 + 2 + 8 + 16\). so have \(a_0 = a_1 = a_3 = a_4 = 1\), the others are \(0\). The bitstring shows:

-
-
bitstring(Int8(27))
-
-
"00011011"
-
-
-

Which bit pattern is used?

-
-
- -
-
-
-
- -
-
- - -
- - -
-
-
-
- - - - -
-
-

Negative numbers are stored using two’s complement format:

-
    -
  • represent the positive number
    • -
  • flip 0 to 1; 1 to 0
    • -
  • add 1 to the value (long addition with carrying)
    • -
-

For \(-27\) we have

-
    -
  • first \(27\): 00011011
    • -
  • then flip each: 11100100
    • -
  • finally add \(1\): 11100101
    • -
-
-
bitstring(Int8(-27))
-
-
"11100101"
-
-
-

The largest positive number is \(127\) for 8-bits and is represented by 01111111. What is the bit pattern of \(-127\)?

-
-
- -
-
-
-
- -
-
- - -
- - -
-
-
- -
- - - - -
-
-

The smallest negative number is \(-128\). Why?

-
-
- -
-
-
-
- -
-
- - -
- - -
-
-
-
- - - - -
-
-

Why all this fuss? Couldn’t there be an easier way?

-

This storage has a big advantage when adding numbers. Let’s look at adding \(-5\) to \(6\). we have:

-
-5 => 11111011
- 6 => 00000110
-      --------
-     100000001  => 00000001
-

The addition is done by carrying a 1 across and then dropping the 9th number when there is such a carry. This leaves the representation for what number?

-
-
- -
-
-
-
- -
-
- - - -
- - -
-
-
-
- - - - -
-
-

The largest positive number that can be represented is \(2^0 + 2^1 + 2^2 + \cdots + 2^7\), where \(7\) is the number of bits minus \(1\). The representation is 01111111. What happens if we add 1 to this number?

-
-
- -
-
-
-
- -
-
- - -
- - -
-
-
-
- - - - -
-
-

The largest possible number for a type is returned by typemax. For Int64 (just Int on most systems) what is the largest number?

-
-
- -
-
-
-
- -
-
-
- -
- -
-
-
-
- - - - -
-
-
-
-
Question
-

The Float64 type uses \(11\) bits for an exponent (base \(2\)) between \(-1023\) and \(1024\). We can see how these are stored as follows:

-
-
bitstring(2.0^(-1023))[2:12], bitstring(2.0^(1024))[2:12]
-
-
("00000000000", "11111111111")
-
-
-

This is the full range of values. However the values are shifted with \(0\) representing \(-1023\) and \(x\) representing \(1024\). The value \(x\) is can be found from:

-
-
2^0 + 2^1 + 2^2 + 2^3 + 2^4 + 2^5 + 2^6 + 2^7 + 2^8 + 2^9 + 2^10
-
-

What is the value of \(x\)?

-
-
- -
-
-
-
- -
-
-
- -
- -
-
-
-
- - - - -
-
-

The value 1023 is called a bias. The exponent is coded as the binary value as a positve integer minus \(1023\). A bias is used, and not the two’s complement format, as storage with a bias makes multiplying by powers of \(2\) as easy as shifting the bits.

-

To find the storage for, say, \(2^4 + 2^2 + 2^0\) or 00000010101 we would add 1023 or 01111111111 and see:

-
  00000010101
-+ 01111111111
-  -----------
-  10000010100
-

Which we can see:

-
-
bitstring(2.0^(2^4 + 2^2 + 2^0))[2:12]
-
-
"10000010100"
-
-
-
-
- - -

Footnotes

- -
    -

  1. The output of bitstring is 64 characters. The first is the sign bit, the second through twelfth the exponent, the rest the significand. The notation [13:end] is used to return just those for the significand. A value of [2:12] would return the bits for the exponent.↩︎

    2. -
-
- - -
- - - - - \ No newline at end of file diff --git a/quarto/basics/numbers_types-II.qmd b/quarto/basics/numbers_types-II.qmd index 8e56a0b..b3df377 100644 --- a/quarto/basics/numbers_types-II.qmd +++ b/quarto/basics/numbers_types-II.qmd @@ -657,7 +657,7 @@ val = sum(2^i for i in 0:10) numericq(val) ``` -The value `1023` is called a bias. The exponent is coded as the binary value as a positve integer minus $1023$. A bias is used, and not the two's complement format, as storage with a bias makes multiplying by powers of $2$ as easy as shifting the bits. +The value `1023` is called a bias. The exponent is coded as the binary value as a positive integer minus $1023$. A bias is used, and not the two's complement format, as storage with a bias makes multiplying by powers of $2$ as easy as shifting the bits. To find the storage for, say, $2^4 + 2^2 + 2^0$ or `00000010101` we would add `1023` or `01111111111` and see: diff --git a/quarto/basics/numbers_types.qmd b/quarto/basics/numbers_types.qmd index da47b11..399e784 100644 --- a/quarto/basics/numbers_types.qmd +++ b/quarto/basics/numbers_types.qmd @@ -677,7 +677,7 @@ val = sum(2^i for i in 0:10) numericq(val) ``` -The value `1023` is called a bias. The exponent is coded as the binary value as a positve integer minus $1023$. A bias is used, and not the two's complement format, as storage with a bias makes multiplying by powers of $2$ as easy as shifting the bits. +The value `1023` is called a bias. The exponent is coded as the binary value as a positive integer minus $1023$. A bias is used, and not the two's complement format, as storage with a bias makes multiplying by powers of $2$ as easy as shifting the bits. To find the storage for, say, $2^4 + 2^2 + 2^0$ or `00000010101` we would add `1023` or `01111111111` and see: diff --git a/quarto/basics/vectors.qmd b/quarto/basics/vectors.qmd index 1247b5e..83c8d9c 100644 --- a/quarto/basics/vectors.qmd +++ b/quarto/basics/vectors.qmd @@ -33,7 +33,7 @@ $$ This formula agrees with Pythagorean's theorem for right triangles. -For $n$-dimensional points, the same formula may be used with adjustements to the notation. Suppose $P = (x_1, x_2, \dots, x_n)$ and $Q = (y_1, y_2, \cdots, y_n)$. then +For $n$-dimensional points, the same formula may be used with adjustments to the notation. Suppose $P = (x_1, x_2, \dots, x_n)$ and $Q = (y_1, y_2, \cdots, y_n)$. then $$ d = \overline{PQ} = \sqrt{(x_1 - y_1)^2 + (x_2 - y_2)^2 + \cdots + (x_n - y_n)^2}. @@ -56,7 +56,7 @@ A two-dimensional vector has two components, as does a point in the Cartesian pl Suppose a vector is defined as being between two points $P = (x_1, y_1)$ and $Q = (x_2, y_2)$ with $P$ the endpoint, then the vector connecting $P$ to $Q$ would be $\vec{v} = \langle x_2 - x_1, ~ y_2 - y_1 \rangle$. -When $P = (0,0)$, the the point $Q = (x_2, y_2)$ has the same components as the vector $\vec{v} = \langle x_2, ~ y_2 \rangle$ leading to a natural indentification between a point and vector, though they represent different things. +When $P = (0,0)$, the the point $Q = (x_2, y_2)$ has the same components as the vector $\vec{v} = \langle x_2, ~ y_2 \rangle$ leading to a natural identification between a point and vector, though they represent different things. @@ -767,7 +767,7 @@ Broadcasting is a widely used and powerful surface syntax which we will employ o #### Mapping a function over a collection -The `map` function is very much related to broadcasting, in that it applies a function to each element of an iterable. When more than one iterable is specifed, `map` applies the function to the `zip`ped iterables. Unlike broadcasting, `map` does not reshape the underlying iterables. +The `map` function is very much related to broadcasting, in that it applies a function to each element of an iterable. When more than one iterable is specified, `map` applies the function to the `zip`ped iterables. Unlike broadcasting, `map` does not reshape the underlying iterables. Similarly named functions are found in many different programming languages, as `map` is one of the foundational higher-order, functional programming operations. (The "dot" broadcast is mostly limited to `Julia` and mirrors a similar usage of a dot in `MATLAB`.) For those familiar with other programming languages, using `map` may seem more natural. Its syntax is `map(f, xs)`. Additional iterables are passed after `xs`. @@ -787,7 +787,7 @@ sum(map(sin, xs)) This has a performance drawback---there are two passes through the container, one to apply `sin` another to add. -For this task, the `sum` reduction, as others, allows a function to be specified that is applied to each value in the container while the sum is being computed. This argument comes first. A recommened alternative to the previous would be: +For this task, the `sum` reduction, as others, allows a function to be specified that is applied to each value in the container while the sum is being computed. This argument comes first. A recommended alternative to the previous would be: ```{julia} sum(sin, xs) diff --git a/quarto/derivatives/curve_sketching.qmd b/quarto/derivatives/curve_sketching.qmd index e9aa229..ee27dee 100644 --- a/quarto/derivatives/curve_sketching.qmd +++ b/quarto/derivatives/curve_sketching.qmd @@ -112,7 +112,7 @@ let end # (2) periodic behaviour, - i == 2 && (title = "No periodic behavious") + i == 2 && (title = "No periodic behaviours") if i >= 2 end diff --git a/quarto/derivatives/first_second_derivatives.qmd b/quarto/derivatives/first_second_derivatives.qmd index 869d4b4..2a85f45 100644 --- a/quarto/derivatives/first_second_derivatives.qmd +++ b/quarto/derivatives/first_second_derivatives.qmd @@ -58,7 +58,7 @@ A parallel definition with $a < b$ implying $f(a) > f(b)$ would be used for a *s ::: -We introduce a helper function `plotif` from the `CalculusWithJulia` package that highlights the graph of a function $f$ when another function $g(x)$ satisifies $g(x) \geq 0$. This function is called as `plotif(f, g, a, b)`. +We introduce a helper function `plotif` from the `CalculusWithJulia` package that highlights the graph of a function $f$ when another function $g(x)$ satisfies $g(x) \geq 0$. This function is called as `plotif(f, g, a, b)`. To see where a function is positive, we simply pass the function object in for *both* `f` and `g` above. For example, in @fig-plotif-sin-sin-minus-2pi-2pi we look at where $f(x) = \sin(x)$ is positive. diff --git a/quarto/differentiable_vector_calculus/test.html b/quarto/differentiable_vector_calculus/test.html deleted file mode 100644 index 26d9d85..0000000 --- a/quarto/differentiable_vector_calculus/test.html +++ /dev/null @@ -1,641 +0,0 @@ - - - - - - - - - -test - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- -
- -
-
-

Applications with scalar functions

-
- - - -
- - - - -
- - - -
- - -

This section uses these add-on packages:

-
-
using CalculusWithJulia
-using Plots
-plotly()
-using SymPy
-using Roots
-
-
-
Example
-

Consider the function \(f(x,y) = x^2 + 3y^2 -x\) over the region \(x^2 + y^2 \leq 1\). This is a continuous function over a closed set, so will have both an absolute maximum and minimum. Find these from an investigation of the critical points and the boundary points.

-

The gradient is easily found: \(\nabla{f} = \langle 2x - 1, 6y \rangle\), and is \(\vec{0}\) only at \(\vec{a} = \langle 1/2, 0 \rangle\). The Hessian is:

-

\[ -H = -\begin{bmatrix} -2 & 0\\ -0 & 6 -\end{bmatrix}. -\]

-

At \(\vec{a}\) this has positive determinant and \(f_{xx} > 0\), so \(\vec{a}\) corresponds to a local minimum with values \(f(\vec{a}) = (1/2)^2 + 3(0) - 1/2 = -1/4\). The absolute maximum and minimum may occur here (well, not the maximum) or on the boundary, so that must be considered. In this case we can easily parameterize the boundary and turn this into the univariate case:

-
-
fₗ(x,y) = x^2 + 2y^2 - x
-gammaₗ(t) = [cos(t), sin(t)]  # traces out x^2 + y^2 = 1 over [0, 2pi]
-gₗ = splat(fₗ) ∘ gammaₗ
-
-cpsₗ = find_zeros(gₗ', 0, 2pi) # critical points of g
-append!(cpsₗ, [0, 2pi])
-unique!(cpsₗ)
-gₗ.(cpsₗ)
-
-
5-element Vector{Float64}:
- 0.0
- 2.25
- 2.0
- 2.25
- 0.0
-
-
-

We see that maximum value is 2.25 and that the interior point, \(\vec{a}\), will be where the minimum value occurs. To see exactly where the maximum occurs, we look at the values of gamma:

-
-
inds = [2,4]
-cpsₗ[inds]
-
-
2-element Vector{Float64}:
- 2.0943951023931953
- 4.1887902047863905
-
-
-

These are multiples of \(\pi\):

-
-
cpsₗ[inds]/pi
-
-
2-element Vector{Float64}:
- 0.6666666666666666
- 1.3333333333333333
-
-
-

So we have the maximum occurs at the angles \(2\pi/3\) and \(4\pi/3\). Here we visualize, using a hacky trick of assigning NaN values to the function to avoid plotting outside the circle:

-
-
hₗ(x,y) = fₗ(x,y) * (x^2 + y^2 <= 1 ? 1 : NaN)
-
-
hₗ (generic function with 1 method)
-
-
-
-
gr()
-
-
Plots.GRBackend()
-
-
-
-
xs = ys = range(-1,1, length=100)
-plt = surface(xs, ys, hₗ)
-
-ts = cpsₗ  # 2pi/3 and 4pi/3 by above
-xs, ys = cos.(ts), sin.(ts)
-zs = fₗ.(xs, ys)
-scatter3d!(xs, ys, zs)
-#tuple.(xs, ys, zs)
-#plot!(plt, tuple.(xs, ys, zs); linetype=:scatter)
-#plt
-
- -
-
-

A contour plot also shows that some—and only one—extrema happens on the interior:

- -
- -
- - -
- - - - - \ No newline at end of file diff --git a/quarto/differentiable_vector_calculus/test.jl b/quarto/differentiable_vector_calculus/test.jl deleted file mode 100644 index fe58175..0000000 --- a/quarto/differentiable_vector_calculus/test.jl +++ /dev/null @@ -1,827 +0,0 @@ -@show 4 -using QuizQuestions -using LaTeXStrings -using CalculusWithJulia -using Plots -plotly() -using SymPy -using Roots -@show 6 -import Contour: contours, levels, level, lines, coordinates -@show 15 -@syms f_x f_y -n = [1, 0, f_x] × [0, 1, f_y] -@show 27 -#| hold: true -f(x,y) = 6 - x^2 -y^2 -f(x)= f(x...) - -a,b = 1, -1/2 - - -# draw surface -xr = 7/4 -xs = ys = range(-xr, xr, length=100) -surface(xs, ys, f, legend=false) - -# visualize tangent plane as 3d polygon -pt = [a,b] -tplane(x) = f(pt) + gradient(f)(pt) ⋅ (x - [a,b]) - -pts = [[a-1,b-1], [a+1, b-1], [a+1, b+1], [a-1, b+1], [a-1, b-1]] -plot!(unzip([[pt..., tplane(pt)] for pt in pts])...) - -# plot paths in x and y direction through (a,b) -γ_x(t) = pt + t*[1,0] -γ_y(t) = pt + t*[0,1] - -plot_parametric!((-xr-a)..(xr-a), t -> [γ_x(t)..., (f∘γ_x)(t)], linewidth=3) -plot_parametric!((-xr-b)..(xr-b), t -> [γ_y(t)..., (f∘γ_y)(t)], linewidth=3) - -# draw directional derivatives in 3d and normal -pt = [a, b, f(a,b)] -fx, fy = gradient(f)(a,b) -arrow!(pt, [1, 0, fx], linewidth=3) -arrow!(pt, [0, 1, fy], linewidth=3) -arrow!(pt, [-fx, -fy, 1], linewidth=3) # normal - -# draw point in base, x-y, plane -pt = [a, b, 0] -scatter!(unzip([pt])...) -arrow!(pt, [1,0,0], linestyle=:dash) -arrow!(pt, [0,1,0], linestyle=:dash) -@show 33 -function tangent_plane_1st_crack(f, pt) - fx, fy = ForwardDiff.gradient(f, pt) - x -> f(x...) + fx * (x[1]-pt[1]) + fy * (x[2]-pt[2]) -end -@show 35 -function tangent_plane(f, pt) - ∇f = ForwardDiff.gradient(f, pt) # using a variable ∇f - x -> f(pt) + ∇f ⋅ (x - pt) -end -@show 46 -@syms x, y -@show 47 -#| hold: true -f(x,y) = sin(x) * cos(x-y) -f(x) = f(x...) -vars = [x, y] - -gradf = diff.(f(x,y), vars) # or use gradient(f, vars) or ∇((f,vars)) - -pt = [PI/4, PI/3] -gradfa = subs.(gradf, x=>pt[1], y=>pt[2]) - -f(pt) + gradfa ⋅ (vars - pt) -@show 55 -#| hold: true -a = 1 -gamma(t) = a * [1 + cos(t), sin(t), 2sin(t/2) ] -P = gamma(1/2) -n1(x,y,z)= [2*(x-a), 2y, 0] -n2(x,y,z) = [2x,2y,2z] -n1(x) = n1(x...) -n2(x) = n2(x...) - -t = 1/2 -(n1(gamma(t)) × n2(gamma(t))) × gamma'(t) -@show 60 -#| hold: true -a, b = 1, 3 -f(x,y,z) = (x^2 + ((1+b) * y)^2 + z^2 - 1)^3 - x^2 * z^3 - a * y^2 * z^3 - -CalculusWithJulia.plot_implicit_surface(f, xlim=-2..2, ylim=-1..1, zlim=-1..2) -@show 71 -V(r, h) = pi * r^2 * h -V(v) = V(v...) -a₁ = [1,2] -dx₁ = [0.01, 0.01] -ForwardDiff.gradient(V, a₁) ⋅ dx₁ # or use ∇(V)(a) -@show 73 -V(a₁ + dx₁) - V(a₁) -@show 85 -#| hold: true -f(x,y,z) = x^4 -x^3 + y^2 + z^2 -f(v) = f(v...) -a, b,c = ∇(f)(2,2,2) -"$a x + $b y + $c z = $([a,b,c] ⋅ [2,2,2])" -#@show 92 -#| hold: true -@syms a b c d u v -M = [a b; c d] -B = [u, v] -M \ B .|> simplify -@show 96 -#| hold: true -#| echo: false -f(x,y) = 2 - x^2 - y^2 -g(x,y) = 3 - 2x^2 - (1/3)y^2 -xs = ys = range(-3, stop=3, length=100) -zfs = [f(x,y) for x in xs, y in ys] -zgs = [g(x,y) for x in xs, y in ys] - - -ps = Any[] -pf = surface(xs, ys, f, alpha=0.5, legend=false) - -for cl in levels(contours(xs, ys, zfs, [0.0])) - for line in lines(cl) - _xs, _ys = coordinates(line) - plot!(pf, _xs, _ys, 0*_xs, linewidth=3, color=:blue) - end -end - - -pg = surface(xs, ys, g, alpha=0.5, legend=false) -for cl in levels(contours(xs, ys, zgs, [0.0])) - for line in lines(cl) - _xs, _ys = coordinates(line) - plot!(pg, _xs, _ys, 0*_xs, linewidth=3, color=:red) - end -end - -pcnt = plot(legend=false) -for cl in levels(contours(xs, ys, zfs, [0.0])) - for line in lines(cl) - _xs, _ys = coordinates(line) - plot!(pcnt, _xs, _ys, linewidth=3, color=:blue) - end -end - -for cl in levels(contours(xs, ys, zgs, [0.0])) - for line in lines(cl) - _xs, _ys = coordinates(line) - plot!(pcnt, _xs, _ys, linewidth=3, color=:red) - end -end - -l = @layout([a b c]) -plot(pf, pg, pcnt, layout=l) -@show 106 -function newton_step(f, g, xn) - M = [ForwardDiff.gradient(f, xn)'; ForwardDiff.gradient(g, xn)'] - b = -[f(xn), g(xn)] - Delta = M \ b - xn + Delta -end -@show 108 -𝒇(x,y) = 2 - x^2 - y^2 -𝒈(x,y) = 3 - 2x^2 - (1/3)y^2 -𝒇(v) = 𝒇(v...); 𝒈(v) = 𝒈(v...) -𝒙₀ = [1,1] -𝒙₁ = newton_step(𝒇, 𝒈, 𝒙₀) -@show 110 -𝒇(𝒙₁), 𝒈(𝒙₁) -@show 112 -𝒙₂ = newton_step(𝒇, 𝒈, 𝒙₁) -𝒙₃ = newton_step(𝒇, 𝒈, 𝒙₂) -𝒙₄ = newton_step(𝒇, 𝒈, 𝒙₃) -𝒙₅ = newton_step(𝒇, 𝒈, 𝒙₄) -𝒙₅, 𝒇(𝒙₅), 𝒈(𝒙₅) -@show 116 -function nm(f, g, x, n=5) - for i in 1:n - x = newton_step(f, g, x) - end - x -end -@show 123 -#| hold: true -c = 1/2 -f(x,y) = 1 - y^2 - c^2 -g(x,y) = (1 - x^2) - c^2 -f(v) = f(v...); g(v) = g(v...) -nm(f, g, [1/2, 1/3]) -@show 148 -#| hold: true -@syms x, y, Z() -∂x = solve(diff(x^4 -x^3 + y^2 + Z(x,y)^2, x), diff(Z(x,y),x)) -∂y = solve(diff(x^4 -x^3 + y^2 + Z(x,y)^2, y), diff(Z(x,y),y)) -∂x, ∂y -@show 158 -f(x, p) = cos(x) - p*x -p = 2 -xᵅ = find_zero(f, (0, pi/2), p) -@show 160 -p = 2 -xᵅ = find_zero(f, (0, pi/2), p) -fₓ = ForwardDiff.derivative(x -> f(x,p), xᵅ) -fₚ = ForwardDiff.derivative(p -> f(xᵅ, p), p) -- fₚ / fₓ -@show 163 -function find_zero_derivative(f, x₀, p) - xᵅ = find_zero(f, x₀, p) - fₓ = ForwardDiff.derivative(x -> f(x,p), xᵅ) - fₚ = ForwardDiff.derivative(p -> f(xᵅ, p), p) - - fₚ / fₓ -end -F(p) = find_zero_derivative(f, (0, pi/2), p) -plot(F, 0.01, 5) # p > 0 -@show 183 -#| hold: true -f(x,y)= exp(-(x^2 + y^2)/5) * cos(x^2 + y^2) -xs = ys = range(-4, 4, length=100) -surface(xs, ys, f, legend=false) -@show 190 -#| hold: true -f(x,y) = x*y -xs = ys = range(-3, 3, length=100) -surface(xs, ys, f, legend=false) - -plot_parametric!(-4..4, t -> [t, 0, f(t, 0)], linewidth=5) -plot_parametric!(-4..4, t -> [0, t, f(0, t)], linewidth=5) -@show 203 -fₖ(x,y) = exp(-(x^2 + y^2)/5) * cos(x^2 + y^2) -Hₖ = sympy.hessian(fₖ(x,y), (x,y)) -@show 205 -H₀₀ = subs.(Hₖ, x=>0, y=>0) -@show 207 -H₀₀[1,1] < 0 && det(H₀₀) > 0 -@show 209 -#| hold: true -gradfₖ = diff.(fₖ(x,y), [x,y]) -a = [sqrt(2PI + atan(-Sym(1)//5)), 0] -subs.(gradfₖ, x => a[1], y => a[2]) -@show 211 -#| hold: true -a = [sqrt(PI + atan(-Sym(1)//5)), 0] -H_a = subs.(Hₖ, x => a[1], y => a[2]) -det(H_a) -@show 216 -fⱼ(x,y) = 4x*y - x^4 - y^4 -gradfⱼ = diff.(fⱼ(x,y), [x,y]) -@show 217 -all_ptsⱼ = solve(gradfⱼ, [x,y]) -ptsⱼ = filter(u -> all(isreal.(u)), all_ptsⱼ) -@show 219 -Hⱼ = sympy.hessian(fⱼ(x,y), (x,y)) -function classify(H, pt) - Ha = subs.(H, x => pt[1], y => pt[2]) - (det=det(Ha), f_xx=Ha[1,1]) -end -[classify(Hⱼ, pt) for pt in ptsⱼ] -@show 221 -#| hold: true -xs = ys = range(-3/2, 3/2, length=100) -p = surface(xs, ys, fⱼ, legend=false) -for pt ∈ ptsⱼ - scatter!(p, unzip([N.([pt...,fⱼ(pt...)])])..., - markercolor=:black, markersize=5) # add each pt on surface -end -p -@show 228 -fₗ(x,y) = x^2 + 2y^2 - x -fₗ(v) = fₗ(v...) -gammaₗ(t) = [cos(t), sin(t)] # traces out x^2 + y^2 = 1 over [0, 2pi] -gₗ = fₗ ∘ gammaₗ - -cpsₗ = find_zeros(gₗ', 0, 2pi) # critical points of g -append!(cpsₗ, [0, 2pi]) -unique!(cpsₗ) -gₗ.(cpsₗ) -@show 230 -inds = [2,4] -cpsₗ[inds] -@show 232 -cpsₗ[inds]/pi -@show 234 -hₗ(x,y) = fₗ(x,y) * (x^2 + y^2 <= 1 ? 1 : NaN) -@show 235 -#| hold: true -xs = ys = range(-1,1, length=100) -surface(xs, ys, hₗ) - -ts = cpsₗ # 2pi/3 and 4pi/3 by above -xs, ys = cos.(ts), sin.(ts) -zs = fₗ.(xs, ys) -scatter3d!(xs, ys, zs) -@show 237 -#| hold: true -xs = ys = range(-1,1, length=100) -contour(xs, ys, hₗ) -@show 243 -@syms x1 y1 x2 y2 x3 y3 -d2(p,x) = (p[1] - x[1])^2 + (p[2]-x[2])^2 -d2_1, d2_2, d2_3 = d2((x,y), (x1, y1)), d2((x,y), (x2, y2)), d2((x,y), (x3, y3)) -exₛ = d2_1 + d2_2 + d2_3 -@show 245 -gradfₛ = diff.(exₛ, [x,y]) -xstarₛ = solve(gradfₛ, [x,y]) -@show 248 -Hₛ = subs.(hessian(exₛ, [x,y]), x=>xstarₛ[x], y=>xstarₛ[y]) -@show 259 -usₛ = [[cos(t), sin(t)] for t in (0, 2pi/3, 4pi/3)] -polygon(ps) = unzip(vcat(ps, ps[1:1])) # easier way to plot a polygon - -pₛ = scatter([0],[0], markersize=2, legend=false, aspect_ratio=:equal) - -asₛ = (1,2,3) -plot!(polygon([a*u for (a,u) in zip(asₛ, usₛ)])...) -[arrow!([0,0], a*u, alpha=0.5) for (a,u) in zip(asₛ, usₛ)] -pₛ -@show 261 -asₛ₁ = (1, -1, 3) -scatter([0],[0], markersize=2, legend=false) -psₛₗ = [a*u for (a,u) in zip(asₛ₁, usₛ)] -plot!(polygon(psₛₗ)...) -@show 263 -euclid_dist(x; ps=psₛₗ) = sum(norm(x-p) for p in ps) -euclid_dist(x,y; ps=psₛₗ) = euclid_dist([x,y]; ps=ps) -@show 264 -#| hold: true -xs = range(-1.5, 1.5, length=100) -ys = range(-3, 1.0, length=100) - -p = plot(polygon(psₛₗ)..., linewidth=3, legend=false) -scatter!(p, unzip(psₛₗ)..., markersize=3) -contour!(p, xs, ys, euclid_dist) - -# add some gradients along boundary -li(t, p1, p2) = p1 + t*(p2-p1) # t in [0,1] -for t in range(1/100, 1/2, length=3) - pt = li(t, psₛₗ[2], psₛₗ[3]) - arrow!(pt, ForwardDiff.gradient(euclid_dist, pt)) - pt = li(t, psₛₗ[2], psₛₗ[1]) - arrow!(pt, ForwardDiff.gradient(euclid_dist, pt)) -end - -p -@show 266 -#| hold : true -li(t, p1, p2) = p1 + t*(p2-p1) -p = plot(legend=false) -for i in 1:2, j in (i+1):3 - plot!(p, t -> euclid_dist(li(t, psₛₗ[i], psₛₗ[j]); ps=psₛₗ), 0, 1) -end -p -@show 280 -@syms xₗₛ[1:3] yₗₛ[1:3] α β -li(x, alpha, beta) = alpha + beta * x -d₂(alpha, beta) = sum((y - li(x, alpha, beta))^2 for (y,x) in zip(yₗₛ, xₗₛ)) -d₂(α, β) -@show 282 -grad_d₂ = diff.(d₂(α, β), [α, β]) -@show 283 -outₗₛ = solve(grad_d₂, [α, β]) -@show 285 -subs(outₗₛ[β], sum(xₗₛ) => 0) -@show 292 -[k => subs(v, xₗₛ[1]=>1, yₗₛ[1]=>1, xₗₛ[2]=>2, yₗₛ[2]=>3, - xₗₛ[3]=>5, yₗₛ[3]=>8) for (k,v) in outₗₛ] -@show 302 -f₂(x,y) = -exp(-((x-1)^2 + 2(y-1/2)^2)) -f₂(x) = f₂(x...) - -xs₂ = [[0.0, 0.0]] # we store a vector -gammas₂ = [1.0] - -for n in 1:5 - xn = xs₂[end] - gamma₀ = gammas₂[end] - xn1 = xn - gamma₀ * gradient(f₂)(xn) - dx, dy = xn1 - xn, gradient(f₂)(xn1) - gradient(f₂)(xn) - gamman1 = abs( (dx ⋅ dy) / (dy ⋅ dy) ) - - push!(xs₂, xn1) - push!(gammas₂, gamman1) -end - -[(x, f₂(x)) for x in xs₂] -@show 304 -#| hold: true -function surface_contour(xs, ys, f; offset=0) - p = surface(xs, ys, f, legend=false, fillalpha=0.5) - - ## we add to the graphic p, then plot - zs = [f(x,y) for x in xs, y in ys] # reverse order for use with Contour package - for cl in levels(contours(xs, ys, zs)) - lvl = level(cl) # the z-value of this contour level - for line in lines(cl) - _xs, _ys = coordinates(line) # coordinates of this line segment - _zs = offset * _xs - plot!(p, _xs, _ys, _zs, alpha=0.5) # add curve on x-y plane - end - end - p -end - - -offset = 0 -us = vs = range(-1, 2, length=100) -surface_contour(us, vs, f₂, offset=offset) -pts = [[pt..., offset] for pt in xs₂] -scatter3d!(unzip(pts)...) -plot!(unzip(pts)..., linewidth=3) -@show 314 -function peaks(x, y) - z = 3 * (1 - x)^2 * exp(-x^2 - (y + 1)^2) - z += -10 * (x / 5 - x^3 - y^5) * exp(-x^2 - y^2) - z += -1/3 * exp(-(x+1)^2 - y^2) - return z -end -peaks(v) = peaks(v...) -@show 315 -#| hold: true -xs = range(-3, stop=3, length=100) -ys = range(-2, stop=2, length=100) -Ps = surface(xs, ys, peaks, legend=false) -Pc = contour(xs, ys, peaks, legend=false) -plot(Ps, Pc, layout=2) # combine plots -@show 319 -function newton_stepₚ(f, x) - M = ForwardDiff.hessian(f, x) - b = ForwardDiff.gradient(f, x) - x - M \ b -end -@show 321 -xₚ = [0, 1.5] -xₚ = newton_stepₚ(peaks, xₚ) -xₚ = newton_stepₚ(peaks, xₚ) -xₚ = newton_stepₚ(peaks, xₚ) -xₚ, ForwardDiff.gradient(peaks, xₚ) -@show 323 -Hₚ = ForwardDiff.hessian(peaks, xₚ) -@show 325 -#| hold: true -fxx = Hₚ[1,1] -d = det(Hₚ) -fxx, d -@show 335 -#| hold: true -g(x,y) = x^2 + 2y^2 -1 -g(v) = g(v...) - -xs = range(-3, 3, length=100) -ys = range(-1, 4, length=100) - -p = plot(aspect_ratio=:equal, legend=false) -contour!(xs, ys, g, levels=[0]) - -gi(x) = sqrt(1/2*(1-x^2)) # solve for y in terms of x -pts = [[x, gi(x)] for x in (-3/4, -1/4, 1/4, 3/4)] - -for pt in pts - arrow!(pt, ForwardDiff.gradient(g, pt) ) -end - -p -@show 338 -#| hold: true -#| echo: false -r(t) = [cos(t), sin(t)/2] -plot_parametric(pi/12..pi/3, r, legend=false, aspect_ratio=true, linewidth=3) -T(t) = -r'(t) / norm(r'(t)) -No(t) = T'(t) / norm(T'(t)) -t = pi/4 -lambda=1/10 -scatter!(unzip([r(t)])...) -arrow!(r(t), T(t)*lambda) -arrow!(r(t), No(t)* lambda) - -f(x,y)= x^2 + y^2 -f(v) = f(v...) -arrow!(r(t), lambda*ForwardDiff.gradient(f, r(t))) - -xs = range(0.5,1, length=100) -ys = range(0.1, 0.5, length=100) -contour!(xs, ys, f) -@show 344 -#| hold: true -#| echo: false -r(t) = [cos(t), sin(t)/2] -plot_parametric(-pi/6..pi/6,r, legend=false, aspect_ratio=true, linewidth=3) -T(t) = -r'(t) / norm(r'(t)) -No(t) = T'(t) / norm(T'(t)) -t = 0 -lambda=1/10 -scatter!(unzip([r(t)])...) -arrow!(r(t), T(t)*lambda) -arrow!(r(t), No(t)* lambda) - -f(x,y)= x^2 + y^2 -f(v) = f(v...) -arrow!(r(t), lambda*ForwardDiff.gradient(f, r(t))) - -xs = range(0.5,1.5, length=100) -ys = range(-0.5, 0.5, length=100) -contour!(xs, ys, f, levels = [.7, .85, 1, 1.15, 1.3]) -@show 381 -@syms lambda -fₗₐ(x, y) = x^2 - y^2 -gₗₐ(x, y) = x^2 + y^2 -Lₗₐ(x, y, lambda) = fₗₐ(x,y) - lambda * (gₗₐ(x,y) - 1) -dsₗₐ = solve(diff.(Lₗₐ(x, y, lambda), [x, y, lambda])) -@show 383 -[fₗₐ(d[x], d[y]) for d in dsₗₐ] -@show 432 -#| hold: true -@syms y y′ λ C -ex = Eq(-λ*y′^2/sqrt(1 + y′^2) + λ*sqrt(1 + y′^2), y - C) -Δ = sqrt(1 + y′^2) / (y - C) -ex1 = Eq(simplify(ex.lhs()*Δ), simplify(ex.rhs() * Δ)) -ex2 = Eq(ex1.lhs()^2 - 1, simplify(ex1.rhs()^2) - 1) -@show 457 -@syms z lambda1 lambda2 -g1(x, y, z) = x^2 + y^2 - z^2 -g2(x, y, z) = x - 2z - 3 -fₘ(x,y,z)= x^2 + y^2 + z^2 -Lₘ(x,y,z,lambda1, lambda2) = fₘ(x,y,z) - lambda1*(g1(x,y,z) - 0) - lambda2*(g2(x,y,z) - 0) - -∇Lₘ = diff.(Lₘ(x,y,z,lambda1, lambda2), [x, y, z,lambda1, lambda2]) -@show 459 -solve(subs.(∇Lₘ, lambda1 .=> 1)) -@show 461 -outₘ = solve(subs.(∇Lₘ, y .=> 0)) -@show 463 -[fₘ(d[x], 0, d[z]) for d in outₘ] -@show 498 -struct MultiIndex - alpha::Vector{Int} - end -Base.show(io::IO, α::MultiIndex) = println(io, "α = ($(join(α.alpha, ", ")))") - -## |α| = α_1 + ... + α_m -Base.length(α::MultiIndex) = sum(α.alpha) - -## factorial(α) computes α! -Base.factorial(α::MultiIndex) = prod(factorial(Sym(a)) for a in α.alpha) - -## x^α = x_1^α_1 * x_2^α^2 * ... * x_n^α_n -import Base: ^ -^(x, α::MultiIndex) = prod(u^a for (u,a) in zip(x, α.alpha)) - -## ∂^α(ex) = ∂_1^α_1 ∘ ∂_2^α_2 ∘ ... ∘ ∂_n^α_n (ex) -partial(ex::SymPy.SymbolicObject, α::MultiIndex, vars=free_symbols(ex)) = diff(ex, zip(vars, α.alpha)...) -@show 499 -@syms w -alpha = MultiIndex([1,2,1,3]) -length(alpha) # 1 + 2 + 1 + 3=7 -[1,2,3,4]^alpha -exₜ = x^3 * cos(w*y*z) -partial(exₜ, alpha, [w,x,y,z]) -@show 501 -struct MultiIndices - n::Int - k::Int -end - -function Base.length(as::MultiIndices) - n,k = as.n, as.k - n == 1 && return 1 - sum(length(MultiIndices(n-1, j)) for j in 0:k) # recursively identify length -end - -function Base.iterate(alphas::MultiIndices) - k, n = alphas.k, alphas.n - n == 1 && return ([k],(0, MultiIndices(0,0), nothing)) - - m = zeros(Int, n) - m[1] = k - betas = MultiIndices(n-1, 0) - stb = iterate(betas) - st = (k, MultiIndices(n-1, 0), stb) - return (m, st) -end - -function Base.iterate(alphas::MultiIndices, st) - - st == nothing && return nothing - k,n = alphas.k, alphas.n - k == 0 && return nothing - n == 1 && return nothing - - # can we iterate the next on - bk, bs, stb = st - - if stb==nothing - bk = bk-1 - bk < 0 && return nothing - bs = MultiIndices(bs.n, bs.k+1) - val, stb = iterate(bs) - return (vcat(bk,val), (bk, bs, stb)) - end - - resp = iterate(bs, stb) - if resp == nothing - bk = bk-1 - bk < 0 && return nothing - bs = MultiIndices(bs.n, bs.k+1) - val, stb = iterate(bs) - return (vcat(bk, val), (bk, bs, stb)) - end - - val, stb = resp - return (vcat(bk, val), (bk, bs, stb)) - -end -@show 503 -collect(MultiIndices(2, 3)) -@show 505 -union((collect(MultiIndices(2, i)) for i in 0:3)...) -@show 507 -k = 4 -length(MultiIndices(3, k+1)) -@show 509 -#| hold: true -@syms 𝐅() a[1:3] dx[1:3] - -sum(partial(𝐅(a...), α, a) / factorial(α) * dx^α for k in 0:3 for α in MultiIndex.(MultiIndices(3, k))) # 3rd order -@show 513 -#| hold: true -#| echo: false -f(x,y) = sqrt(x + y) -f(v) = f(v...) -pt = [2,2] -dxdy = [.1, .2] -val = f(pt) + dot(ForwardDiff.gradient(f, pt), dxdy) -numericq(val) -@show 516 -#| hold: true -#| echo: false -f(x,y,z) = x*y + y*z + z*x -f(v) = f(v...) -pt = [1,1,1] -dx = [0.1, 0.0, -0.1] -val = f(pt) + ∇(f)(pt) ⋅ dx -numericq(val) -@show 519 -#| hold: true -#| echo: false -f(x,y,z) = x*y + y*z + z*x - 8 -f(v) = f(v...) -pt = [1,1,1] -n = ∇(f)(pt) -d = dot(n, pt) -choices = [ - raw"`` x + y + z = 3``", - raw"`` 2x + y - 2z = 1``", - raw"`` x + 2y + 3z = 6``" -] -answ = 1 -radioq(choices, answ) -@show 523 -#| hold: true -#| echo: false -choices = [ - raw"`` \langle 2xy + y^2 + y, 2xy + x^2 + x\rangle``", - raw"`` y^2 + y, x^2 + x``", - raw"`` \langle 2y + y^2, 2x + x^2``" -] -answ = 1 -radioq(choices, answ) -@show 527 -#| hold: true -#| echo: false -yesnoq(true) -@show 529 -#| hold: true -#| echo: false -f(x,y) = x*y + x*y^2 + x^2 * y -f(v) = f(v...) -val = det(ForwardDiff.hessian(f, [-1/3, -1/3])) -numericq(val) -@show 531 -#| hold: true -#| echo: false -choices = [ - L"The function $f$ has a local minimum, as $f_{xx} > 0$ and $d >0$", - L"The function $f$ has a local maximum, as $f_{xx} < 0$ and $d >0$", - L"The function $f$ has a saddle point, as $d < 0$", - L"Nothing can be said, as $d=0$" -] -answ = 2 -radioq(choices, answ, keep_order=true) -@show 535 -#| hold: true -#| results: "hidden" -f(x,y) = x + 2x^2 + x^3 + y + 2x*y + y^2 -@syms x::real y::real -gradf = gradient(f(x,y), [x,y]) -@show 536 -#| hold: true -#| echo: false -yesnoq(true) -@show 538 -#| hold: true -#| results: "hidden" -f(x,y) = x + 2x^2 + x^3 + y + 2x*y + y^2 -@syms x::real y::real -gradf = gradient(f(x,y), [x,y]) - -solve(gradf, [x,y]) -@show 539 -#| hold: true -#| echo: false -numericq(2) -@show 541 -#| hold: true -f(x,y) = x + 2x^2 + x^3 + y + 2x*y + y^2 -@syms x::real y::real -gradf = gradient(f(x,y), [x,y]) - -sympy.hessian(f(x,y), [x,y]) -@show 543 -#| hold: true -#| echo: false -choices = [ - L"The function $f$ has a local minimum, as $f_{xx} > 0$ and $d >0$", - L"The function $f$ has a local maximum, as $f_{xx} < 0$ and $d >0$", - L"The function $f$ has a saddle point, as $d < 0$", - L"Nothing can be said, as $d=0$", - L"The test does not apply, as $\nabla{f}$ is not $0$ at this point." -] -answ = 3 -radioq(choices, answ, keep_order=true) -@show 545 -#| hold: true -#| echo: false -choices = [ - L"The function $f$ has a local minimum, as $f_{xx} > 0$ and $d >0$", - L"The function $f$ has a local maximum, as $f_{xx} < 0$ and $d >0$", - L"The function $f$ has a saddle point, as $d < 0$", - L"Nothing can be said, as $d=0$", - L"The test does not apply, as $\nabla{f}$ is not $0$ at this point." -] -answ = 1 -radioq(choices, answ, keep_order=true) -@show 547 -#| hold: true -#| echo: false -choices = [ - L"The function $f$ has a local minimum, as $f_{xx} > 0$ and $d >0$", - L"The function $f$ has a local maximum, as $f_{xx} < 0$ and $d >0$", - L"The function $f$ has a saddle point, as $d < 0$", - L"Nothing can be said, as $d=0$", - L"The test does not apply, as $\nabla{f}$ is not $0$ at this point." -] -answ = 5 -radioq(choices, answ, keep_order=true) -@show 553 -#| hold: true -#| echo: false -yesnoq(true) -@show 557 -#| hold: true -#| echo: false -yesnoq(false) -@show 559 -#| hold: true -#| echo: false -choices =[ - "It is the determinant of the Hessian", - L"It isn't, $b^2-4ac$ is from the quadratic formula" -] -answ = 1 -radioq(choices, answ) -@show 561 -#| hold: true -#| echo: false -choices = [ - L"That $a>0$ and $4ac-b^2 > 0$", - L"That $a<0$ and $4ac-b^2 > 0$", - L"That $4ac-b^2 < 0$" -] -answ = 2 -radioq(choices, answ, keep_order=true) -@show 563 -#| hold: true -#| echo: false -choices = [ - L"That $a>0$ and $4ac-b^2 > 0$", - L"That $a<0$ and $4ac-b^2 > 0$", - L"That $4ac-b^2 < 0$" -] -answ = 3 -radioq(choices, answ, keep_order=true) -@show 569 -#| hold: true -#| echo: false -yesnoq(true) -@show 571 -#| echo: false -choices = [ - raw"`` \langle 2x, 2y\rangle``", - raw"`` \langle 2x, y^2\rangle``", - raw"`` \langle x^2, 2y \rangle``" -] -answ = 1 -radioq(choices, answ) -@show 573 -f(x,y) = exp(-x^2-y^2) * (2x^2 + y^2) -f(v) = f(v...) -r(t) = sqrt(3)*[cos(t), sin(t)] -rat(x) = abs(x[1]/x[2]) - 1 -fn = rat ∘ ∇(f) ∘ r -ts = fzeros(fn, 0, 2pi) -@show 575 -#| eval: false -#| echo: false -f(x,y) = exp(-x^2-y^2) * (2x^2 + y^2) -r(t) = sqrt(3)*[cos(t), sin(t)] -rat(x) = abs(x[1]/x[2]) - 1 -fn = rat ∘ ∇(splat(f)) ∘ r -ts = fzeros(fn, 0, 2pi) - -val = maximum((splat(u)∘r).(ts)) -numericq(val) diff --git a/quarto/differentiable_vector_calculus/test.qmd b/quarto/differentiable_vector_calculus/test.qmd deleted file mode 100644 index 37ab0eb..0000000 --- a/quarto/differentiable_vector_calculus/test.qmd +++ /dev/null @@ -1,98 +0,0 @@ -# Applications with scalar functions - - -{{< include ../_common_code.qmd >}} - -This section uses these add-on packages: - - -```{julia} -using CalculusWithJulia -using Plots -plotly() -using SymPy -using Roots -``` - -##### Example - - -Consider the function $f(x,y) = x^2 + 3y^2 -x$ over the region $x^2 + y^2 \leq 1$. This is a continuous function over a closed set, so will have both an absolute maximum and minimum. Find these from an investigation of the critical points and the boundary points. - - -The gradient is easily found: $\nabla{f} = \langle 2x - 1, 6y \rangle$, and is $\vec{0}$ only at $\vec{a} = \langle 1/2, 0 \rangle$. The Hessian is: - - -$$ -H = -\begin{bmatrix} -2 & 0\\ -0 & 6 -\end{bmatrix}. -$$ - -At $\vec{a}$ this has positive determinant and $f_{xx} > 0$, so $\vec{a}$ corresponds to a *local* minimum with values $f(\vec{a}) = (1/2)^2 + 3(0) - 1/2 = -1/4$. The absolute maximum and minimum may occur here (well, not the maximum) or on the boundary, so that must be considered. In this case we can easily parameterize the boundary and turn this into the univariate case: - - -```{julia} -fₗ(x,y) = x^2 + 2y^2 - x -gammaₗ(t) = [cos(t), sin(t)] # traces out x^2 + y^2 = 1 over [0, 2pi] -gₗ = splat(fₗ) ∘ gammaₗ - -cpsₗ = find_zeros(gₗ', 0, 2pi) # critical points of g -append!(cpsₗ, [0, 2pi]) -unique!(cpsₗ) -gₗ.(cpsₗ) -``` - -We see that maximum value is `2.25` and that the interior point, $\vec{a}$, will be where the minimum value occurs. To see exactly where the maximum occurs, we look at the values of gamma: - -```{julia} -inds = [2,4] -cpsₗ[inds] -``` - -These are multiples of $\pi$: - - -```{julia} -cpsₗ[inds]/pi -``` - -So we have the maximum occurs at the angles $2\pi/3$ and $4\pi/3$. Here we visualize, using a hacky trick of assigning `NaN` values to the function to avoid plotting outside the circle: - - -```{julia} -hₗ(x,y) = fₗ(x,y) * (x^2 + y^2 <= 1 ? 1 : NaN) -``` - -```{julia} -gr() -``` - -```{julia} -#| hold: true -xs = ys = range(-1,1, length=100) -plt = surface(xs, ys, hₗ) - -ts = cpsₗ # 2pi/3 and 4pi/3 by above -xs, ys = cos.(ts), sin.(ts) -zs = fₗ.(xs, ys) -scatter3d!(xs, ys, zs) -#tuple.(xs, ys, zs) -#plot!(plt, tuple.(xs, ys, zs); linetype=:scatter) -#plt -``` - -A contour plot also shows that some---and only one---extrema happens on the interior: - - diff --git a/quarto/integrals/area_between_curves.qmd b/quarto/integrals/area_between_curves.qmd index 490b87c..34017de 100644 --- a/quarto/integrals/area_between_curves.qmd +++ b/quarto/integrals/area_between_curves.qmd @@ -718,7 +718,7 @@ let O = (0, 0) x1, y1 = P = (2, 2) x2, y2 = Q = (1, 3) - plt = plot(; legend=false, aspect_ratio=:equal, framestyle=:orgin) + plt = plot(; legend=false, aspect_ratio=:equal, framestyle=:origin) plot!(plt, [O,P,Q,O]; line=(1, :black)) scatter!(plt, [O, P, Q]; marker=(5, :black)) annotate!(plt, [ diff --git a/quarto/integrals/center_of_mass.qmd b/quarto/integrals/center_of_mass.qmd index 9a1d058..e1f579a 100644 --- a/quarto/integrals/center_of_mass.qmd +++ b/quarto/integrals/center_of_mass.qmd @@ -227,10 +227,10 @@ The bottom integral is just the area (or total mass if the $\rho$ were not cance ##### Example -Find the center of mass formed by the intersection of the parabolas $y=1 - x^2$ and $y=(x-1)^2 - 2$. @fig-center-of-mass-of-two-parabola-1-minus-xsquared-and-x-minus-1-sqared-minus-2 shows that for the $x$ direction, it is close to $1/2$. +Find the center of mass formed by the intersection of the parabolas $y=1 - x^2$ and $y=(x-1)^2 - 2$. @fig-center-of-mass-of-two-parabola-1-minus-xsquared-and-x-minus-1-squared-minus-2 shows that for the $x$ direction, it is close to $1/2$. -::: {#fig-center-of-mass-of-two-parabola-1-minus-xsquared-and-x-minus-1-sqared-minus-2} +::: {#fig-center-of-mass-of-two-parabola-1-minus-xsquared-and-x-minus-1-squared-minus-2} ```{julia} #| echo: false f1(x) = 1 - x^2 diff --git a/quarto/integrals/substitution.qmd b/quarto/integrals/substitution.qmd index 7e97f85..5924a4a 100644 --- a/quarto/integrals/substitution.qmd +++ b/quarto/integrals/substitution.qmd @@ -22,7 +22,7 @@ using SymPy The technique of $u$-[substitution](https://en.wikipedia.org/wiki/Integration_by_substitution) is derived from reversing the chain rule: $[f(g(x))]' = f'(g(x)) g'(x)$. -::: {.definition title="Subsitution"} +::: {.definition title="Substitution"} Suppose that $g$ is continuous and $u(x)$ is differentiable with $u'(x)$ being Riemann integrable. Then both these integrals are defined and are equal: diff --git a/quarto/integrals/surface_area.qmd b/quarto/integrals/surface_area.qmd index 321fd7d..2d80bdd 100644 --- a/quarto/integrals/surface_area.qmd +++ b/quarto/integrals/surface_area.qmd @@ -668,7 +668,7 @@ f(t) = 2(1 + cos(t)) * sin(t) plot(g, f, 0, 1pi) ``` -Paremeterized curve to rotate about $x$ axis +Parameterized curve to rotate about $x$ axis ::: The integrand simplifies to $8\sqrt{2}\pi \sin(t) (1 + \cos(t))^{3/2}$. This lends itself to $u$-substitution with $u=\cos(t)$. diff --git a/quarto/precalc/julia_overview.qmd b/quarto/precalc/julia_overview.qmd index 4fa6389..7eed97b 100644 --- a/quarto/precalc/julia_overview.qmd +++ b/quarto/precalc/julia_overview.qmd @@ -626,7 +626,7 @@ xs = range(0, 2pi, length=251) ys = [sin(2x) + sin(3x) + sin(4x) for x in xs] plot(xs, ys) ``` -Plot of $f(x) = \sin(2x) + \sin(3x) + \sin(4x)$ over $[0, 2\pi]$ made by constructing vectors `xs` , `ys` holding $x$ and $y$ coordiinates of points to include +Plot of $f(x) = \sin(2x) + \sin(3x) + \sin(4x)$ over $[0, 2\pi]$ made by constructing vectors `xs` , `ys` holding $x$ and $y$ coordinates of points to include ::: There are different plotting interfaces. Though not shown, all of these `plot` commands produce a plot of `f`, though with minor differences: diff --git a/quarto/precalc/transformations.qmd b/quarto/precalc/transformations.qmd index c70df7e..a7a8772 100644 --- a/quarto/precalc/transformations.qmd +++ b/quarto/precalc/transformations.qmd @@ -89,7 +89,7 @@ Starting with two functions and composing them requires nothing more than a soli ::: {.callout-note} ## Infix operator -Composition of two functions does have an infix operator, `∘`, entered as `\circ[tab]`. This mirrors the mathematical usage of this syntax, though the order of operations are such that calling the composed function on a value requires an extra set of parentheses: `(f∘g)(x)`, as the expresssion `f∘g(x)` evaluates `g(x)` before the composition. +Composition of two functions does have an infix operator, `∘`, entered as `\circ[tab]`. This mirrors the mathematical usage of this syntax, though the order of operations are such that calling the composed function on a value requires an extra set of parentheses: `(f∘g)(x)`, as the expression `f∘g(x)` evaluates `g(x)` before the composition. ::: diff --git a/quarto/precalc/trig_functions.qmd b/quarto/precalc/trig_functions.qmd index 16de81e..7c3693c 100644 --- a/quarto/precalc/trig_functions.qmd +++ b/quarto/precalc/trig_functions.qmd @@ -1173,7 +1173,7 @@ The sine function is an *odd* function. #| echo: false choices = ["odd", "even", "neither"] answer = 1 -explanation = "subsitute `-x` into the exponential formula to see" +explanation = "substitute `-x` into the exponential formula to see" buttonq(choices, answer; explanation) ``` @@ -1185,7 +1185,7 @@ buttonq(choices, answer; explanation) #| echo: false choices = ["odd", "even", "neither"] answer = 2 -explanation = L"The value of $\cosh(-x)$ is the $y$ position of the point $(x,y)$ refelected through the $y$ axis, so is unchanged." +explanation = L"The value of $\cosh(-x)$ is the $y$ position of the point $(x,y)$ reflected through the $y$ axis, so is unchanged." buttonq(choices, answer; explanation) ``` From 5d99f9a672a2f09e731e50f42710847848a8bdbf Mon Sep 17 00:00:00 2001 From: jverzani Date: Tue, 11 Aug 2026 17:48:39 -0400 Subject: [PATCH 6/7] typos too --- _typos.toml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/_typos.toml b/_typos.toml index 9381bfc..8efb4b2 100644 --- a/_typos.toml +++ b/_typos.toml @@ -17,3 +17,13 @@ infiniment = "infiniment" typ = "typ" Comput = "Comput" + +Liousville = "Liousville" + +numer = "numer" +denom = "denom" +desolve = "desolve" +eqal = "eqal" + +adn = "adn" +optin = "optin" From 265fe8509cbec406cc9a89ea944617a019def34a Mon Sep 17 00:00:00 2001 From: jverzani Date: Tue, 11 Aug 2026 17:50:01 -0400 Subject: [PATCH 7/7] no fight --- _typos.toml | 2 +- quarto/basics/variables.qmd | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/_typos.toml b/_typos.toml index 8efb4b2..55994ba 100644 --- a/_typos.toml +++ b/_typos.toml @@ -23,7 +23,7 @@ Liousville = "Liousville" numer = "numer" denom = "denom" desolve = "desolve" -eqal = "eqal" +egal = "egal" adn = "adn" optin = "optin" diff --git a/quarto/basics/variables.qmd b/quarto/basics/variables.qmd index dba6085..feea32a 100644 --- a/quarto/basics/variables.qmd +++ b/quarto/basics/variables.qmd @@ -221,7 +221,7 @@ Repeating this last line will generate new values of `x` based on the previous o ::: {.callout-note} ## Use of = -The distinction between ``=`` versus `=` is important and one area where common math notation and common computer notation diverge. The mathematical ``=`` indicates *equality*, and is often used with equations and also for assignment. Later, when symbolic math is introduced, the `~` symbol will be used to indicate an equation, though this is by convention and not part of base `Julia`. The computer syntax use of `=` is for *assignment* and *re-assignment*. Equality is tested with `==` and identicalness (or egal) with `===`. +The distinction between ``=`` versus `=` is important and one area where common math notation and common computer notation diverge. The mathematical ``=`` indicates *equality*, and is often used with equations and also for assignment. Later, when symbolic math is introduced, the `~` symbol will be used to indicate an equation, though this is by convention and not part of base `Julia`. The computer syntax use of `=` is for *assignment* and *re-assignment*. Equality is tested with `==` and identicalness with `===`. :::