11 lines
10 KiB
JSON
11 lines
10 KiB
JSON
{
|
||
"hash": "dd0d8a6cbd80e27a39fcd07ff16b09de",
|
||
"result": {
|
||
"markdown": "# Symbolic derivatives\n\n\n\nThis section uses this add-on package:\n\n``` {.julia .cell-code}\nusing TermInterface\n```\n\n\n\n\n---\n\n\nThe ability to breakdown an expression into operations and their arguments is necessary when trying to apply the differentiation rules. Such rules are applied from the outside in. Identifying the proper \"outside\" function is usually most of the battle when finding derivatives.\n\n\nIn the following example, we provide a sketch of a framework to differentiate expressions by a chosen symbol to illustrate how the outer function drives the task of differentiation.\n\n\nThe `Symbolics` package provides native symbolic manipulation abilities for `Julia`, similar to `SymPy`, though without the dependence on `Python`. The `TermInterface` package, used by `Symbolics`, provides a generic interface for expression manipulation for this package that *also* is implemented for `Julia`'s expressions and symbols.\n\n\nAn expression is an unevaluated portion of code that for our purposes below contains other expressions, symbols, and numeric literals. They are held in the `Expr` type. A symbol, such as `:x`, is distinct from a string (e.g. `\"x\"`) and is useful to the programmer to distinguish between the contents a variable points to from the name of the variable. Symbols are fundamental to metaprogramming in `Julia`. An expression is a specification of some set of statements to execute. A numeric literal is just a number.\n\n\nThe three main functions from `TermInterface` we leverage are `istree`, `operation`, and `arguments`. The `operation` function returns the \"outside\" function of an expression. For example:\n\n::: {.cell execution_count=4}\n``` {.julia .cell-code}\noperation(:(sin(x)))\n```\n\n::: {.cell-output .cell-output-display execution_count=5}\n```\n:sin\n```\n:::\n:::\n\n\nWe see the `sin` function, referred to by a symbol (`:sin`). The `:(...)` above *quotes* the argument, and does not evaluate it, hence `x` need not be defined above. (The `:` notation is used to create both symbols and expressions.)\n\n\nThe arguments are the terms that the outside function is called on. For our purposes there may be $1$ (*unary*), $2$ (*binary*), or more than $2$ (*nary*) arguments. (We ignore zero-argument functions.) For example:\n\n::: {.cell execution_count=5}\n``` {.julia .cell-code}\narguments(:(-x)), arguments(:(pi^2)), arguments(:(1 + x + x^2))\n```\n\n::: {.cell-output .cell-output-display execution_count=6}\n```\n(Any[:x], Any[:pi, 2], Any[1, :x, :(x ^ 2)])\n```\n:::\n:::\n\n\n(The last one may be surprising, but all three arguments are passed to the `+` function.)\n\n\nHere we define a function to decide the *arity* of an expression based on the number of arguments it is called with:\n\n::: {.cell execution_count=6}\n``` {.julia .cell-code}\nfunction arity(ex)\n\t\tn = length(arguments(ex))\n\t\tn == 1 ? Val(:unary) :\n\t\tn == 2 ? Val(:binary) : Val(:nary)\nend\n```\n\n::: {.cell-output .cell-output-display execution_count=7}\n```\narity (generic function with 1 method)\n```\n:::\n:::\n\n\nDifferentiation must distinguish between expressions, variables, and numbers. Mathematically expressions have an \"outer\" function, whereas variables and numbers can be directly differentiated. The `istree` function in `TermInterface` returns `true` when passed an expression, and `false` when passed a symbol or numeric literal. The latter two may be distinguished by `isa(..., Symbol)`.\n\n\nHere we create a function, `D`, that when it encounters an expression it *dispatches* to a specific method of `D` based on the outer operation and arity, otherwise if it encounters a symbol or a numeric literal it does the differentiation:\n\n::: {.cell execution_count=7}\n``` {.julia .cell-code}\nfunction D(ex, var=:x)\n\tif istree(ex)\n\t\top, args = operation(ex), arguments(ex)\n\t\tD(Val(op), arity(ex), args, var)\n\telseif isa(ex, Symbol) && ex == :x\n\t\t1\n\telse\n\t\t0\n\tend\nend\n```\n\n::: {.cell-output .cell-output-display execution_count=8}\n```\nD (generic function with 2 methods)\n```\n:::\n:::\n\n\nNow to develop methods for `D` for different \"outside\" functions and arities.\n\n\nAddition can be unary (`:(+x)` is a valid quoting, even if it might simplify to the symbol `:x` when evaluated), *binary*, or *nary*. Here we implement the *sum rule*:\n\n::: {.cell execution_count=8}\n``` {.julia .cell-code}\nD(::Val{:+}, ::Val{:unary}, args, var) = D(first(args), var)\n\nfunction D(::Val{:+}, ::Val{:binary}, args, var)\n\ta′, b′ = D.(args, var)\n\t:($a′ + $b′)\nend\n\nfunction D(::Val{:+}, ::Val{:nary}, args, var)\n\ta′s = D.(args, var)\n\t:(+($a′s...))\nend\n```\n\n::: {.cell-output .cell-output-display execution_count=9}\n```\nD (generic function with 5 methods)\n```\n:::\n:::\n\n\nThe `args` are always held in a container, so the unary method must pull out the first one. The binary case should read as: apply `D` to each of the two arguments, and then create a quoted expression containing the sum of the results. The dollar signs interpolate into the quoting. (The \"primes\" are unicode notation achieved through `\\prime[tab]` and not operations.) The *nary* case does something similar, only uses splatting to produce the sum.\n\n\nSubtraction must also be implemented in a similar manner, but not for the *nary* case:\n\n::: {.cell execution_count=9}\n``` {.julia .cell-code}\nfunction D(::Val{:-}, ::Val{:unary}, args, var)\n\ta′ = D(first(args), var)\n\t:(-$a′)\nend\nfunction D(::Val{:-}, ::Val{:binary}, args, var)\n\ta′, b′ = D.(args, var)\n\t:($a′ - $b′)\nend\n```\n\n::: {.cell-output .cell-output-display execution_count=10}\n```\nD (generic function with 7 methods)\n```\n:::\n:::\n\n\nThe *product rule* is similar to addition, in that $3$ cases are considered:\n\n::: {.cell execution_count=10}\n``` {.julia .cell-code}\nD(op::Val{:*}, ::Val{:unary}, args, var) = D(first(args), var)\n\nfunction D(::Val{:*}, ::Val{:binary}, args, var)\n a, b = args\n a′, b′ = D.(args, var)\n :($a′ * $b + $a * $b′)\nend\n\nfunction D(op::Val{:*}, ::Val{:nary}, args, var)\n a, bs... = args\n b = :(*($(bs...)))\n\ta′ = D(a, var)\n b′ = D(b, var)\n\t:($a′ * $b + $a * $b′)\nend\n```\n\n::: {.cell-output .cell-output-display execution_count=11}\n```\nD (generic function with 10 methods)\n```\n:::\n:::\n\n\nThe *nary* case above just peels off the first factor and then uses the binary product rule.\n\n\nDivision is only a binary operation, so here we have the *quotient rule*:\n\n::: {.cell execution_count=11}\n``` {.julia .cell-code}\nfunction D(::Val{:/}, ::Val{:binary}, args, var)\n\tu,v = args\n\tu′, v′ = D(u, var), D(v, var)\n\t:( ($u′*$v - $u*$v′)/$v^2 )\nend\n```\n\n::: {.cell-output .cell-output-display execution_count=12}\n```\nD (generic function with 11 methods)\n```\n:::\n:::\n\n\nPowers are handled a bit differently. The power rule would require checking if the exponent does not contain the variable of differentiation, exponential derivatives would require checking the base does not contain the variable of differentation. Trying to implement both would be tedious, so we use the fact that $x = \\exp(\\log(x))$ (for `x` in the domain of `log`, more care is necessary if `x` is negative) to differentiate:\n\n::: {.cell execution_count=12}\n``` {.julia .cell-code}\nfunction D(::Val{:^}, ::Val{:binary}, args, var)\n\ta, b = args\n D(:(exp($b*log($a))), var) # a > 0 assumed here\nend\n```\n\n::: {.cell-output .cell-output-display execution_count=13}\n```\nD (generic function with 12 methods)\n```\n:::\n:::\n\n\nThat leaves the task of defining a rule to differentiate both `exp` and `log`. We do so with *unary* definitions. In the following we also implement `sin` and `cos` rules:\n\n::: {.cell execution_count=13}\n``` {.julia .cell-code}\nfunction D(::Val{:exp}, ::Val{:unary}, args, var)\n\ta = first(args)\n\ta′ = D(a, var)\n\t:(exp($a) * $a′)\nend\n\nfunction D(::Val{:log}, ::Val{:unary}, args, var)\n\ta = first(args)\n\ta′ = D(a, var)\n\t:(1/$a * $a′)\nend\n\nfunction D(::Val{:sin}, ::Val{:unary}, args, var)\n\ta = first(args)\n\ta′ = D(a, var)\n\t:(cos($a) * $a′)\nend\n\nfunction D(::Val{:cos}, ::Val{:unary}, args, var)\n\ta = first(args)\n\ta′ = D(a, var)\n\t:(-sin($a) * $a′)\nend\n```\n\n::: {.cell-output .cell-output-display execution_count=14}\n```\nD (generic function with 16 methods)\n```\n:::\n:::\n\n\nThe pattern is similar for each. The `$a′` factor is needed due to the *chain rule*. The above illustrates the simple pattern necessary to add a derivative rule for a function. More could be, but for this example the above will suffice, as now the system is ready to be put to work.\n\n::: {.cell execution_count=14}\n``` {.julia .cell-code}\nex₁ = :(x + 2/x)\nD(ex₁, :x)\n```\n\n::: {.cell-output .cell-output-display execution_count=15}\n```\n:(1 + (0 * x - 2 * 1) / x ^ 2)\n```\n:::\n:::\n\n\nThe output does not simplify, so some work is needed to identify `1 - 2/x^2` as the answer.\n\n::: {.cell execution_count=15}\n``` {.julia .cell-code}\nex₂ = :( (x + sin(x))/sin(x))\nD(ex₂, :x)\n```\n\n::: {.cell-output .cell-output-display execution_count=16}\n```\n:(((1 + cos(x) * 1) * sin(x) - (x + sin(x)) * (cos(x) * 1)) / sin(x) ^ 2)\n```\n:::\n:::\n\n\nAgain, simplification is not performed.\n\n\nFinally, we have a second derivative taken below:\n\n::: {.cell execution_count=16}\n``` {.julia .cell-code}\nex₃ = :(sin(x) - x - x^3/6)\nD(D(ex₃, :x), :x)\n```\n\n::: {.cell-output .cell-output-display execution_count=17}\n```\n:((((-(sin(x)) * 1) * 1 + cos(x) * 0) - 0) - (((((exp(3 * log(x)) * (0 * log(x) + 3 * ((1 / x) * 1))) * (0 * log(x) + 3 * ((1 / x) * 1)) + exp(3 * log(x)) * ((0 * log(x) + 0 * ((1 / x) * 1)) + (0 * ((1 / x) * 1) + 3 * (((0 * x - 1 * 1) / x ^ 2) * 1 + (1 / x) * 0)))) * 6 + (exp(3 * log(x)) * (0 * log(x) + 3 * ((1 / x) * 1))) * 0) - ((exp(3 * log(x)) * (0 * log(x) + 3 * ((1 / x) * 1))) * 0 + x ^ 3 * 0)) * 6 ^ 2 - ((exp(3 * log(x)) * (0 * log(x) + 3 * ((1 / x) * 1))) * 6 - x ^ 3 * 0) * (exp(2 * log(6)) * (0 * log(6) + 2 * ((1 / 6) * 0)))) / (6 ^ 2) ^ 2)\n```\n:::\n:::\n\n\nThe length of the expression should lead to further appreciation for simplification steps taken when doing such a computation by hand.\n\n",
|
||
"supporting": [
|
||
"symbolic_derivatives_files"
|
||
],
|
||
"filters": [],
|
||
"includes": {}
|
||
}
|
||
} |