Dual Numbers and Differentiation. Part 3.

Previously we looked at using Dual numbers get the value of the first derivative of a function. As useful as this is there is more potential if we can also obtain the second derivative. My initial, naive, approach to this was to extend the idea of a Dual to that of a Triple like this. data Triple a = T a a a deriving (Show). Creating Triple somehow seemed ‘wrong’, or if not wrong then certainly clumsy as can be seen in some of the code below.

Note how messy the code is! It’s the result of apply the quotient rule to the result of applying the quotient rule. At the very least it told me that I didn’t want to have to take this approach and have to write
instance (Fractional a, Floating a) => Floating (Triple a a a) !

The approach taken in ‘Functional Differentiation of Computer Programs‘ involves a recursive definition and what are eloquently termed ‘infinite towers of derivatives’ which exploits the lazy nature of Haskell to give not just first and second derivates but a potentially infinite list of increasingly higher order derivatives. Of course the computation is not ‘free’ but with lazy Haskell you only pay for what you take and you pay when you take it. Here’s the rather innocuous looking start point.

I must admit that I found parts of the paper quite heavy and it took some effort to grasp it but the key thing to remember is that we are now dealing with a recursive data type and so the functions to express the derivatives will need to reflect this recursive structure. So, in the same way that we created instances of Num, Fractional and Floating for Dual we will do the same for Diff. Here’s one I made earlier!

In each case we’ve just implemented the required functions along with a helper function, dlift.
Looking at the cycle function first we see, by example.

So seeing how cycle works and knowing that the differential of sin is cos and of cos is -sin then we can see that, for example, cycle [sin, cos, negate . sin, negate . cos] keeps providing the repeated differential of sin. With this intuition we can see that the dlift helper function takes each, in turn, from cycle and constructs a Diff with that and a recursive call to itself.

If, for the moment, we restrict ourselves to at most the second derivative then a simple implementation of show need only go two levels deep. Like this

And armed with this show function we can do some simple tests to check we get the expected values for f, f’ and f”.
Let’s try 1/x^2 at x=2

That looks good. How about x^3 at x=3.5?

The next batch of examples are all re-runs of the functions shown in the first post but with, of course, the second derivative being calculated.

Rather than rely on a show method that’s limited in its depth we can write

which can be used to get up to the nth derivative. As we are focussing on going to just the second derivative we can then first write

and then create

which will allow us to get the value along with the first and second derivatives using a clearly named function.

I think that’s enough for the moment. In the next post we’ll look at calculating second order partial derivatives and maybe an application of them. As usual all the code is in Github.

Thanks for reading!

Leave a Reply

Your email address will not be published. Required fields are marked *

ˆ Back To Top