Why are we differentiating?
Flashback: The beginning
I remember my very first day of primary school like it was yesterday, it was a completely new experience. Back in the day you didn't have to start at creche/ kindergarten before moving on to primary school, so primary school was the first time I had dipped my toe into the education system.
My parents walked with me to my first class, said a few words to my grade 1 teacher, said goodbye to me and told me to be good… This was pretty much the standard protocol it seemed since every other parent seemed to do just about the same thing. I was excited as hell not to be at home, this seemed not to be the standard protocol, every other kid that walked through the door either cried or refused to let go of their parents. But after some re-assuring words from our teacher tear dried up and death grips finally loosened.
The rest of primary school seemed to kinda rush by. Although I was happy to be at school I wasn't all that concerned with school work. My grades were just slightly above average but no where near child prodigy levels. I just didn't see the point of studying. I had built up my own philosophy in my head, that leaned towards thinking that studying for a test was a form of cheating. Teachers always said that we shouldn't fear exams and tests since they just wanted to see what we knew… and well that was what I knew, without the deception of studying.
Flashback: The middle
Having the right tools for doing math meant that I didn’t need to memorize a whole bunch of formulas
This all changed when I got to high school and was introduced to wonders of high school maths. This was a subject I could relate to. Math was a subject of tools. Learn how to use some basic tools and those tools would help you solve just about any problem. I thought of maths as a subject with a strong foundation. They basic principles were always valid and always applicable no matter how complicated the equations appear at first. And if your foundations are strong maths allows you to keep adding to those same foundations.
Having the right tools for doing math meant that I didn’t need to memorize a whole bunch of formulas of even multiplication tables, I was always able to at least solve something from first principles… in an emergency :)
One of the additions to the foundations was calculus.
High school had taught us how rates of change and areas under graphs could be calculated using calculus… but like most people I thought, so what?? It was difficult to see where it could all become useful in the real world, besides calculate velocity or acceleration, that is until I got to university.
Flashback: The end, the real beginning
I studied computer systems Engineering, which was a combination of a software development and electrical engineering degree. We got to mess around with electronics and high level language coding like c, c++ and some lower level assembly languages for micro-controllers like PIC and AVR’s. Putting electronics, code and a secret ingredient of math together finally turned on the light in my head when it came to linking calculus with a real world application.
Calculus in the real world — Your car can do calculus all day long
Many people have never heard of PID control but most people have experienced its effects. PID is an acronym for Proportional Integral Differential and is basically what your car uses to maintain its speed when you activate your cruise control, or how the more sophisticated drones are able to fly so smoothly compared to drones without PID control. These devices are basically solving a calculus problem and a given interval of time, usually in microseconds… continuously. But a bit more on that later.
… find rates of change
So you’ll probably have a vague memory about someone mentioning that you can use calculus to find rates of change (differentiation) or areas under graphs (integration) as i mentioned above, that was the basic notion I had of calculus but I never had a clear geometric way to visualize it, that is until I started my engineering degree.
Rates of change
Whenever someone wants to describe differentiation they usually show you something like the image below, which shows the derivative for 𝕩²
That image probably means a whole lot of nothing and it isn't all that intuitive.
There is a better way to show what the derivative of 𝕩² is, and it involves expressing it a little differently…
Look at the function 𝕩² instead as y=𝕩 * 𝕩 and lets take an arbitrary point where y=1
therefore 1=𝕩 x 𝕩, this is basically saying multiply two numbers together to give me a value of 1, or if we express this as the area of a square, we can ask which two numbers (the same number) multiplied together gives us an area (the blue square) of one.
The answer is obviously 1 x 1 = 1 or said differently y = 𝕩², we can check our answer for 𝕩 = 2 as well, this would give us 2 x 2 = 4 which means the area of our square is 4 or 𝕩² when 𝕩=2. Great but that isn’t the derivative of 𝕩². To find the derivative of 𝕩² we need to understand what it is we’re trying to find first when we say “Find the derivative”.
Back to first principles
Calculus comes in two flavors: differentiation and integration, We’re going to first have a look at differentiation.
In high school I was always taught that the derivative of a function was the gradient of the tangent to the curve that function creates, yeah what the #uck does that mean 😳… although the definition is correct it isn’t very useful when understanding what a derivative is. Lets get to the definition by looking at our earlier equation/ function: 𝕩² or f(𝕩) = 𝕩² (read as function of 𝕩).
Start with the graph for 𝕩² below, which you could plot out on your own by drawing out the Cartesian plane and then selecting values for 𝕩 starting at some random point like 𝕩=3 and get the corresponding y value when running your chosen 𝕩 value through function 𝕩² for example for the point x =2 the y value will equal 2² which is 4 so you make a point a x=2,y=4 and so on and so on, I know that’s obvious but foundations are important so there :)
Now lets say we want to see what happens when we nudge a value of 𝕩 by some amount, this will reflect our change in 𝕩, since derivatives are all about change.
We can pick any initial point for 𝕩, lets say for shits and giggles we pick x = 2, and lets say we want to nudge 𝕩 by some amount which we’ll call d𝕩 (think of it like difference of 𝕩) the size of the nudge isn't really important. Now don’t freak out! d𝕩 is just a place holder for our nudged value it could be equal to something like 0.1 or 0.2 or 1 or 3 or anything.
Now we have a starting value for 𝕩 which is 2 and we have a value for d𝕩 (the value for d𝕩 is not important but lets just keep it small).
First I’d like to show what it looks like to find the derivative on a graph like we have above then I’ll show how we represent that geometrically which is way more intuitive:
Find the graphs derivative, the non-intuitive way
To find the derivative we essentially need to find the gradient of the green line. The gradient is basically the slope of the green line, however we need to find the gradient of the line where the width of the shaded blue section (or d𝕩) approaches 0. So there’s a reason why it needs to approach zero and can’t equal 0, a very important reason. If we made the difference zero we would essentially be left with the value of 𝕩 which we chose to be 2, this would also mean that there would be no other point available to plot a line, we would just have a single point with a 𝕩 and y coordinate. This is why we say d𝕩 which is our nudged 𝕩 value approaches zero. Remember d𝕩, our nudged 𝕩 value is essentially the thickness of the vertical blue line.
OK so let’s work out the derivative or the gradient of that green line. To work out a gradient you need a change in your y value divided by a change in you 𝕩 value.This means we need a co-ordinate (x y value) for or original position when 𝕩 = 2 and y = 4 and we need a co-ordinate for our new nudged value where 𝕩 = 𝕩 + d𝕩 (or 𝕩 plus our change in 𝕩) and y = (𝕩 + d𝕩)².
The formula is equivalent to: Gradient (slope) = change in y / change in 𝕩, as the change in 𝕩 or d𝕩 approaches 0 (d𝕩 -> 0). Note the change in 𝕩 at the bottom (the denominator) is just d𝕩 because if we write out 𝕩+d𝕩 — 𝕩 we are left with d𝕩 alone.
So if we decided to nudge 𝕩 which is equal to 2 by a value of d𝕩 we could write:
this would then expand to give us:
the terms at the top and bottom both have a d𝕩 so this can simplify to:
Since the d𝕩 term approaches zero we can remove it which means the derivative of 𝕩² when 𝕩 = 2 is 4
This is a non-intuitive way of visualising a derivative, a better way is to represent it geometrically using the change in area.
A more intuitive representation
Let’s go back to our geometric representation of 𝕩² using an image of a square
If we were to nudge the value of 𝕩 by some small amount which we’ll call d𝕩 we can easily represent that change as follows:
The grey sections represent how much we’ve increased the area of our square, Imagine stretching our blue square out. To calculate the how much our little blue square’s area has increased, or said differently, to calculate the rate of change of the area of our square we can simply add up the bits that have changed.
Since the length of the longer grey blocks are 𝕩 and the width of the blocks are d𝕩, we can make a simple formula to calculate our total area:
or with a bit of manipulation:
And like before for our derivative d𝕩 will approach zero so the d𝕩 term can fall away since it will have little impact on the formula:
Which is the same as we got when we used the graph to derive the derivative of 𝕩²… plugging in a value of x=2 into 2𝕩 gives us 4… the same answer as before.
Finding the derivative of something like 𝕩³ can also be represented geometrically by drawing a cube and calculating how the volume of that cube would increase with a small d𝕩 change in 𝕩.
Put it all together
So now we have a nice way to visualise a derivative in a geometric way as the increase in are of a square or increase in volume of a cube, derivatives of trigonometric functions such as sine and cosine can also be represented geometrically but that will be a topic for another post, but basically it involves drawing a unit circle and making small nudges to the angle θ.
Flashback: Electronics labs — PID control
The project that made all this madness come together was when we were asked to create an inverted self balancing pendulum. Basically you have a toy car with a rod extending vertically with a weight at the end of it that is free to swing back and forward along a single plane, something like this:
The idea is that when the pendulum moves away from its 90|degree vertical position the car should move forward or backwards to compensate and therefore force the pendulum to remain vertical instead of falling completely forward / horizontal.
Sounds kinda easy in principle since many of us have at some point in our lives tried to balance a stick or something similar on the palm of our hands, however coding such a system is super tricky without the help of calculus, and calculus namely integration and differentiation form the build blocks of the solution to coding up something that can balance a pendulum — The PID controller.
A PID controller is a form of feedback controller. This means that the controller will adjust certain inputs based on an output. So the output of the system is always fed back into the system. In the case of the inverted pendulum, the angle of the pendulum is fed back into the system, then the system will adjust the direction (back or forward) and speed of the little car. This is essentially a control loop. Control loops are used everywhere! Another arbitrary but useful example is when you are running a warm bath — Your hand senses the temperature and then based on the temperature you either increase or decrease the flow of cold or hot water, that’s a form of a control loop :)
As I mentioned earlier PID is an acronym for Proportional Differential Integral. I went through the basics of differentiation at length but now it’s time to look at its opposite: Integration, then we’ll get to the fun part, putting them together to create a PID controller.