Life as an Iterative Process

August 13, 2009

I was reading something by Eliezer today. And he mentioned something about morality being an iterative process. This got me to thinking about iterative processes and life in general. I mean, I spent the greater part of this summer working with an iterative process (namely, Levenberg-Marquadt) for minimizing a certain cost function in order to fit a model to data. And if I spent so much time with it, it would only stand to reason that I SHOULD be able to apply that idea somehow to real life. I mean, it isn’t a one-to-one correspondence. I’m not literally going to use some mathematical algorithm to ‘optimize’ my life. Though that would really be cool.

Here’s my thought process on this: for some iterative method (take Newton’s method in one dimension, since that’s pretty standard fair in any Calc I course and is pretty easy to think about), the goal is to find some value for x, call it x*, that stands as the ‘answer’ to some problem. Usually, at least with Newton’s method, that x* represents the solution, or zero, to some function. But you could just as easily use Newton’s method to minimize a 1D function by finding the zero of the derivative of the function (say the original function is something nasty that you can’t solve analytically, like f(x) = x * exp(x) – 5*x^2). But that’s just a random digression into numerical analysis.

My point is, with all these methods, the first thing you have to do is determine an initial iterate. With a high-power method like Newton’s method (which has quadratic convergence, for Jebus’ sake!), you want your initial guess to be within the neighborhood of the correct answer. Otherwise, you’re going to diverge like woah and never get to the answer you were looking for. If you want to get there more slowly, but also more surely, you might want to use something like the bisection method. That only has linear convergence, but you’ll DEFINITELY get there.

Again, that’s a bit of a divergence (but NOT del dot F!) from the main thrust of what I’m thinking which is this: you need that initial iterate to get the process started. It doesn’t matter how good the method is, if you don’t pick a first guess and then plug it into the technique, you stand 0% chance of getting to where you’re going. Which I guess is just a really fancy-ass way of saying, “You miss 100% of the shots you don’t make.” But somehow at the moment that I thought of this analogy, it sounded really profound.

Anyway, what’s the takeaway message? In numerical analysis, as in life, the main thing you can do to make sure something gets done is to take the first step, pick the initial iterate, and then see what happens. It might be the case that you picked something that doesn’t fulfill any of the conditions for convergence (damn you, fixed point method!). In that case, you note the failure of the method (or the iterate), evaluate your situation, and try again. And again. And again. You may have to try a hundred different iterates and a dozen different methods before you converge to the right answer. But that’s okay. You’re living anyway. Might as well make the most of it and ride the gradient to that optimal solution.

And take heed: the first extrema you find might not be global. You may need some sort of momentum term built into your algorithm to make sure you don’t run into a rut. But all things considered, that’s rarely the problem with your problems. More often than not, it’s the simple fact that you don’t seem to want to get started out of fear that you’ll ‘do it wrong.’ But there is no wrong iterate other than no iterate. Anything you do will give you feedback on what you could be doing better. Expect for doing nothing. That gives you feedback, but all it tells you is that you should be doing something!

So start iterating!

Sidenote: I wonder if I’m going to start thinking more in these sorts of terms the more I get into applied math. I would find that both amusing and terrifying. This is both a new toolkit of metaphors to look at life with and a scary way to sound retarded to the rest of humanity. Let’s hope I do a lot of the former and not much of the latter.

Advertisements

2 Responses to “Life as an Iterative Process”

  1. daveinthewest said

    Hahaha oh man, Dave. You remind me of all my crazy math professors.

    I agree with the idea of taking the first step in the right direction. The more exact or appropriate the first step is, the better it builds momentum for the direction you’re heading.

    I’ll put in my two cents and leave it at that. I won’t tell anyone how to live because fuck, look at me, lol.

    Any idea of using math to control how one live scares the living shit out of me. Now… that being said, I use math, timing, and feedback when I’m pursuing a goal, but when I’m searching for an “answer” or “satisfaction” or what have you, I just look within. I think the reason that math hasn’t really satisfied anyone’s human desires or personal oneness is because its unhealthy to think of life determinately.

    There needs to be a certain element of mystery or surprise in order for someone to be happy, or else, to quote NIN, every day is exactly the same.

  2. Mark K said

    This is something I started looking at a couple years ago. I read an article about Paul MacCready and really started looking at life as an iterative process. I’d imagine you’ve already stumbled across this, but here some info on him below:
    http://37signals.com/svn/posts/2861-how-nature-and-naivet-helped-paul-maccready-build-a-human-powered-airplane-in-only-six-months

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: