November 10, 2009

When you are a young person, all of this feels inevitable. It feels inevitable that you’ll be on television, or that you’ll be an astronaut, or that you’ll be the president. It’s hard-wired into every gland, this ambition to be known and renowned. And then of course you grow older, pudgier, stouter, portly, you have children or get a job or are drawn by fate to one life or another. Only the deranged don’t notice that the possibilities for their life are narrowing. And only the truly happy look around and say, “That’s fine.”

– John Hodgman from More Information Than You Require

Well-Posed Problems

August 23, 2009

To continue along with the mathematical analogies that I built in my previous essay on iterative processes, I thought I would examine and riff on the concept of ‘well-posed problems.’ To be honest, this isn’t something that I have much experience with in mathematics. Because, typically, in all the math classes I’ve had, the problems are ALWAYS well-posed. That is, they’re always something that can be solved.

But life isn’t like math class. Typically, the majority of problems in my life aren’t well-posed. Heck, half of them aren’t even POSED. And if that’s the case, how can I expect to get anywhere with them?

To go back to the math metaphor again, what does it even mean for a problem to be well-posed? It means that a solution exists. That you can get there from here. So, on the flip-side, what does it mean if a problem isn’t well-posed? What does it mean for a problem to be ill-posed? It means that there IS NO SOLUTION. No matter how hard you try, you can’t get there from here. You can scratch your head, jump on one foot while patting your belly, or even bang your head against the wall continuously. That won’t get you to the solution. Because quite simply, that solution doesn’t exist.

And in life? You can go about life half asleep. That’s analogous to always dealing with ill-posed problems. And yes, it has its advantages. Like the fact that you don’t have to think quite so much. Because thinking is, like, hard. And if all you ever do is try to solve one ill-posed problem after another, you’ll always feel like you’re doing something. And if just doing ‘something’ is what you’re after, then damn if you’re not doing a good job!

But if you’re interested in doing anything more than ‘something,’ say ‘something in particular,’ then it’s going to require that you find a well-posed problem. Although life has the slight advantage over math in that sometimes you might start with an ill-posed problem and still get an answer (even a ‘correct’ answer, if that terminology has any meaning in the real world). But then you’re just being lucky, you’re not being smart. And although those two domains are not strictly exclusive, one is well within my control while the other is, almost by definition, not.

So, to move things forward you need to pose the ‘thing’ well. I guess that’s analogous to setting up real, honest, concrete goals. And I would imagine that setting those goals must also involve writing the thing down. Because the mind has a silly way of making it seem like you ‘know’ and ‘remember’ things, just because you happen to be able to keep a fuzzy concept of those things in your mind’s eye for a few seconds. Like this whole whole ‘well-posed problems’ analogy. It was stuck in my mind when I first thought of it, but it certainly didn’t look anything like it does now.

And I certainly don’t really have any concrete details as to how to tell whether a ‘life’ problem is well-posed. That will require more thinking (eke!) on my part.

Time to pose some problems. And iterate.

I was reading something by Eliezer today. And he mentioned something about morality being an iterative process. This got me to thinking about iterative processes and life in general. I mean, I spent the greater part of this summer working with an iterative process (namely, Levenberg-Marquadt) for minimizing a certain cost function in order to fit a model to data. And if I spent so much time with it, it would only stand to reason that I SHOULD be able to apply that idea somehow to real life. I mean, it isn’t a one-to-one correspondence. I’m not literally going to use some mathematical algorithm to ‘optimize’ my life. Though that would really be cool.

Here’s my thought process on this: for some iterative method (take Newton’s method in one dimension, since that’s pretty standard fair in any Calc I course and is pretty easy to think about), the goal is to find some value for x, call it x*, that stands as the ‘answer’ to some problem. Usually, at least with Newton’s method, that x* represents the solution, or zero, to some function. But you could just as easily use Newton’s method to minimize a 1D function by finding the zero of the derivative of the function (say the original function is something nasty that you can’t solve analytically, like f(x) = x * exp(x) – 5*x^2). But that’s just a random digression into numerical analysis.

My point is, with all these methods, the first thing you have to do is determine an initial iterate. With a high-power method like Newton’s method (which has quadratic convergence, for Jebus’ sake!), you want your initial guess to be within the neighborhood of the correct answer. Otherwise, you’re going to diverge like woah and never get to the answer you were looking for. If you want to get there more slowly, but also more surely, you might want to use something like the bisection method. That only has linear convergence, but you’ll DEFINITELY get there.

Again, that’s a bit of a divergence (but NOT del dot F!) from the main thrust of what I’m thinking which is this: you need that initial iterate to get the process started. It doesn’t matter how good the method is, if you don’t pick a first guess and then plug it into the technique, you stand 0% chance of getting to where you’re going. Which I guess is just a really fancy-ass way of saying, “You miss 100% of the shots you don’t make.” But somehow at the moment that I thought of this analogy, it sounded really profound.

Anyway, what’s the takeaway message? In numerical analysis, as in life, the main thing you can do to make sure something gets done is to take the first step, pick the initial iterate, and then see what happens. It might be the case that you picked something that doesn’t fulfill any of the conditions for convergence (damn you, fixed point method!). In that case, you note the failure of the method (or the iterate), evaluate your situation, and try again. And again. And again. You may have to try a hundred different iterates and a dozen different methods before you converge to the right answer. But that’s okay. You’re living anyway. Might as well make the most of it and ride the gradient to that optimal solution.

And take heed: the first extrema you find might not be global. You may need some sort of momentum term built into your algorithm to make sure you don’t run into a rut. But all things considered, that’s rarely the problem with your problems. More often than not, it’s the simple fact that you don’t seem to want to get started out of fear that you’ll ‘do it wrong.’ But there is no wrong iterate other than no iterate. Anything you do will give you feedback on what you could be doing better. Expect for doing nothing. That gives you feedback, but all it tells you is that you should be doing something!

So start iterating!

Sidenote: I wonder if I’m going to start thinking more in these sorts of terms the more I get into applied math. I would find that both amusing and terrifying. This is both a new toolkit of metaphors to look at life with and a scary way to sound retarded to the rest of humanity. Let’s hope I do a lot of the former and not much of the latter.

Graduation Fluff

June 22, 2009

Listening to my son’s high school graduation ceremony last night, I was struck by how completely implausible were many speaker claims, such as:

  • Never let anyone tell you there is something you can’t do.
  • You’ll have setbacks, but never let them discourage you.
  • If I can succeed, so can you.
  • We’ll always treasure our memories of high school.
  • We students are so thankful to have such a friendly principal.
  • I was embarrassed to be associated with such transparent falsehoods, but apparently I’m in a minority.

    – Robin Hanson

    Okay, that’s a pretty suggestive title. I don’t mean to say that Einstein and Newton didn’t accomplish amazing work. Or that all of us can be a Newton or an Einstein.

    What I do mean to express is that these gentleman (and Nobel Prize winners, Fields Metal recipients, etc.) are not necessarily gods among men.

    The first individual that pointed me in this direction (at least, to the point of feeling it necessary to write a post about this) was my graduate student assistant. We were discussing solving a linear first order differential equation (you may recall that perhaps the simplest method for solving such an equation is by using an integrating factor). I said something to the effect of, “How did these guys some up with that stuff?” And the graduate student responded by saying, “Well, actually, if you just think about it for a little while, it makes complete sense.”

    Later that same day, I mentioned how impressed I was with Newton that he had developed the calculus. The grad student again responded by saying, “Yeah, actually, it’s not as impressive as what you would think. They (ie mathematicians at the time) knew about derivatives and integrals. I mean, slopes and areas under a curve are kind of intuitive. What Newton managed to do, his great accomplishment, was combining these two concepts through a form of the fundamental theorem of calculus.”

    Deflated again. But he has a point. Discoveries do not happen in a vacuum. In fact, they happen a pretty big, complicated network. This is evident from the fact that calculus was developed independently by both Newton and Leibniz. Both men had available to them all the mathematics at the time. And both managed to come up with basically the same idea independently. (Newton gets all the credit, whereas most of the notation we use stems from Leibniz).

    Similarly, Einstein didn’t just sit down one day and think really hard about space-time and then come up with his theories of relativity. I’m reading the book Einstein: His Life and Universe right now. As a biography, it traces the genesis of his ideas about relativity. Basically, he completely surrounded himself with the theories of the luminaries of his time: Boltzmann, Mach, Helmholtz, Planck, etc. He read and studied their textbooks and papers studiously. And only THEN, after reading through their works, grappling with their ideas, did he manage to make the next logical step and develop his theories of relativity, quantum mechanics, and brownian motion.

    In other words, it wasn’t magic. He wasn’t somehow independently brilliant. If you stripped him away of his education, he would have been a really bright, hard working kid. But he wouldn’t have come up with relativity.

    I write all this because it gives me a sort of hope. Because I know that I’m not ‘Einstein brilliant.’ That is, I know that I couldn’t possibly ever stand up to the popular conception of Einstein. But I’m a decently hard worker that shows a decent aptitude for mathematics and physics. If I want to follow in Einstein’s footsteps, that is, the real Einstein, then the next logical step is to start engrossing myself in the works of the masters of this day. That might take a while, because there are notably more masters now then there were in the late 1800s and early 1900s. But still, it is easy enough to pick a speciality and stick to. Read everything there is to read in that speciality. And then start trying to build on it. Not quantum leaps. Baby steps.

    I should mention that a decent amount of my thinking on this topic originated from the book The Talent Code and the article Einstein’s Superpowers. Both are exceptionally good reads, both entertaining and informative.

    I had an epiphany this past Friday about why I don’t really enjoy participating in sports all that much.

    I went out golfing with the grad student assisting my research group and another REU-er. The grad student brought his friends, and all of them had golfed before. Both I and the other undergraduate had never golfed before. I actually didn’t end up golfing (I came up with some lame excuse involving the fact that there were five of us and they really only wanted groups of four). Instead, I tagged along and acted as the caddy to the other undergrad. I’m pretty sure I enjoyed myself much more than I would have had I golfed.

    Listening to the other undergrads experience helped crystallize in my mind why I never really enjoyed sports. Throughout the night, he got more and more excited as he improved (and he did improve considerably from the first hole to the eighteenth hole). He told me that he gets pretty competitive and always wants to improve whenever he plays a sport.

    And there it was: I’m not really all that competitive. At least, I don’t think I am. That is, I’m not consciously all that competitive. So, when I walk on a sporting field, having spent very little time honing my skills, I’m already below average in terms of general athleticism (catching / throwing objects, blocking people, running and avoiding obstacles, etc.). Add to that below average athleticism that fact that I also (a) don’t really care to get batter and (b) don’t like looking like an idiot in front of other people, it’s really no wonder that I dislike sports. They’re basically a giant incubator for (potentially) looking like an idiot in front of tons of people (mainly because they’re meant to NOT look like an idiot and thus signal your fitness, skills, etc.) and are meant TO allow for improvement and competitiveness. That second trait is almost the definition of a sport.

    I don’t know what that says about me. Probably nothing good. But it does say that if I ever want to improve on my sporting skills and actually make playing recreational sports fun, I will have to drastically rework the way I think about sports in general. And I don’t really know if I think it’s worth it. I’ve learned how to ride a bike (thanks Dave!), and I can swim passably well. Those are survival skills if nothing else. Throwing and catching a ball only becomes a survival inasmuch as it is a great way to socialize. So by not participating in such activities out of apathy, I’m closing down a viable source of group bonding.

    Yeah, that’s probably not good.

    In a side note, I wonder why I enjoy martial arts? I started that at a young age (maybe around 2nd grade… I don’t really remember). I never ever disliked it, through the 10+ years that I participated in it a group program. And yet martial arts displays all the same indicators of things that I don’t enjoy: performing in front of a group, progressively becoming better via feedback and competition, etc. It’s kind of weird that I would like that and not like other sports. Maybe the very basic, mechanical level of martial arts interests me more? And now, it probably has more to do with the Eastern philosophy aspect and the general necessity of keeping in shape (the same reason that I enjoy running).

    There you have it. After all that speculation, we have ended where we started: I suck at games involving balls. :P

    This is priceless. The name of the ‘show’ is Cartoon All-Stars to the Rescue. It tracks the descent of a teenager into drug culture. But that worse-than-laughable anti-drug cliches aren’t even the best part. No, that honor has to go to seeing all of my favorite childhood cartoon characters telling the kid to ‘just say no.’

    The sad thing is that the anti-drug campaign hasn’t progressed much in 20+ years. They’re still making the same hackneyed, counterfactual claims to defend their twisted worldview. We can only hope that someday they’ll grow up and join the rest of us in a place called reality.

    April 19, 2009

    Of course, the battle has already been half-lost once you have a category “drugs”. Eliezer once mentioned something about how considering {Adolf Hitler, Joe Stalin, John Smith} a natural category isn’t going to do John Smith any good, no matter how nice a man he may be. In the category “drugs”, which looks like {cocaine, heroin, LSD, marijuana}, LSD and marijuana get to play the role of John Smith.

    – Yvain from The Trouble with “Good”

    April 16, 2009

    The world existed before me and would continue to exist without me … It was no more than an entertainment to which I had been invited without knowing why or how, and the meaning of which I could not grasp, if indeed it had one. But this entertainment was none the less not without its interest. That is why I turned my eyes towards nature rather than towards abstract ideas. When I had to leave the entertainment I would do so regretfully, because I found it interesting. But in time it would no doubt end by boring me. Besides, in any case, I had no choice. And what did it matter? When one crushes an ant the world goes on just the same.

    – Dr. Marcel Carret

    April 10, 2009

    Adulthood ultimately means becoming a hypocrite on your own terms.

    – Merlin Mann