Original Source Here
The Most Important Class I’ve Never Taken
I’ve always wondered what class should be taught in college (or even high school) but never was. I did a bit of thinking about this and came to the conclusion that it was the opposite of what school actually does teach.
Every essay, project submission or code push is usually expected to be perfect. They say it needs to be right and complete the first time. After submission, you move on to the next assignment usually with the assumption that you’ve submitted the best possible version of what you’re capable of.
Sure that’s all fine and dandy but I quickly realized working on non-school related projects that this was never the case.
What’s unique about building your own software product, writing your own book or shipping something valuable that people in real life will use is that the first version is usually the worst. It is only after multiple times of shipping, improving and getting feedback that what you originally had in mind comes to fruition. And that’s why iteration is important!
10,000 iterations > 10,000 hours
I came across this quote from Naval Ravikant a few years ago and it has since stuck with me. He plays on Malcolm Gladwell’s well-known adage that 10,000 hours is the key to mastery of any topic, subject or skill. What’s funny is Naval’s insight is actually more true than Gladwell’s although both are right.
In working more on developing a skill, you’re invariably going to iterate more and by iterating more you’re going to be spending more time on this skill. However, just because you iterate more doesn’t necessarily mean you are bound by some time constraint which is the nuance in this inequality. Just spending 10,000 hours doesn’t necessarily mean you’re improving, iterations almost always does.
Iterations can happen as quickly or slowly as the human making them is comfortable with. 10,000 iterations might take 10,000 hours or it may only take 100. Let’s borrow an example from the deep learning community to explain this further.
When training learning algorithms, there are two ways you can iterate: stochastic gradient descent and gradient descent.
Both are fairly similar except that stochastic gradient descent tries more random samples instead of going through all of them one by one. Imagine choosing cards randomly instead of seeing them laid out in front of you on a table.
Now what’s interesting is that stochastic gradient descent, or SGD as it’s called takes more iterations to converge. One might naively conclude that an algorithm that takes less iterations to converge is actually better. However, this is not the case because iterations are not a good proxy for performance, epochs are! Epochs are essentially the amount of times you’ve looked at your data once through in it’s entirety. In other words, the number of times you’ve looked at all the examples before you’ve learned the concept.
Using this metric, SGD converges much faster and spends less time looking at the training examples although it takes more iterations.
You can also view your own work in a similar light. When launching a first version of anything — either an app, a book or even a design — getting feedback first, making changes and learning from your mistakes is the likely the fastest way to close the gap between the perfect version you had in mind and the not so perfect one you have in real life.
Just like an AI algorithm, more iterations closes this delta — the difference between reality and your perception of reality — much faster.
It’s the same reason that Y Combinator urges their startup founders to ship early no matter how rudimentary their first prototype is. Less time spent talking to customers and figuring out what they really want actually slows down a startup. This is bad because time is money and money is oxygen for a startup.
To conclude, faster iteration leads to faster progress because you’re not sitting around thinking what might make something better. If you launch and share it with others, you’ll get real feedback that trumps any brainstorming one can do.
I feel that this is true for ideas, books, design, software, really anything that gets better with time and requires market adoption to be successful. Ship faster, get feedback and iterate quickly to perfection!
Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot