There’s a worn-out analogy in software development that you cannot build
a skyscraper the same way you build a dog house. The idea is that
techniques that will work on a small scale will not work on a larger
scale. You need more formality to build large software systems.
The analogy is always applied in one direction: up. It’s always an
exhortation to use techniques appropriate for larger projects.
But the analogy works in the other direction as well: it’s inappropriate
to build a dog house the same way you’d build a skyscraper. It would bepossible
to build a dog house the way you’d build a skyscraper, but it
would be very expensive. Amateur carpentry methods don’t scale up, but
professional construction methods don’t scale down economically.
Bias for over-engineering
There’s a bias toward over-engineering because it works, albeit
inefficiently, whereas under-engineering does not. You can use a
sledgehammer to do a hammer’s job. It’ll be clumsy, and you might hurt
yourself, but it can work. And there are tasks where a hammer just won’t
get the job done.
Another reason for the bias toward over-engineering is asymmetric risk.
If an over-engineered approach fails, you’ll face less criticism than if
a simpler approach fails. As the old saying goes, nobody got fired for
Simple solutions require context to appreciate. If you do something
simple, you’re open to the criticism “But that won’t scale!” You have to
defend your solution by explaining that it will scale far enough, and
that it avoids costs associated with scaling further than necessary.
Suppose a group is debating whether to walk or drive to lunch. Someone
advocating driving requires less context to make his point. He can
simply say “Driving is faster than walking,” which is generally true.
The burden is on the person advocating walking to explain why walking
would actually be faster under the circumstances.
I was using some database-like features in Emacs org-mode this morning
and that’s what prompted me to write this post. I can just hear someone
say “That won’t scale!” I often get this reaction from someone when I
write about a simple, low-tech way to do something on a small scale.
Using a text file as a database doesn’t scale. But I have 88 rows, so I
think I’ll be OK. A relational database would be better for storing
million of records, but that’s not what I’m working on at the moment.
More posts on scale