Vyvoj Linuxu - ehm, tedy evoluce

[Ladislav Dobias]

Ahoj,

vcera zacal v linux-kernel zajimavy thread o vyvoji (tedy
spise evoluci :-) Linuxu. Prikladam 5 mailu od Linuse.

Doufam, ze se pobavite a poucite jako ja :-)

Pro ty, co nemaji cas to cist cele, strucny vytah (ty body,
u kterych nepisu, kdo je autor, rekl Linus Torvalds):

- zacalo to tim, ze Larry McVoy napsal obdivny mail o vyvoji
  Solarisu; jak se mu libi, ze to dela jen par lidi a maji
  vsechno pekne naplanovano
- Rik van Riel podotkl, ze souhlasi s Larrym a ze vyvoj
  Linuxu ma uspech jen diky "naprostemu stesti"
- na to reagoval Linus Torvalds, ze s tim "naprostym stestim"
  zcela souhlasi; a zacal rozebirat vyvoj nejslozitejsiho
  systemu (- ani Solarisu ani Linuxu) - cloveka
- proc tedy vyvoj cloveka funguje:
  - dostupnost "zdrojoveho kodu" (DNA)
  - nepromijejici prostredi nahrazujici spatne verze
    lepsimi (prezije ten, kdo se nejlip prizpusobi)
  - masivni nerizeny paralelni vyvoj (pokus - omyl, stale
    dokola)
- proste: "at zije evoluce"

- Tim Hockin podotkl, ze nemame desitky tisic let na
  vyvoj...
- Linus oponoval, ze rizena evoluce je celkem rychla - napr.
  slechteni psich plemen apod.
- nekdy je take vyvoj urychlen katastrofami (asi jako treba
  meteor)
- ale na druhou stranu, evoluce smeruje ke "stabilnim"
  zivotnim formam (napr. zraloci a mnoho obojzivelniku se
  miliony let nevyviji); neni na ne vyvijen tlak
- jiz zminovany Solaris je pry odsouzen k zaniku; protoze se
  moc specializuje na jednu vec; a to neni dobre pro preziti
- Microsoft to ma lepsi v tom, ze ma vetsi populaci a ta ho
  proste nuti se nespecializovat na jedinou vec

- Victor Todaiken rekl, ze Linux je tim, cim je, diky svemu
  _navrhu_, nikoli nahodou
- Linus se priznal, ze Linux nikdy nebyl "navrhovan"
- sice neco prevzal z Unixu, ale to je jako bychom rekli, ze
  budeme vyrabet auto, a tudiz ze bude mit 4 kola a predni
  svetla - to je vse, co bylo "navrzeno"
- rekl, ze pred 10 lety netusil, jak bude vypadat Linux dnes
- take tvrdi, ze zadny velky softwarovy projekt, ktery je
  uspesny na trhu nikdy neprosel takovymi temi peknymi
  stadii vyvoje softwaru, jak se to ucime ve skole (analyza,
  navrh, implementace, sireni)
- software se tedy "evolucne vyviji" neni nikym "navrhovan";
  staci jen primerene "ridit evoluci"; ale moc rizeni
  zabiji

- tak Viktor Yodaiken rekl, ze na zacatku (UNIXu) prece
  existoval "navrh" - od Denise a Kena
- Linus se vsadil o pet dolaru, ze tito panove by souhlasili
  spise s nim (Linusem), nez v Viktorem
- precizne navrzene jsou napr. systemy: L4 a Plan-9; jak
  jsou ale dnes popularni? "Navrh" je jako nabozenstvi -
  kdyz je ho moc, tak je neflexibilni a nepopularni
- Unix se vyvijel "evolucne" -zakladni myslenky sice
  v Unixu byly, ale zakladni myslenky nevytvori system
- Viktor Y. take tvrdil, ze jedna z dobrych metod "navrhu"
  Linuxu je: precist si literaturu, premyslet o ni, zkusit
  si neco, a pak to naimplementovat (a Linus to muze nazyvat
  klidne "rizenou mutaci")
- Linus oponuje tim, ze toto neni moc casty pripad - a spis to
  plati pro nekterou konkretni malou cast
- tedy ze na mikroskopicke urovni funguje tato "rizena
  mutace", ale na makroskopicke urovni neni zadny dany smer
- Linus priznal, ze se mu libi "Braunuv pohyb"
- a ze "navrh" funguje tam, kde vime, ze jdeme presne z bodu
  A do body B; ale vime my, ze chceme presne do bodu B?

Pro dalsi podrobnosti se podivejte dolu do tech 5 mailu dole
a nebo do archivu l-k na tento thread...

Zdravi
Lada Dobias

P.S.
Jeste ted Viktor Y. napsal, ze chce $5 a citoval Richieho
z nejake knihy. Take citoval Linuse, z dob, kdy rikal neco
jineho :-)

A ted uz ty maily:
======================================================
>From torvalds na transmeta.com Sat Dec  1 12:29:28 2001
Date: Fri, 30 Nov 2001 16:50:34 -0800 (PST)
From: Linus Torvalds <torvalds na transmeta.com>
To: Rik van Riel <riel na conectiva.com.br>
Cc: Andrew Morton <akpm na zip.com.au>, Larry McVoy <lm na bitmover.com>,
     Daniel Phillips <phillips na bonn-fries.net>,
     Henning Schmiedehausen <hps na intermeta.de>,
     Jeff Garzik <jgarzik na mandrakesoft.com>, linux-kernel na vger.kernel.org
Subject: Re: Coding style - a non-issue


On Fri, 30 Nov 2001, Rik van Riel wrote:
>
> I'm very interested too, though I'll have to agree with Larry
> that Linux really isn't going anywhere in particular and seems
> to be making progress through sheer luck.

Hey, that's not a bug, that's a FEATURE!

You know what the most complex piece of engineering known to man in the
whole solar system is?

Guess what - it's not Linux, it's not Solaris, and it's not your car.

It's you. And me.

And think about how you and me actually came about - not through any
complex design.

Right. "sheer luck".

Well, sheer luck, AND:
 - free availability and _crosspollination_ through sharing of "source
   code", although biologists call it DNA.
 - a rather unforgiving user environment, that happily replaces bad
   versions of us with better working versions and thus culls the herd
   (biologists often call this "survival of the fittest")
 - massive undirected parallel development ("trial and error")

I'm deadly serious: we humans have _never_ been able to replicate
something more complicated than what we ourselves are, yet natural
selection did it without even thinking.

Don't underestimate the power of survival of the fittest.

And don't EVER make the mistake that you can design something better than
what you get from ruthless massively parallel trial-and-error with a
feedback cycle. That's giving your intelligence _much_ too much credit.

Quite frankly, Sun is doomed. And it has nothing to do with their
engineering practices or their coding style.

		Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo na vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

>From torvalds na transmeta.com Sat Dec  1 12:35:25 2001
Date: Fri, 30 Nov 2001 18:57:44 -0800 (PST)
From: Linus Torvalds <torvalds na transmeta.com>
To: Tim Hockin <thockin na hockin.org>
Cc: Rik van Riel <riel na conectiva.com.br>, Andrew Morton <akpm na zip.com.au>,
     Larry McVoy <lm na bitmover.com>, Daniel Phillips <phillips na bonn-fries.net>,
     Henning Schmiedehausen <hps na intermeta.de>,
     Jeff Garzik <jgarzik na mandrakesoft.com>, linux-kernel na vger.kernel.org
Subject: Re: Coding style - a non-issue


On Fri, 30 Nov 2001, Tim Hockin wrote:
>
> > I'm deadly serious: we humans have _never_ been able to replicate
> > something more complicated than what we ourselves are, yet natural
> > selection did it without even thinking.
>
> a very interesting argument, but not very pertinent - we don't have 10's of
> thousands of year or even really 10's of years.  We have to use intellect
> to root out the obviously bad ideas, and even more importantly the
> bad-but-not-obviously-bad ideas.

Directed evolution - ie evolution that has more specific goals, and faster
penalties for perceived failure, works on the scale of tens or hundreds of
years, not tens of thousands. Look at dog breeding, but look even more at
livestock breeding, where just a few decades have made a big difference.

The belief that evolution is necessarily slow is totally unfounded.

HOWEVER, the belief that _too_ much direction is bad is certainly not
unfounded: it tends to show up major design problems that do not show up
with less control. Again, see overly aggressive breeding of some dogs
causing bad characteristics, and especially the poultry industry.

And you have to realize that the above is with entities that are much more
complex than your random software project, and where historically you have
not been able to actually influence anything but selection itself.

Being able to influence not just selection, but actually influencing the
_mutations_ that happen directly obviously cuts down the time by another
large piece.

In short, your comment about "not pertinent" only shows that you are
either not very well informed about biological changes, or, more likely,
it's just a gut reaction without actually _thinking_ about it.

Biological evolution is alive and well, and does not take millions of
years to make changes. In fact, most paleolontologists consider some of
the changes due to natural disasters to have happened susprisingly fast,
even in the _absense_ of "intelligent direction".

Of course, at the same time evolution _does_ heavily tend to favour
"stable" life-forms (sharks and many amphibians have been around for
millions of years). That's not because evolution is slow, but simply
because good lifeforms work well in their environments, and if the
environment doesn't change radically they have very few pressures to
change.

There is no inherent "goodness" in change. In fact, there are a lot of
reasons _against_ change, something we often forget in technology. The
fact that evolution is slow when there is no big reason to evolve is a
_goodness_, not a strike against it.

> > Quite frankly, Sun is doomed. And it has nothing to do with their
> > engineering practices or their coding style.
>
> I'd love to hear your thoughts on why.

You heard them above. Sun is basically inbreeding. That tends to be good
to bring out specific characteristics of a breed, and tends to be good for
_specialization_. But it's horrible for actual survival, and generates a
very one-sided system that does not adapt well to change.

Microsoft, for all the arguments against them, is better off simply
because of the size of its population - they have a much wider consumer
base, which in turn has caused them largely to avoid specialization. As a
result, Microsoft has a much wider appeal - and suddenly most of the
niches that Sun used to have are all gone, and its fighting for its life
in many of its remaining ones.

Why do you think Linux ends up being the most widely deployed Unix? It's
avoided niches, it's avoided inbreeding, and not being too directed means
that it doesn't get the problems you see with unbalanced systems.

Face it, being one-sided is a BAD THING. Unix was dying because it was
becoming much too one-sided.

Try to prove me wrong.

		Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo na vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

>From torvalds na transmeta.com Sat Dec  1 12:39:14 2001
Date: Fri, 30 Nov 2001 19:15:55 -0800 (PST)
From: Linus Torvalds <torvalds na transmeta.com>
To: Victor Yodaiken <yodaiken na fsmlabs.com>
Cc: Rik van Riel <riel na conectiva.com.br>, Andrew Morton <akpm na zip.com.au>,
     Larry McVoy <lm na bitmover.com>, Daniel Phillips <phillips na bonn-fries.net>,
     Henning Schmiedehausen <hps na intermeta.de>,
     Jeff Garzik <jgarzik na mandrakesoft.com>, linux-kernel na vger.kernel.org
Subject: Re: Coding style - a non-issue


On Fri, 30 Nov 2001, Victor Yodaiken wrote:
>
> > And don't EVER make the mistake that you can design something better than
> > what you get from ruthless massively parallel trial-and-error with a
> > feedback cycle. That's giving your intelligence _much_ too much credit.
>
> Linux is what it is because of design, not accident. And you know
> that better than anyone.

Let's just be honest, and admit that it wasn't designed.

Sure, there's design too - the design of UNIX made a scaffolding for the
system, and more importantly it made it easier for people to communicate
because people had a mental _model_ for what the system was like, which
means that it's much easier to discuss changes.

But that's like saying that you know that you're going to build a car with
four wheels and headlights - it's true, but the real bitch is in the
details.

And I know better than most that what I envisioned 10 years ago has
_nothing_ in common with what Linux is today. There was certainly no
premeditated design there.

And I will claim that nobody else "designed" Linux any more than I did,
and I doubt I'll have many people disagreeing. It grew. It grew with a lot
of mutations - and because the mutations were less than random, they were
faster and more directed than alpha-particles in DNA.

> The question is whether Linux can still be designed at
> current scale.

Trust me, it never was.

And I will go further and claim that _no_ major software project that has
been successful in a general marketplace (as opposed to niches) has ever
gone through those nice lifecycles they tell you about in CompSci classes.
Have you _ever_ heard of a project that actually started off with trying
to figure out what it should do, a rigorous design phase, and a
implementation phase?

Dream on.

Software evolves. It isn't designed. The only question is how strictly you
_control_ the evolution, and how open you are to external sources of
mutations.

And too much control of the evolution will kill you. Inevitably, and
without fail. Always. In biology, and in software.

Amen.

		Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo na vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

>From torvalds na transmeta.com Sat Dec  1 12:42:09 2001
Date: Fri, 30 Nov 2001 19:34:30 -0800 (PST)
From: Linus Torvalds <torvalds na transmeta.com>
To: Larry McVoy <lm na bitmover.com>
Cc: Victor Yodaiken <yodaiken na fsmlabs.com>,
     Rik van Riel <riel na conectiva.com.br>, Andrew Morton <akpm na zip.com.au>,
     Daniel Phillips <phillips na bonn-fries.net>,
     Henning Schmiedehausen <hps na intermeta.de>,
     Jeff Garzik <jgarzik na mandrakesoft.com>, linux-kernel na vger.kernel.org
Subject: Re: Coding style - a non-issue


On Fri, 30 Nov 2001, Larry McVoy wrote:
>
> I can't believe the crap you are spewing on this one and I don't think you
> do either.  If you do, you need a break.  I'm all for letting people explore,
> let software evolve, that's all good.  But somebody needs to keep an eye on
> it.

Like somebody had to keep an eye on our evolution so that you had a chance
to be around?

Who's naive?

> If that's not true, Linus, then bow out.   You aren't needed and *you*
> just proved it.

Oh, absolutely.

I wish more people realized it. Some people realize it only when they get
really pissed off at me and say "Go screw yourself, I can do this on my
own". And you know what? They are right too, even if they come to that
conclusion for what I consider the wrong reasons.

The reason I'm doing Linux is not because I think I'm "needed". It's
because I enjoy it, and because I happen to believe that I'm better than
most at it. Not necessarily better than everybody else around there, but
good enough, and with the social ties to make me unbeatable right now.

But "indispensable"? Grow up, Larry. You give me too much credit.

And why should I bow out just because I'm not indispenable? Are you
indispensable for the continued well-being of humanity? I believe not,
although you are of course free to disagree. Should you thus "bow out" of
your life just because you're strictly speaking not really needed?

Do I direct some stuff? Yes. But, quite frankly, so do many others. Alan
Cox, Al Viro, David Miller, even you. And a lot of companies, which are
part of the evolution whether they realize it or not. And all the users,
who end up being part of the "fitness testing".

And yes, I actually do believe in what I'm saying.

		Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo na vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

>From torvalds na transmeta.com Sat Dec  1 12:49:19 2001
Date: Fri, 30 Nov 2001 21:15:50 -0800 (PST)
From: Linus Torvalds <torvalds na transmeta.com>
To: Victor Yodaiken <yodaiken na fsmlabs.com>
Cc: Rik van Riel <riel na conectiva.com.br>, Andrew Morton <akpm na zip.com.au>,
     Larry McVoy <lm na bitmover.com>, Daniel Phillips <phillips na bonn-fries.net>,
     Henning Schmiedehausen <hps na intermeta.de>,
     Jeff Garzik <jgarzik na mandrakesoft.com>, linux-kernel na vger.kernel.org
Subject: Re: Coding style - a non-issue


On Fri, 30 Nov 2001, Victor Yodaiken wrote:
>
> Ok. There was no design, just "less than random mutations".
> Deep.

I'm not claiming to be deep, I'm claiming to do it for fun.

I _am_ claiming that the people who think you "design" software are
seriously simplifying the issue, and don't actually realize how they
themselves work.

> There was a overall architecture, from Dennis and Ken.

Ask them. I'll bet you five bucks they'll agree with me, not with you.
I've talked to both, but not really about this particular issue, so I
might lose, but I think I've got the much better odds.

If you want to see a system that was more thoroughly _designed_, you
should probably point not to Dennis and Ken, but to systems like L4 and
Plan-9, and people like Jochen Liedtk and Rob Pike.

And notice how they aren't all that popular or well known? "Design" is
like a religion - too much of it makes you inflexibly and unpopular.

The very architecture of UNIX has very much been an evolution. Sure, there
are some basic ideas, but basic ideas do not make a system.

When they say that the devil is in the details, they are trying to tell
you that the details MATTER. In fact, the details matter quite a lot more
than the design ever does.

> Here's a characteristic good Linux design method ,( or call it "less than random
> mutation method" if that makes you feel happy): read the literature,
> think hard, try something, implement

Hah.

I don't think I've seen very many examples of that particular design
methodology.

It's impressive that you think this is how stuff gets done, but from
personal experience I would say that it's certainly not true to any
noticeable degree. The impressive part is that Linux development could
_look_ to anybody like it is that organized.

Yes, people read literature too, but that tends to be quite spotty. t's
done mainly for details like TCP congestion control timeouts etc - they
are _important_ details, but at the same time we're talking about a few
hundred lines out of 20 _million_.

And no, I'm no tclaiming that the rest is "random". But I _am_ claiming
that there is no common goal, and that most development ends up being done
for fairly random reasons - one persons particular interest or similar.

It's "directed mutation" on a microscopic level, but there is very little
macroscopic direction. There are lots of individuals with some generic
feeling about where they want to take the system (and I'm obviously one of
them), but in the end we're all a bunch of people with not very good
vision.

And that is GOOD.

A strong vision and a sure hand sound like good things on paper. It's just
that I have never _ever_ met a technical person (including me) whom I
would trust to know what is really the right thing to do in the long run.

Too strong a strong vision can kill you - you'll walk right over the edge,
firm in the knowledge of the path in front of you.

I'd much rather have "brownian motion", where a lot of microscopic
directed improvements end up pushing the system slowly in a direction that
none of the individual developers really had the vision to see on their
own.

And I'm a firm believer that in order for this to work _well_, you have to
have a development group that is fairly strange and random.

To get back to the original claim - where Larry idolizes the Sun
engineering team for their singlemindedness and strict control - and the
claim that Linux seems ot get better "by luck": I really believe this is
important.

The problem with "singlemindedness and strict control" (or "design") is
that it sure gets you from point A to point B in a much straighter line,
and with less expenditure of energy, but how the HELL are you going to
consistently know where you actually want to end up? It's not like we know
that B is our final destination.

In fact, most developers don't know even what the right _intermediate_
destinations are, much less the final one. And having somebody who shows
you the "one true path" may be very nice for getting a project done, but I
have this strong belief that while the "one true path" sometimes ends up
being the right one (and with an intelligent leader it may _mostly_ be the
right one), every once in a while it's definitely the wrong thing to do.

And if you only walk in single file, and in the same direction, you only
need to make one mistake to die.

In contrast, if you walk in all directions at once, and kind of feel your
way around, you may not get to the point you _thought_ you wanted, but you
never make really bad mistakes, because you always ended up having to
satisfy a lot of _different_ opinions. You get a more balanced system.

This is what I meant by inbreeding, and the problem that UNIX
traditionally had with companies going for one niche.

(Linux companies also tend to aim for a niche, but they tend to aim for
_different_ niches, so the end result is the very "many different
directions" that I think is what you _want_ to have to survive).

> > And I will go further and claim that _no_ major software project that has
> > been successful in a general marketplace (as opposed to niches) has ever
> > gone through those nice lifecycles they tell you about in CompSci classes.
>
> That's classic:
> 	A) "trust me"
> 	B) now here's a monster bit of misdirection for you to choke on.
>
> Does anyone believe in those stupid software lifcycles?
> No.
> So does it follow that this has anything to do with design?
> No.

Hey, the whole _notion_ of "design" is that you know what you're going to
write, and you design for it.

Or do you have some other meaning of the word "design" that I haven't
heard about.

		Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo na vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/