Anyway, this is all in theory...
And practically speaking now, if a piece of software has multiple authors, each author can sue. You don't need the agreement of all authors for that. So there are thousands of developers who can actually sue when somebody infringes the copyrights of Linux.
No, they can sue for the _parts_ in Linux that they hold the copyright on. To enforce the copyright of the whole Linux kernel is impossible without an agreement from all parties that have copyrighted material in it.
Removing the bits which are copyrighted by somebody isn't really feasible if they have contributed a lot. You need to find out what a person has committed over the years and which work is based on that, etc. And it's not like you can remove the virtual memory code and run without it or rewrite in a few hours. There are a few developers like Linus who could sue and then you really don't have anything usable left.
You'd have a stronger poitn if say Ingo Molnar sued, Linus' contribution are quite small these days.
And they would also have to prove that they wrote the code, which is a bit easier these days, but in the good old days you didn't even have a detailed ChangeLog.
Cheers.