It's one of the parts of writing that isn't given that much prominence - the various metamorphoses an idea can go through before it becomes the final story. In my own writing, I've become aware that there are two types of rewriting process. There's the obvious rewriting of correcting spelling, grammar, cutting bad passages, switching words to get a better meaning etc - what Hemingway referred to as 'getting the words right' - but, deeper than that, there's the matter of getting the story right or, more aptly in this case, getting the right story.
In Misery, Stephen King refers to the phenomenon of 'the hole in the page' opening, that moment when an idea takes on a life of its own and becomes something grounded in a kind of new reality. In my experience, it's when you tap into an almost subconscious reality that the idea has created and you're no longer creating, but describing what is happening in this reality. Characters become real people, settings become places and only part of your mind is actually focused on writing the story, the other part is exploring this new reality and discovering that it may have a few surprises lying in wait for you. Writers often talk of characters going off on their own paths, not following the plot the writer thought they would.
As an example, on the project I've just completed with my writing partner, I had the sudden realisation one day that what we had assumed to be a character's assumed name was actually his real name and the backstory we were thinking of for him was something he had created himself to hide his true origins. Introducing that wasn't a case of having to change everything around to make it fit in, but instead it was making sense of a lot of things in that world. Plus, it made perfect sense in terms of the overall theme of the piece - the thin line between what's real and what's fake.. It wasn't a case of 'let's make this character do that' more 'this character is that, and we've just not noticed it before.'
It's both fascinating and frustrating. Fascinating because you have this sense of a whole new world to explore, with real people and places to write about and tell stories of, but also frustrating because it can reveal that there's a story there that's not the one you set out to write. Or, you discover the story you set out to write, but it's not the one you are actually writing at that time. There's a story at the core of this world, but you have to tunnel down through it to get to it. Sometimes you get lucky and hit it dead centre on your first attempt, other times you get close, and only have to backtrack a little, take your bearings and then hit it. Then there are the times when you go wildly off course and you have to go right back to the beginning and start again from another angle.
It's why the 'getting the right story' part of rewriting can be the most frustrating. In basic rewriting you're just getting rid of the bad parts and things that don't work, but when you're trying to find the story you came looking for you might have to throw out things that are perfectly good in themselves - interesting characters, fascinating scenes - but aren't part of the story you're looking to tell. Sometimes those characters can go on and have a life in another story, but a lot of the time they end up just dumped by the wayside, their stories only ever told in drafts no one gets to see.
A couple of years ago I went on an Arvon Foundation course on 'Writing Science Fiction' taught by Ken Macleod and Molly Brown. On one of the nights of the course, Molly read her story 'Doing Things Differently' and talked about the process behind its creation from the original idea through a number of different drafts (17 in total, if I remember rightly) to the final published story. Each draft might not have been an improvement in terms of style, but it was getting closer to the story at the heart of the idea, the one that expressed it best. Ken tells a similar story about the creation of The Star Fraction, that it didn't seem to be working until he turned it around and realised who was telling the story.
There's a saying that 'art is never completed, only abandoned' and in a sense that does apply to the search for the story at the core. It could be argued that no author ever gets to the actual core and it remains as some kind of ur-story, always just out of reach no matter how many attempts you make to reach it from however many angles. The version of a story that gets published is just the final draft, after all, the one that gets closest to the goal and lets the author think they can hit the core story for the next idea.
Going back to the Dahl papers, I think the reason they're of interest is that we never normally get to see these drafts and works-in-progress, we only get to witness the final product, the story that we believe to be the correct one, as it's the only one we're offered. Maybe that can explain why writers don't like people seeing their early drafts of a project - we know they're not the story we're looking for and so don't want people to see them and think that we have found it then be disappointed in the story we finally produce.
However, the Web has opened up the process of creation, at least in some fields. For instance, sites like Drew's Script-o-Rama contain early drafts of many screenplays and film treatments given us the chance to look behind the scenes. It's not really a true look at the creation process - while scripts are called 'first drafts' that's more 'first version submitted to the studio' rather than 'first version completed' and, in a sense, the development process can turn it away from the writer's original vision of the core behind the idea into some producer's script-noted no man's land.
It does open up the prospect, though, of writers using the Web to let people see earlier drafts of published work. It's rare for an earlier (or even different) version of a book to be produced - partly because of authorial refusal to countenance such a thing, but also because publishers doubt such works could be profitable to them. While films may get re-released in a director's cut, or with additional deleted scenes available on DVD, a book is the author's property much more than a film is and so doesn't need to be recut or expanded. There are probably others, but the only example I can think of right now is the extended version of King's The Stand, restoring the sections that were cut from the original to make it affordable to publish. Or, in a slightly different format, there's Peter F Hamilton's publication of his background notes for the Night's Dawn trilogy as The Confederation Handbook.
In these days when early drafts are no longer mouldering piles of paper, but Word documents filed away in the recesses of the computer, how long will it be before an author decides to let people take a look at his or her earlier versions of a story?
This is actually the second version of this post that I've written. Obviously taking advantage of an opportunity to inflict irony upon me by forcing me to rewrite, Blogger ate the first entry which I had written directly into the posting window without saving it first.