The phenomenon actually has a name: Shiny New Idea Syndrome (SNIS).
The question that comes up, of course, is whether you should continue writing the book you were working on—or whether you should start writing the new idea you’ve had…the one you’re excited about.
I've had this happen a couple of times in the last year—I’ve gotten ideas for new series or gotten ideas for things that wouldn’t fit in well with the current series I’m writing.
Because I’m under contract for the books I’m writing (and have already been paid for them), then I didn’t have a choice about which to write. I just spoke into a voice recorder all the ideas I had connected to the new idea and then saved it for later and continued working on the current manuscript.
But it seems to me that there are some questions to ask yourself if you’ve gotten a case of SNIS:
What’s your goal—right now—with writing? Publishing? Writing for your own enjoyment?
If you’re writing to please yourself, decide which story you think you’ll take the most pleasure in writing—is it a turbulent story that will actually be cathartic? Something light and humorous?
If you’re writing for publication, which story is more marketable?
Which story is fuller-formed and better-realized?
Which requires more research (and do you have the time to do it?)
Has this happened just because you’ve reached a difficult spot in your current manuscript? Could you jot down your idea and just brainstorm ways to work through the rough part? And remind yourself that it’s fine to write a horrible or disorganized first draft?
Could you put your current manuscript down for a couple of weeks and work on the new manuscript? You may find that the new manuscript isn’t all that exciting or that you return to the old one with fresh eyes and can easily polish off the first draft on it.
If you’re in the middle of revisions for the old manuscript, you could work on both books at once (I frequently have to do this.) That way you’re doing something creative (penning a first draft) while doing something a little more mechanical (editing/revising.)
I think the danger in putting down an unfinished manuscript is that we won’t return to it. Or that we’re setting a precedent with ourselves that it’s okay to leave a manuscript unfinished.
Have you ever been struck by Shiny New Idea Syndrome? How did you handle it?