I hold the Transcendentalists, including Ralph Waldo Emerson and Henry David Thoreau, in extremely high regard. Reading their essays, their poems, their journals, I think they have come as close to the true nature and purpose of living as our great spiritual leaders like the Buddha, the Dalai Lama, and Jesus (among others).
In the quote above, Thoreau's point isn't the superficial interpretation that all memory is bad--he isn't part of a group labeled the Transcendentalists for nothing. :) Thoreau is blessing what Buddhists call non-attachment. It is not that we repress our memories in our practice of mindfulness in the present moment. Memories are necessary for learning, for growth: imagine trying to meditate when you can't (or won't) remember the instructions given to you by your teacher! Rather, Thoreau observed that people are obsessed with their memories. They replay painful experiences over and over in their minds, which resubmerges them in the flood of anxiety, anger, or jealousy they experienced. People are just as obsessed with pleasant memories--they relive their last vacation, a beautiful sunset, or their elation when they adopted their puppy.
The question arises, "What's wrong with this? Shouldn't I relive the painful experiences to learn from them and the pleasant ones to enjoy my life through reminiscing?" The answer to that question is yes, with reservations. Reliving painful experiences in search of growth is great, when you consciously choose to do so. Thoreau condemns the unconscious practice that we all fall into in which these memories flash across our mental screens, unrequested, and distract us from the wonder that is our present moment of life. It requires a brutally honest look into your mind to truly see the driver of these recollections.
Reliving pleasant memories is a wonderful pastime. However, if recalling that beautiful sunset of a year ago in Hawaii distracts you from mindfully experiencing the red sun descend below the horizon happening right now, today, then that memory has truly done you a disservice. Or if in thinking about when you adopted your fur-baby, and in the process you miss the adorable, inquisitive way she pokes and prods her favorite toy, you've just lost that moment.
With the manner in which he lived his life, Thoreau showed us that a truly mindful, present, experience of what is happening NOW is life's greatest gift. Choose your moments of recollection, and when you do, make them your present moment. But at all other times, don't grasp at these memories--let them go so that you can form wonderful new memories, untainted by your past.
1-Minute Contemplation: Identify a memory that is distracting you from living right now. It may be a hurtful experience you keep reliving, or a pleasant one that you now see distracts you from your real life, here and now. For 1 minute, analyze that memory. What part of it are you attached to such that it keeps arising? Find the reason inside you that will allow you to unattach yourself from this memory. Know that you can choose to return to it whenever you wish. Whenever YOU wish.