How can I turn the memories off ?

Since the release of Seasons/Patch 1.42 you are now able to turn off memories completely. This won’t delete existing ones but you should not be getting any new ones. For me, it doesn’t work that great. I have to turn them off every time I start the game which of course is a bug. Not everyone seems affected. Details see here:  Turn off memory notifications

How can I delete existing memories ?

Twallan’s MasterController has an option to delete all existing memories from all your Sims or from a subset of Sims. You can also turn them off entirely with the mod.

How do I turn the system off completely and in a reliable way ?

There’s a few mods around that can help you with that.

Velocitygrass made a little tool which lets YOU decide, which memories are worth recording and which ones aren’t. All further details can be found in the mod description itself: No (or fewer) automatic memories

Shimrod’s Memories Disabled turn them off completely. This mods reduces the number of allowed memories to 0. Existing memories will still need to be deleted manually or with MasterController.

MasterController also has the option to turn them off.

Do those memories serve any purpose ?

You can look at the scrap book ingame to see what inactives were up to while you weren’t playing them. Other than that. There is no real ingame purpose

You can however upload them to your Sims 3 Profile/facebook to tell stories. You even get a few goodies for uploading them: Memories and the Sims 3 Store. Deleted memories still count towards your total.