Deleting

Description

Deleting content items in Plone programmatically. How link integrity checks works and how to avoid it.

Introduction

This document tells how to programmatically delete objects in Plone.

Deleting content by id

Deleting content objects is done by IObjectManager.

IObjectManager definition.

Example:

# manage_delObjects takes list of ids as an argument
folder.manage_delObjects(["list", "of", "ids", "to", "delete"])

Or:

parent = context.aq_parent
parent.manage_delObjects([context.getId()])

Permissions

The user must have Zope 2 Delete objects permission on the content item being deleted. This is checked in Products.CMFPlone.PloneFolder.manage_delObjects().

Otherwise Unauthorized exception is risen.

Example how to check for this permission:

from Products.CMFCore import permissions

hospital = self.portal.country.hospital
item = hospital.patient1

mt = getToolByName(self.portal, 'portal_membership')
if mt.checkPermission(permissions.DeleteObjects, item):
    # Can delete
    raise AssertionError("Oooops. Deletion allowed")
else:
    pass

Bypassing permissios

This is handy if you work e.g. in a debug shell and you are deleting badly behaving objects.

from AccessControl.SecurityManagement import newSecurityManager
admin=app.acl_users.getUserById("admin")
app.folder_sits.sitsngta.manage_delObjects("examples")
# Try harder:
# app.folder_sits.sitsngta._delObject("examples", suppress_events=True)
import transaction ; transaction.commit()

Deleting all content in a folder

Little tricky. An example:

    ids = folder.objectIds() # Plone 3 or older
    ids = folder.keys()      # Plone 4 or newer

if len(ids) > 0:
    # manage_delObject will mutate the list
    # so we cannot give it tuple returned by objectIds()
    ids = list(ids)
    folder.manage_delObjects(ids)

Bypassing link integrity check

If link integrity check is on in the site setup, you cannot delete objects which themselves are link targets or their children are link targets.

Instead, a LinkIntegrityException is raised. The LinkIntegrityException constains information about objects referring to the content which is not allowed to delete.

plone.app.linkintegrity.browser.remote module contains code which allows you to delete the object in any case. It catches the exception, modifies the HTTP request to contain a marker interface allowing delete to happen and then replays the transaction.

In the case the link integrity check fails for manage_delObjects(), you will be shown a confirmation dialog. The orignal request payload gots pickled and is stored in HTML form as an encoded.

When the user presses confirm, the orignal request gets unpickled from HTTP POST payload. Then the view modifies Zope publisher so that it will play the orignal unpickled HTTP POST with the marker interface "Do not care about link integrity breaches" turned on.

Here is an sample batch delete code which tries to work around the link integrity check:

from zope.component import queryUtility
from Products.CMFCore.interfaces import IPropertiesTool

# We need to disable link integrity check,
# because it cannot handle several delete calls in
# one request
ptool = queryUtility(IPropertiesTool)
props = getattr(ptool, 'site_properties', None)
old_check = props.getProperty('enable_link_integrity_checks', False)
props.enable_link_integrity_checks = False

for b in items:
    count += 1
    obj = b.getObject()
    logger.info("Deleting:" + obj.absolute_url() + " " + str(obj.created()))

    try:
        obj.aq_parent.manage_delObjects([obj.getId()])
    except Exception, e:
        # E.g. linkintegrityerror or some other
        logger.error("Could not remove item:" + obj.absolute_url())
        logger.exception(e)
        continue

    if count % transaction_threshold == 0:
        # Prevent transaction becoming too large (memory buffer)
        # by committing now and then
        logger.info("Committing transaction")
        transaction.commit()

props.enable_link_integrity_checks = old_check

logger.info(msg)

Fail safe deleting

Sometimes object delete might not be possible, because deletion dispatches events which might raise exception due to bad broken objects or badly behaving code.

OFS.ObjectManager which is base class for Zope folders, provides internal method to delete objects from folder without firing any events:

# Delete object with id "broken-folder" without firing any delete events
site._delObject("broken-folder", suppress_events=True)

The best way to clean up bad objects on your site is via command line script in which case remember to commit the transaction after removing the broken objects.

Purging site from old content

This ZMI script allows you to find content items of certain type and delete if they are created too long ago.

# Delete FeedfeederItem content items which are more than three months old

from StringIO import StringIO
import DateTime

buf = StringIO()

# DateTime deltas are days as floating points
end = DateTime.DateTime() - 30*3
start = DateTime.DateTime(2000, 1,1)

date_range_query = { 'query':(start,end), 'range': 'min:max'}

items = context.portal_catalog.queryCatalog({"portal_type":"FeedFeederItem",
                                     "created" : date_range_query,
                                     "sort_on" : "created"
                                    })

items = list(items)

print >> buf, "Found %d items to be purged" % len(items)

count = 0
for b in items:
    count += 1
    obj = b.getObject()
    print >> buf, "Deleting:" + obj.absolute_url() + " " + str(obj.created())
    obj.aq_parent.manage_delObjects([obj.getId()])


return buf.getvalue()

Below is an advanced version for old item date based deletion code which issuitable for huge sites. This snippet is from Products.feedfeeder package. It will look for Feedfeeder items (automatically generated from RSS) which are older than X days and delete them.

It's based on Zope 3 page registration (sidenote: I noticed that views do not need to be based on BrowserView page class).

• Transaction thresholds make sure the code runs faster
• Logging to Plone event log files
• Number of days to look into past is not hardcoded
• Manage rights needed to execute the code

You can call this view like:

http://localhost:9999/plonecommunity/@@feed-mega-cleanup?days=90

Here is the view Python source code:

import logging

import transaction
from zope import interface
from zope import component
import DateTime
import zExceptions

logger = logging.getLogger("feedfeeder")

class MegaClean(object):
    """ Clean-up old feed items by deleting them on the site.

    This is intended to be called from cron weekly.
    """

    def __init__(self, context, request):
        self.context = context
        self.request = request

    def clean(self, days, transaction_threshold=100):
        """ Perform the clean-up by looking old objects and deleting them.

        Commit ZODB transaction for every N objects to that commit buffer does not grow
        too long (timewise, memory wise).

        @param days: if item has been created before than this many days ago it is deleted

        @param transaction_threshold: How often we commit - for every nth item
        """

        logger.info("Beginning feed clean up process")

        context = self.context.aq_inner
        count = 0


        # DateTime deltas are days as floating points
        end = DateTime.DateTime() - days
        start = DateTime.DateTime(2000, 1,1)

        date_range_query = { 'query':(start,end), 'range': 'min:max'}

        items = context.portal_catalog.queryCatalog({"portal_type":"FeedFeederItem",
                                             "created" : date_range_query,
                                             "sort_on" : "created"
                                            })

        items = list(items)

        logger.info("Found %d items to be purged" % len(items))

        for b in items:
            count += 1
            obj = b.getObject()
            logger.info("Deleting:" + obj.absolute_url() + " " + str(obj.created()))
            obj.aq_parent.manage_delObjects([obj.getId()])

            if count % transaction_threshold == 0:
                # Prevent transaction becoming too large (memory buffer)
                # by committing now and then
                logger.info("Committing transaction")
                transaction.commit()

        msg = "Total %d items removed" % count
        logger.info(msg)

        return msg

    def __call__(self):

        days = self.request.form.get("days", None)
        if not days:
            raise zExceptions.InternalError("Bad input. Please give days=60 as HTTP GET query parameter")

        days = int(days)

        return self.clean(days)

Then we have the view ZCML registration:

<page
    name="feed-mega-cleanup"
    for="Products.CMFCore.interfaces.ISiteRoot"
    permission="cmf.ManagePortal"
    class=".feed.MegaClean"
    />