Zope¶

Description

Hosting and administrative tasks for Zope application server / Plone server.

Introduction
Zope control command
Adding users from command-line (reset admin password)
Timezone
Creating additional debug instances
Virtual hosting
- Supressing virtual host monster
Import and export
Copying a remote site database
Pack and copy big Data.fs
Creating a sanitized data drop
Log rotate

Introduction ¶

This page contains instructions how to configure Zope application server.

Zope control command ¶

The command for Zope tasks is bin/instance in buildout based Plones.

List available command:

bin/instance help

For older Plone releases, the command is zopectl.

Adding users from command-line (reset admin password)¶

You need to do this when you forget admin password or the database is damaged.

Add user with Zope Manager permissions:

bin/instance stop # stop the site first
bin/instance adduser <user_name> <user_password>
bin/instance start

You need to stop the site first.

You also cannot override existing admin user, so you probably want to add admin2.

Timezone ¶

Add to [instance] in buildout.cfg:

zope-conf-additional=

    <environment>
        TZ Europe/Helsinki
    </environment>

Creating additional debug instances ¶

You might want to keep your production buildout.cfg and development configuration in sync automatically as possible.

A good idea is to use same buildout.cfg on every environment. If conditional things, like setting debug mode on, are needed, you can extend the buildout sections, which in turn create "additional Zopes" under bin/ folder:

[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:x
http-address = 8080
debug-mode = off
verbose-security = off

...

environment=
    PTS_LANGUAGES=en fi

#
# Create a launcher script which will start one Zope instance in debug mode
#
[debug-instance]
# Extend the main production instance
<= instance

# Here override specific settings to make the instance run in debug mode
debug-mode = on
verbose-security = on
event-log-level = DEBUG

And now you can start your development Zope as:

bin/debug-instance fg

And your main Zope instance stays in production mode:

bin/instance

Note

Using fg switch foces Zope always into debug mode, but does not concern log level.

Virtual hosting ¶

Zope has a component called VirtualHostMonster which does the virtual host mapping inside the Zope.

Supressing virtual host monster ¶

In the case you have set virtual hosting rules so that Zope does no longer allow you to access the management interface, you can add _SUPPRESS_ACCESSRULE" to the URL to disable VirtualHostMonster.

Import and export ¶

Zope application server offers copying parts of the tree structure via import/export feature. Exported file is basically a Python pickle containing the chosen node and all child nodes.

Importable .zexp files must be placed on /parts/instance/import buildout folder on the server. If you are using clustered ZEO set-up, always run imports through a specific front-end instance by using direct port access. Note that parts folder structure is pruned on each buildout run.

When files are placed on the server to correct folder, Import/Export tab in ZMI will pick them up in the selection drop down. You do not need to restart Zope.

More information

http://quintagroup.com/services/support/tutorials/import-export-plone/

Copying a remote site database ¶

Below is a UNIX shell script to copy a remote Plone site(s) database to your local computer. This is useful for synchronizing the development copy of a site from a live server.

copy-plone-site.sh

#!/bin/sh
#
# Copies a Plone site data from a remote computer to a local computer
#
# Copies
#
# - Data.fs
#
# - blobstorage
#
# Standard var/ folder structure is assumed in the destination
# and the source
#

if [ $# -ne 2 ] ; then
cat <<EOF
$0
Copy a remote Plone site database to local computer over SSH
Error in $0 - Invalid Argument Count
Syntax: $0 [SSH-source to buildout folder] [buildout target folder]
Example: ./copy-plone-site.sh yourserver.com:/srv/plone/mysite .
EOF
exit 64 # Command line usage error
fi

SOURCE=$1
TARGET=$2

STATUS=`$TARGET/bin/instance status`

if [ "$STATUS" != "daemon manager not running" ] ; then
    echo "Please stop your Plone site first"
    exit 1
fi

rsync -av --progress --compress-level=9 "$SOURCE"/var/filestorage/Data.fs "$TARGET"/var/filestorage

# Copy blobstorage on rsync pass
# (We don't need compression for blobs as they most likely are compressed images already)
rsync -av --progress "$SOURCE"/var/blobstorage "$TARGET"/var/

Pack and copy big Data.fs ¶

Pack Data.fs using the pbzip2, efficient multicore bzip2 compressor, before copying:

# Attach to a screen or create new one if not exist so that
# the packing process is not interrupted even if you lose a terminal
screen -x

# The command won't abort in the middle of the run if terminal lost
cd /srv/plone/yoursite/zeocluster/var/filestorage
tar -c --ignore-failed-read Data.fs | pbzip2 -c > /tmp/Data.fs.tar.bz2

# Alternative version using standard bzip2
# tar -c --ignore-failed-read -jf /tmp/Data.fs.tar.bzip2 Data.fs

Then copy to your own computer:

scp unixuser@server.com:/tmp/Data.fs.tar.bz2 .

... or using rsync which can resume:

rsync -av --progress --inplace --partial user@server.com:/tmp/Data.fs.tar.bz2 .

Creating a sanitized data drop ¶

A sanitized data drop is a Plone site where:

all user passwords have been reset to one known one;
all history information is deleted (packed), so that it does not contain anything sensitive;
other possible sensitive data has been removed.

It should safe to give a sanitized copy to a third party.

Below is a sample script which will clean a Plone site in-place.

Note

Because sensitive data varies depending on your site this script is just an example.

How to use:

Create a temporary copy of your Plone site on your server, running in a different port.
Run the cleaner by entering the URL. It is useful to run the temporary copy in foreground to follow the progress.
Give the sanitized copy away.

This script has two options for purging data:

Safe purge using the Plone API (slow, calls all event handlers).
Unsafe purge by directly pruning data, rebuilding the catalog without triggering the event handlers.

The sample clean.py:

""" Pack Plone database size and clean sensitive data.
    This makes output ideal as a developent drop.

    It also resets all kinds of users password to "admin".

    Limitations:

    1) Assumes only one site per Data.fs

    TODO: Remove users unless they are whitelisted.

"""

import logging
import transaction

logger = logging.getLogger("cleaner")

# Folders which entries are cleared
DELETE_POINTS = """
intranet/mydata

"""
# Save these folder entries as sampple
WHITELIST = """
intranet/mydata/sample-page
"""

# All users will receive this new password
PASSWORD="123123"

def is_white_listed(path):
    """
    """
    paths = [ s.strip() for s in WHITELIST.split("\n") if s.strip() != ""]

    if path in paths:
        return True
    return False

def purge(site):
    """
    Purge the site using standard Plone deleting mechanism (slow)
    """
    i = 0
    for dp in DELETE_POINTS.split("\n"):

        dp = dp.string()
        if dp == "":
            continue

        folder = site.unrestrictedTraverse(dp)

        for id in folder.objectIds():
            full_path = dp + "/" + id
            if not is_white_listed(full_path):
                logger.info("Deleting path:" + full_path)
                try:
                    folder.manage_delObjects([id])
                except Exception, e:
                    # Bad delete handling code - e.g. catalog indexes b0rk out
                    logger.error("*** COULD NOT DELETE ***")
                    logger.exception(e)
                i += 1
                if i % 100 == 0:
                    transaction.commit()

def purge_harder(site):
    """
    Purge using forced delete and then catalog rebuild.

    Might be faster if a lot of content.
    """
    i = 0

    logger.info("Kill it with fire")
    for dp in DELETE_POINTS.split("\n"):

        if dp.strip() == "":
            continue
        folder = site.unrestrictedTraverse(dp)

        for id in folder.objectIds():
            full_path = dp + "/" + id
            if not is_white_listed(full_path):
                logger.info("Hard deleting path:" + full_path)
                # http://collective-docs.readthedocs.org/en/latest/content/deleting.html#fail-safe-deleting
                folder._delObject(id, suppress_events=True)

                i += 1
                if i % 100 == 0:
                    transaction.commit()

    site.portal_catalog.clearFindAndRebuild()


def pack(app):
    """
    @param app Zope application server root
    """
    logger.info("Packing database")
    cpanel = app.unrestrictedTraverse('/Control_Panel')
    cpanel.manage_pack(days=0, REQUEST=None)

def change_zope_passwords(app):
    """
    """
    logger.info("Changing Zope passwords")
    # Products.PluggableAuthService.plugins.ZODBUserManager
    users = app.acl_users.users
    for id in users.listUserIds():
        users.updateUserPassword(id, PASSWORD)

def change_site_passwords(site):
    """
    """
    logger.info("Changing Plone instance passwords")
    # Products.PlonePAS.plugins.ufactory
    users = site.acl_users.source_users
    for id in users.getUserIds():
        users.doChangeUser(id, PASSWORD)

def change_membrane_password(site):
    """
    Reset membrane passwords (if membrane installed)
    """

    if not "membrane_users" in site.acl_users.objectIds():
        return

    logger.info("Changing membrane passwords")
    # Products.PlonePAS.plugins.ufactory
    users = site.acl_users.membrane_users
    for id in users.getUserNames():
        try:
            users.doChangeUser(id, PASSWORD)
        except:
            # XXX: We should actually delete membrane users before content folders
            # or we will break here
            pass

class Cleaner(object):
    """
    Clean the current Plone site for sensitive data.

    Usage::

        http://localhost:8080/site/@@create-sanitized-copy

    or::

        http://localhost:8080/site/@@create-sanitized-copy?pack=false

    """

    def __init__(self, context, request):
        self.context = context
        self.request = request

    def __call__(self):
        """
        """
        app = self.context.restrictedTraverse('/') # Zope application server root
        site = self.context.portal_url.getPortalObject()

        purge_harder(site)
        change_zope_passwords(app)
        change_site_passwords(site)
        #change_membrane_password(site)

        if self.request.form.get("pack", None) != "false":
            pack(app)

        # Obligatory Die Hard quote
        return "Yippikayee m%&€/ f/€%&/€%&/ Remember to login again with new password."

Example view registration in ZCML requiring admin privileges to run the cleaner:

<browser:page
 for="Products.CMFCore.interfaces.ISiteRoot"
 name="create-sanitized-copy"
 class=".clean.Cleaner"
 permission="cmf.ManagePortal"
/>

Log rotate ¶

Log rotation prevents log files from growing indefinitely by creating a new file for a certain timespan and dropping old files.

The unix tool logrotate is used for log rotation.

You need to rotate Zope access and error logs, plus possible front-end web server logs. The latter is usually taken care of your operating system.

To set-up log rotation for Plone:

Install logrotate on the system (if you don't already have one).
You need to know the effective UNIX user as which Plone processes run.
Edit log rotation configuration files to include Plone log directories.
Do a test run.

To add a log rotation configuration file for Plone add a file /etc/logrotate.d/yoursite as root.

Note

This recipe applies only for single-process Zope installs. If you use ZEO clustering you need to do this little bit differently.

The file contains:

# This is the path + selector for the log files
/srv/plone/yoursite/Plone/zinstance/var/log/instance*.log {
        daily
        missingok
        # How many days to keep logs
        # In our cases 60 days
        rotate 60
        compress
        delaycompress
        notifempty
        # File owner and permission for rotated files
        # For additional safety this can be a different
        # user so your Plone UNIX user cannot
        # delete logs
        create 640 root root

        # This signal will tell Zope to open a new file-system inode for the log file
        # so it doesn't keep reserving the old log file handle for evenif the file is deleted
        postrotate
            [ ! -f /srv/plone/yoursite/Plone/zinstance/var/instance.pid ] || kill -USR2 `cat /srv/plone/yoursite/Plone/zinstance/var/instance.pid`
        endscript
}

Then do a test run of logrotate, as root:

# -f = force rotate
# -d = debug mode
logrotate -f -d /etc/logrotate.conf

And if you want to see the results right away:

# -f = force rotate
logrotate -f /etc/logrotate.conf

In normal production, logrotate is added to your operating system crontab for daily runs automatically.

More info:

Log rotate and chroot ¶

chroot'ed environments don't usually get their own cron. In this case you can trigger the log rotate from the parent system.

Add in the parent /etc/cron.daily/yourchrootname-logrotate

#!/bin/sh
schroot -c yoursitenet -u root -r logrotate /etc/logrotate.conf

Log rotate generation via buildout using UNIX logrotate command ¶

buildout.cfg:

[logrotate]
recipe = collective.recipe.template
input =  ${buildout:directory}/templates/logrotate.conf
output = ${buildout:directory}/etc/logrotate.conf

templates/logrotate.conf:

rotate 4
weekly
create
compress
delaycompress
missingok

${buildout:directory}/var/log/instance1.log ${buildout:directory}/var/log/instance1-Z2.log {
    sharedscripts
    postrotate
        /bin/kill -USR2 $(cat ${buildout:directory}/var/instance1.pid)
    endscript
}

${buildout:directory}/var/log/instance2.log ${buildout:directory}/var/log/instance2-Z2.log {
    sharedscripts
    postrotate
        /bin/kill -USR2 $(cat ${buildout:directory}/var/instance2.pid)
    endscript
}

More info:

http://stackoverflow.com/a/9437677/315168

Log rotate on Windows ¶

Use iw.rotatezlogs

http://stackoverflow.com/a/9434150/315168

Email notifications for errors ¶

Please see:

http://stackoverflow.com/questions/5993334/error-notification-on-plone-4

Zope¶

Introduction ¶

Zope control command ¶

Adding users from command-line (reset admin password)¶

Timezone ¶

Creating additional debug instances ¶

Virtual hosting ¶

Supressing virtual host monster ¶

Import and export ¶

Copying a remote site database ¶

Pack and copy big Data.fs ¶

Creating a sanitized data drop ¶

Log rotate ¶

Log rotate and chroot ¶

Log rotate generation via buildout using UNIX logrotate command ¶

Log rotate on Windows ¶

Email notifications for errors ¶

Table Of Contents

About Plone

Edit this document

Navigation

Zope¶

Quick search

About Plone

Navigation

Edit this document