Package CedarBackup2 :: Module peer :: Class RemotePeer
[hide private]
[frames] | no frames]

Class RemotePeer

source code

object --+
         |
        RemotePeer

Backup peer representing a remote peer in a backup pool.

This is a class representing a remote (networked) peer in a backup pool. Remote peers are backed up using an rcp-compatible copy command. A remote peer has associated with it a name (which must be a valid hostname), a collect directory, a working directory and a copy method (an rcp-compatible command).

You can also set an optional local user value. This username will be used as the local user for any remote copies that are required. It can only be used if the root user is executing the backup. The root user will su to the local user and execute the remote copies as that user.

The copy method is associated with the peer and not with the actual request to copy, because we can envision that each remote host might have a different connect method.

The public methods other than the constructor are part of a "backup peer" interface shared with the LocalPeer class.

Instance Methods [hide private]
 
__init__(self, name=None, collectDir=None, workingDir=None, remoteUser=None, rcpCommand=None, localUser=None, rshCommand=None, cbackCommand=None, ignoreFailureMode=None)
Initializes a remote backup peer.
source code
 
stagePeer(self, targetDir, ownership=None, permissions=None)
Stages data from the peer into the indicated local target directory.
source code
 
checkCollectIndicator(self, collectIndicator=None)
Checks the collect indicator in the peer's staging directory.
source code
 
writeStageIndicator(self, stageIndicator=None)
Writes the stage indicator in the peer's staging directory.
source code
 
executeRemoteCommand(self, command)
Executes a command on the peer via remote shell.
source code
 
executeManagedAction(self, action, fullBackup)
Executes a managed action on this peer.
source code
 
_setName(self, value)
Property target used to set the peer name.
source code
 
_getName(self)
Property target used to get the peer name.
source code
 
_setCollectDir(self, value)
Property target used to set the collect directory.
source code
 
_getCollectDir(self)
Property target used to get the collect directory.
source code
 
_setWorkingDir(self, value)
Property target used to set the working directory.
source code
 
_getWorkingDir(self)
Property target used to get the working directory.
source code
 
_setRemoteUser(self, value)
Property target used to set the remote user.
source code
 
_getRemoteUser(self)
Property target used to get the remote user.
source code
 
_setLocalUser(self, value)
Property target used to set the local user.
source code
 
_getLocalUser(self)
Property target used to get the local user.
source code
 
_setRcpCommand(self, value)
Property target to set the rcp command.
source code
 
_getRcpCommand(self)
Property target used to get the rcp command.
source code
 
_setRshCommand(self, value)
Property target to set the rsh command.
source code
 
_getRshCommand(self)
Property target used to get the rsh command.
source code
 
_setCbackCommand(self, value)
Property target to set the cback command.
source code
 
_getCbackCommand(self)
Property target used to get the cback command.
source code
 
_setIgnoreFailureMode(self, value)
Property target used to set the ignoreFailure mode.
source code
 
_getIgnoreFailureMode(self)
Property target used to get the ignoreFailure mode.
source code

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__

Static Methods [hide private]
 
_getDirContents(path)
Returns the contents of a directory in terms of a Set.
source code
 
_copyRemoteDir(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceDir, targetDir, ownership=None, permissions=None)
Copies files from the source directory to the target directory.
source code
 
_copyRemoteFile(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceFile, targetFile, ownership=None, permissions=None, overwrite=True)
Copies a remote source file to a target file.
source code
 
_pushLocalFile(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceFile, targetFile, overwrite=True)
Copies a local source file to a remote host.
source code
 
_executeRemoteCommand(remoteUser, localUser, remoteHost, rshCommand, rshCommandList, remoteCommand)
Executes a command on the peer via remote shell.
source code
 
_buildCbackCommand(cbackCommand, action, fullBackup)
Builds a Cedar Backup command line for the named action.
source code
Properties [hide private]
  name
Name of the peer (a valid DNS hostname).
  collectDir
Path to the peer's collect directory (an absolute local path).
  remoteUser
Name of the Cedar Backup user on the remote peer.
  rcpCommand
An rcp-compatible copy command to use for copying files.
  rshCommand
An rsh-compatible command to use for remote shells to the peer.
  cbackCommand
A chack-compatible command to use for executing managed actions.
  workingDir
Path to the peer's working directory (an absolute local path).
  localUser
Name of the Cedar Backup user on the current host.
  ignoreFailureMode
Ignore failure mode for peer.

Inherited from object: __class__

Method Details [hide private]

__init__(self, name=None, collectDir=None, workingDir=None, remoteUser=None, rcpCommand=None, localUser=None, rshCommand=None, cbackCommand=None, ignoreFailureMode=None)
(Constructor)

source code 

Initializes a remote backup peer.

Parameters:
  • name (String, must be a valid DNS hostname) - Name of the backup peer
  • collectDir (String representing an absolute path on the remote peer) - Path to the peer's collect directory
  • workingDir (String representing an absolute path on the current host.) - Working directory that can be used to create temporary files, etc.
  • remoteUser (String representing a username, valid via remote shell to the peer) - Name of the Cedar Backup user on the remote peer
  • localUser (String representing a username, valid on the current host) - Name of the Cedar Backup user on the current host
  • rcpCommand (String representing a system command including required arguments) - An rcp-compatible copy command to use for copying files from the peer
  • rshCommand (String representing a system command including required arguments) - An rsh-compatible copy command to use for remote shells to the peer
  • cbackCommand (String representing a system command including required arguments) - A chack-compatible command to use for executing managed actions
  • ignoreFailureMode (One of VALID_FAILURE_MODES) - Ignore failure mode for this peer
Raises:
  • ValueError - If collect directory is not an absolute path
Overrides: object.__init__

Note: If provided, each command will eventually be parsed into a list of strings suitable for passing to util.executeCommand in order to avoid security holes related to shell interpolation. This parsing will be done by the util.splitCommandLine function. See the documentation for that function for some important notes about its limitations.

stagePeer(self, targetDir, ownership=None, permissions=None)

source code 

Stages data from the peer into the indicated local target directory.

The target directory must already exist before this method is called. If passed in, ownership and permissions will be applied to the files that are copied.

Parameters:
  • targetDir (String representing a directory on disk) - Target directory to write data into
  • ownership (Tuple of numeric ids (uid, gid)) - Owner and group that the staged files should have
  • permissions (UNIX permissions mode, specified in octal (i.e. 0640).) - Permissions that the staged files should have
Returns:
Number of files copied from the source directory to the target directory.
Raises:
  • ValueError - If target directory is not a directory, does not exist or is not absolute.
  • ValueError - If a path cannot be encoded properly.
  • IOError - If there were no files to stage (i.e. the directory was empty)
  • IOError - If there is an IO error copying a file.
  • OSError - If there is an OS error copying or changing permissions on a file
Notes:
  • The returned count of copied files might be inaccurate if some of the copied files already existed in the staging directory prior to the copy taking place. We don't clear the staging directory first, because some extension might also be using it.
  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.
  • Unlike the local peer version of this method, an I/O error might or might not be raised if the directory is empty. Since we're using a remote copy method, we just don't have the fine-grained control over our exceptions that's available when we can look directly at the filesystem, and we can't control whether the remote copy method thinks an empty directory is an error.

checkCollectIndicator(self, collectIndicator=None)

source code 

Checks the collect indicator in the peer's staging directory.

When a peer has completed collecting its backup files, it will write an empty indicator file into its collect directory. This method checks to see whether that indicator has been written. If the remote copy command fails, we return False as if the file weren't there.

If you need to, you can override the name of the collect indicator file by passing in a different name.

Parameters:
  • collectIndicator (String representing name of a file in the collect directory) - Name of the collect indicator file to check
Returns:
Boolean true/false depending on whether the indicator exists.
Raises:
  • ValueError - If a path cannot be encoded properly.

Note: Apparently, we can't count on all rcp-compatible implementations to return sensible errors for some error conditions. As an example, the scp command in Debian 'woody' returns a zero (normal) status even when it can't find a host or if the login or path is invalid. Because of this, the implementation of this method is rather convoluted.

writeStageIndicator(self, stageIndicator=None)

source code 

Writes the stage indicator in the peer's staging directory.

When the master has completed collecting its backup files, it will write an empty indicator file into the peer's collect directory. The presence of this file implies that the staging process is complete.

If you need to, you can override the name of the stage indicator file by passing in a different name.

Parameters:
  • stageIndicator (String representing name of a file in the collect directory) - Name of the indicator file to write
Raises:
  • ValueError - If a path cannot be encoded properly.
  • IOError - If there is an IO error creating the file.
  • OSError - If there is an OS error creating or changing permissions on the file

Note: If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.

executeRemoteCommand(self, command)

source code 

Executes a command on the peer via remote shell.

Parameters:
  • command (String command-line suitable for use with rsh.) - Command to execute
Raises:
  • IOError - If there is an error executing the command on the remote peer.

executeManagedAction(self, action, fullBackup)

source code 

Executes a managed action on this peer.

Parameters:
  • action - Name of the action to execute.
  • fullBackup - Whether a full backup should be executed.
Raises:
  • IOError - If there is an error executing the action on the remote peer.

_getDirContents(path)
Static Method

source code 

Returns the contents of a directory in terms of a Set.

The directory's contents are read as a FilesystemList containing only files, and then the list is converted into a set object for later use.

Parameters:
  • path (String representing a path on disk) - Directory path to get contents for
Returns:
Set of files in the directory
Raises:
  • ValueError - If path is not a directory or does not exist.

_copyRemoteDir(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceDir, targetDir, ownership=None, permissions=None)
Static Method

source code 

Copies files from the source directory to the target directory.

This function is not recursive. Only the files in the directory will be copied. Ownership and permissions will be left at their default values if new values are not specified. Behavior when copying soft links from the collect directory is dependent on the behavior of the specified rcp command.

Parameters:
  • remoteUser (String representing a username, valid via the copy command) - Name of the Cedar Backup user on the remote peer
  • localUser (String representing a username, valid on the current host) - Name of the Cedar Backup user on the current host
  • remoteHost (String representing a hostname, accessible via the copy command) - Hostname of the remote peer
  • rcpCommand (String representing a system command including required arguments) - An rcp-compatible copy command to use for copying files from the peer
  • rcpCommandList (Command as a list to be passed to util.executeCommand) - An rcp-compatible copy command to use for copying files
  • sourceDir (String representing a directory on disk) - Source directory
  • targetDir (String representing a directory on disk) - Target directory
  • ownership (Tuple of numeric ids (uid, gid)) - Owner and group that the copied files should have
  • permissions (UNIX permissions mode, specified in octal (i.e. 0640).) - Permissions that the staged files should have
Returns:
Number of files copied from the source directory to the target directory.
Raises:
  • ValueError - If source or target is not a directory or does not exist.
  • IOError - If there is an IO error copying the files.
Notes:
  • The returned count of copied files might be inaccurate if some of the copied files already existed in the staging directory prior to the copy taking place. We don't clear the staging directory first, because some extension might also be using it.
  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.
  • We don't have a good way of knowing exactly what files we copied down from the remote peer, unless we want to parse the output of the rcp command (ugh). We could change permissions on everything in the target directory, but that's kind of ugly too. Instead, we use Python's set functionality to figure out what files were added while we executed the rcp command. This isn't perfect - for instance, it's not correct if someone else is messing with the directory at the same time we're doing the remote copy - but it's about as good as we're going to get.
  • Apparently, we can't count on all rcp-compatible implementations to return sensible errors for some error conditions. As an example, the scp command in Debian 'woody' returns a zero (normal) status even when it can't find a host or if the login or path is invalid. We try to work around this by issuing IOError if we don't copy any files from the remote host.

_copyRemoteFile(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceFile, targetFile, ownership=None, permissions=None, overwrite=True)
Static Method

source code 

Copies a remote source file to a target file.

Parameters:
  • remoteUser (String representing a username, valid via the copy command) - Name of the Cedar Backup user on the remote peer
  • remoteHost (String representing a hostname, accessible via the copy command) - Hostname of the remote peer
  • localUser (String representing a username, valid on the current host) - Name of the Cedar Backup user on the current host
  • rcpCommand (String representing a system command including required arguments) - An rcp-compatible copy command to use for copying files from the peer
  • rcpCommandList (Command as a list to be passed to util.executeCommand) - An rcp-compatible copy command to use for copying files
  • sourceFile (String representing a file on disk, as an absolute path) - Source file to copy
  • targetFile (String representing a file on disk, as an absolute path) - Target file to create
  • ownership (Tuple of numeric ids (uid, gid)) - Owner and group that the copied should have
  • permissions (UNIX permissions mode, specified in octal (i.e. 0640).) - Permissions that the staged files should have
  • overwrite (Boolean true/false.) - Indicates whether it's OK to overwrite the target file.
Raises:
  • IOError - If the target file already exists.
  • IOError - If there is an IO error copying the file
  • OSError - If there is an OS error changing permissions on the file
Notes:
  • Internally, we have to go through and escape any spaces in the source path with double-backslash, otherwise things get screwed up. It doesn't seem to be required in the target path. I hope this is portable to various different rcp methods, but I guess it might not be (all I have to test with is OpenSSH).
  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.
  • We will not overwrite a target file that exists when this method is invoked. If the target already exists, we'll raise an exception.
  • Apparently, we can't count on all rcp-compatible implementations to return sensible errors for some error conditions. As an example, the scp command in Debian 'woody' returns a zero (normal) status even when it can't find a host or if the login or path is invalid. We try to work around this by issuing IOError the target file does not exist when we're done.

_pushLocalFile(remoteUser, localUser, remoteHost, rcpCommand, rcpCommandList, sourceFile, targetFile, overwrite=True)
Static Method

source code 

Copies a local source file to a remote host.

Parameters:
  • remoteUser (String representing a username, valid via the copy command) - Name of the Cedar Backup user on the remote peer
  • localUser (String representing a username, valid on the current host) - Name of the Cedar Backup user on the current host
  • remoteHost (String representing a hostname, accessible via the copy command) - Hostname of the remote peer
  • rcpCommand (String representing a system command including required arguments) - An rcp-compatible copy command to use for copying files from the peer
  • rcpCommandList (Command as a list to be passed to util.executeCommand) - An rcp-compatible copy command to use for copying files
  • sourceFile (String representing a file on disk, as an absolute path) - Source file to copy
  • targetFile (String representing a file on disk, as an absolute path) - Target file to create
  • overwrite (Boolean true/false.) - Indicates whether it's OK to overwrite the target file.
Raises:
  • IOError - If there is an IO error copying the file
  • OSError - If there is an OS error changing permissions on the file
Notes:
  • We will not overwrite a target file that exists when this method is invoked. If the target already exists, we'll raise an exception.
  • Internally, we have to go through and escape any spaces in the source and target paths with double-backslash, otherwise things get screwed up. I hope this is portable to various different rcp methods, but I guess it might not be (all I have to test with is OpenSSH).
  • If you have user/group as strings, call the util.getUidGid function to get the associated uid/gid as an ownership tuple.

_setName(self, value)

source code 

Property target used to set the peer name. The value must be a non-empty string and cannot be None.

Raises:
  • ValueError - If the value is an empty string or None.

_setCollectDir(self, value)

source code 

Property target used to set the collect directory. The value must be an absolute path and cannot be None. It does not have to exist on disk at the time of assignment.

Raises:
  • ValueError - If the value is None or is not an absolute path.
  • ValueError - If the value cannot be encoded properly.

_setWorkingDir(self, value)

source code 

Property target used to set the working directory. The value must be an absolute path and cannot be None.

Raises:
  • ValueError - If the value is None or is not an absolute path.
  • ValueError - If the value cannot be encoded properly.

_setRemoteUser(self, value)

source code 

Property target used to set the remote user. The value must be a non-empty string and cannot be None.

Raises:
  • ValueError - If the value is an empty string or None.

_setLocalUser(self, value)

source code 

Property target used to set the local user. The value must be a non-empty string if it is not None.

Raises:
  • ValueError - If the value is an empty string.

_setRcpCommand(self, value)

source code 

Property target to set the rcp command.

The value must be a non-empty string or None. Its value is stored in the two forms: "raw" as provided by the client, and "parsed" into a list suitable for being passed to util.executeCommand via util.splitCommandLine.

However, all the caller will ever see via the property is the actual value they set (which includes seeing None, even if we translate that internally to DEF_RCP_COMMAND). Internally, we should always use self._rcpCommandList if we want the actual command list.

Raises:
  • ValueError - If the value is an empty string.

_setRshCommand(self, value)

source code 

Property target to set the rsh command.

The value must be a non-empty string or None. Its value is stored in the two forms: "raw" as provided by the client, and "parsed" into a list suitable for being passed to util.executeCommand via util.splitCommandLine.

However, all the caller will ever see via the property is the actual value they set (which includes seeing None, even if we translate that internally to DEF_RSH_COMMAND). Internally, we should always use self._rshCommandList if we want the actual command list.

Raises:
  • ValueError - If the value is an empty string.

_setCbackCommand(self, value)

source code 

Property target to set the cback command.

The value must be a non-empty string or None. Unlike the other command, this value is only stored in the "raw" form provided by the client.

Raises:
  • ValueError - If the value is an empty string.

_setIgnoreFailureMode(self, value)

source code 

Property target used to set the ignoreFailure mode. If not None, the mode must be one of the values in VALID_FAILURE_MODES.

Raises:
  • ValueError - If the value is not valid.

_executeRemoteCommand(remoteUser, localUser, remoteHost, rshCommand, rshCommandList, remoteCommand)
Static Method

source code 

Executes a command on the peer via remote shell.

Parameters:
  • remoteUser (String representing a username, valid on the remote host) - Name of the Cedar Backup user on the remote peer
  • localUser (String representing a username, valid on the current host) - Name of the Cedar Backup user on the current host
  • remoteHost (String representing a hostname, accessible via the copy command) - Hostname of the remote peer
  • rshCommand (String representing a system command including required arguments) - An rsh-compatible copy command to use for remote shells to the peer
  • rshCommandList (Command as a list to be passed to util.executeCommand) - An rsh-compatible copy command to use for remote shells to the peer
  • remoteCommand (String command-line, with no special shell characters ($, <, etc.)) - The command to be executed on the remote host
Raises:
  • IOError - If there is an error executing the remote command

_buildCbackCommand(cbackCommand, action, fullBackup)
Static Method

source code 

Builds a Cedar Backup command line for the named action.

Parameters:
  • cbackCommand - cback command to execute, including required options
  • action - Name of the action to execute.
  • fullBackup - Whether a full backup should be executed.
Returns:
String suitable for passing to _executeRemoteCommand as remoteCommand.
Raises:
  • ValueError - If action is None.

Note: If the cback command is None, then DEF_CBACK_COMMAND is used.


Property Details [hide private]

name

Name of the peer (a valid DNS hostname).

Get Method:
_getName(self) - Property target used to get the peer name.
Set Method:
_setName(self, value) - Property target used to set the peer name.

collectDir

Path to the peer's collect directory (an absolute local path).

Get Method:
_getCollectDir(self) - Property target used to get the collect directory.
Set Method:
_setCollectDir(self, value) - Property target used to set the collect directory.

remoteUser

Name of the Cedar Backup user on the remote peer.

Get Method:
_getRemoteUser(self) - Property target used to get the remote user.
Set Method:
_setRemoteUser(self, value) - Property target used to set the remote user.

rcpCommand

An rcp-compatible copy command to use for copying files.

Get Method:
_getRcpCommand(self) - Property target used to get the rcp command.
Set Method:
_setRcpCommand(self, value) - Property target to set the rcp command.

rshCommand

An rsh-compatible command to use for remote shells to the peer.

Get Method:
_getRshCommand(self) - Property target used to get the rsh command.
Set Method:
_setRshCommand(self, value) - Property target to set the rsh command.

cbackCommand

A chack-compatible command to use for executing managed actions.

Get Method:
_getCbackCommand(self) - Property target used to get the cback command.
Set Method:
_setCbackCommand(self, value) - Property target to set the cback command.

workingDir

Path to the peer's working directory (an absolute local path).

Get Method:
_getWorkingDir(self) - Property target used to get the working directory.
Set Method:
_setWorkingDir(self, value) - Property target used to set the working directory.

localUser

Name of the Cedar Backup user on the current host.

Get Method:
_getLocalUser(self) - Property target used to get the local user.
Set Method:
_setLocalUser(self, value) - Property target used to set the local user.

ignoreFailureMode

Ignore failure mode for peer.

Get Method:
_getIgnoreFailureMode(self) - Property target used to get the ignoreFailure mode.
Set Method:
_setIgnoreFailureMode(self, value) - Property target used to set the ignoreFailure mode.