ixs' Vodkamelone

Introduction

dss_cli is a small command line program written in Python which can serve as the base for automating tasks on the Open-E Data Storage Server. A sysadmin can use it to control regular maintainance from the shell instead of having to log into the web-interface through a browser.
It can access the existing API via SSH and provides missing functionality by interfacing with the web-server on the DSS appliance. It is using both mechanize and Beautiful Soup to make it resiliant to changes in the webinterface. While it was originally written on a DSS v6, initial tests showed that it mostly works on the DSS v7 release as well.

Background

The "Data Storage Server" from Open-E is a linux based software appliance. After installing the software on a server, the server can then offer NAS and iSCSI storage to attached clients and is manageable through a web-interface.

One interesting feature of the appliance is, that it does offer failover for both iSCSI exported block devices as well as for NFS shared folders, something which makes it very interesting for Bawue.Net. The active/passive failover pair should give us better availability for maintenance as one half of the failover pair can be taken down for maintenance without affecting the virtual machines using the filer as a storage.

During testing of the DSS v6 system we did notice however a certain lack of functionality: The webinterface is great to manage the servers, create volumes, export these and set them up for replication. But using the webinterface is a manual process full of repetitive steps while the tasks at hand call for automation to reduce operator errors and to allow configuration through tools like puppet.

In order to help with automation, the DSS appliance offers an API/CLI access via ssh: Generate a key, connect to the server via ssh and pass some commands:

$ ssh -p 22223 -i filer1.key -l api 192.168.2.220 get_driveslist -v
Unit       Size(GB)   Serial Number  Status
S001       1862.64    4096e40371761527 vg,arc_vol_000
S002       279.40     4096e41532029185 vg,arc_vol_001

Unfortunately, the API is incomplete: It does allow for a lot of automation tasks, it does not export all the functionality to create working failover volumes and destroy them again. If there are plans to use the DSS filer as a storage backend for any kind of automated creation of virtualized servers these functions are sorely needed to prevent the need for manual interaction.

In order to address this lack of functionality, I wrote dss_cli, a command line client aimed at owners and administrators of DSS appliances in order to support all daily administration tasks needed on these filers.

Future plans

Provide a second tool to combine common steps for creation of iSCSI and NAS targets in a cluster.
Otherwise I am also taking nominations for needed functionality.

Installation

The current code is available on GitHub::ixs/dss_cli and is published under the GPLv2. Preqrequisites to running the dss_cli command is a recent Python installation with the Paramiko module for SSH connectivity and mechanize and Beautiful Soup for the web-scraping functionality.

Installation is simple: Download the latest code, unzip it in a new directory and edit config.ini to reflect your environment.
The [failovergroup] section contains your failover pairs, one group per line.
The example below defines one failovergroup called main, containing the dss1 and the dss2 filer.
The [dss1] and [dss2] section define their address, their admin passwords, the ssh_key needed for the API functionality and whether they are the primary or the secondary host in the failover group.

[failovergroups]
main = dss1 dss2

[dss1]
address = 192.168.220.1
password = admin
sshkey = dss1_api.key
mode = primary

[dss2]
address = 192.168.220.2
password = admin
sshkey = dss2_api.key
mode = secondary

Usage

./dss_cli --help
Usage: dss_cli [options]  

Command Line Interface to interact with an Open-E DSS Storage Server

Options:
  -h, --help            show this help message and exit
  -f FILE, --file=FILE  Configuration file to use
  -l, --list            List all commands available
  -g, --failovergroup   List all configured failover groups
  -d, --debug

Use --list to get a list of all supported commands. Each command should
support the --help parameter to get a list of accepted arguments.

Running ./dss-cli -l dss1 does give a list of all commands supported on that device:

$ ./dss_cli -l filer1
build                     - Lists and sets default build.
check_mk_agent            - Returns information from check_mk monitor
create_iscsilv            - Creates a logical iSCSI Volume.
create_naslv              - Creates a logical NAS volume.
date                      - Sets time and date; please use the following format: yyyy-mm-dd hh:mm:ss
failover                  - This function allows you to stop, run or change the operation mode for the given server.
failover_task             - Manage a failover task
get_TXbytes               - Returns total number of bytes transmitted for the given interface.
get_TXpackets             - Returns total number of packets transmitted for the given interface.
get_driveslist            - Fetches a list of drives.
get_hwstatus              - Returns information from system hardware monitor.
get_memorystatus          - Fetches memory status.
get_nichealth             - Fetches the status of the given Network Interface Card.
get_nicslist              - Lists Network Interface Cards.
get_raidstatus            - Returns information about RAID.
help                      - Lists all available methods
iscsi_target_access       - Configure Target IP access
iscsi_target_assign       - Assign lv with given name to existing iSCSI target.
iscsi_target_create       - Creates a new iSCSI target.
iscsi_target_list         - Lists iSCSI targets (syntax: alias;name).
iscsi_target_remove       - Remove an existing iSCSI target
iscsi_target_restart      - Restart iSCSI target service.
iscsi_target_sessions     - Shows and manages iSCSI target sessions.
iscsi_target_status       - Lists the parameters of the selected target.
iscsi_target_unassign     - Unassign from given iSCSI target lvname.
lv_remove                 - Remove a logical volume
nas_settings_http         - Enables and disables access to shares via HTTP.
nas_share_access_afp      - Modifies AFP share access.
nas_share_access_ftp      - Enables and disables access to shares via FTP
nas_share_access_http     - Enables and disables access to shares via HTTP.
nas_share_access_nfs      - Enables and disables access to the given share via NFS.
nas_share_access_smb      - Modifies SMB/AFP share access.
nas_share_create          - Create share on specified volume.
nas_share_details         - Display detailed configuration of share
nas_share_edit            - Changes share location or comment.
nas_share_groups          - Groups manipulation functions.
nas_share_list            - Lists shares
nas_share_remove          - Removes the given share.
nas_share_toggle_smb      - Enable or disable SMB support for a share
nas_share_users           - Users manipulation functions.
nas_user_add              - Create user in the system.
nas_user_groups           - Adding and removing users to groups.
nas_user_remove           - Removes the given user from the system.
nas_user_rename           - Rename NAS user.
ntp                       - Fetches the time and date from an NTP server.
reboot                    - Reboots the system.
set_nic                   - Configures Network Interface Cards.
set_powersettings         - Sets the power button action scheme.
shutdown                  - Shuts the system down.
snapshot_task             - Starts and stops snapshots.
task                      - This function allows you to start task.
test                      - Generates an example of a help message.
unit_manager              - Creates new volume group or adds unit(s) to existing volume group.
update                    - Initiates and checks the status of software update.
version                   - Fetches the software version.
volume_group_status       - Lists Volume Groups.
volume_iscsi_remove       - Removes a logical iSCSI volume
volume_replication        - Adds and removes replication to volume.
volume_replication_mode   - Set volume replication mode to source or destination
volume_replication_remove - Removes replication from Volume
volume_replication_task_create - Create a volume replication task
volume_replication_task_remove - Remove a replication task
volume_replication_task_status - Status of a replication task
volume_replication_task_stop - Stop a replication task
volume_status             - Displays storage info.

Example

The following commands would serve to create a failover iSCSI volume on dss1 and dss2:

Create the logical volumes on both filers as part of the arc_vol_000 volume group. Command line arguments are create_iscsilv <vg_name> <size> blockio
The size argument is specified in 32MB blocks. 150GB * 1024 / 32 = 4800.

$ ./dss_cli filer1 create_iscsilv arc_vol_000 4800 blockio lvarc_vol_00000
$ ./dss_cli filer2 create_iscsilv arc_vol_000 4800 blockio lvarc_vol_00000

Enable volume replication for both logical volumes on both filers and set the logical volume on filer2 to be a secondary volume/replication destination.

$ ./dss_cli filer1 volume_replication add lvarc_vol_00000
$ ./dss_cli filer2 volume_replication add lvarc_vol_00000
$ ./dss_cli filer2 volume_replication_mode lvarc_vol_00000 secondary

Create, start and monitor the replication task on the primary filer and give it 80MBps bandwidth for initial synchronisation.

$ ./dss_cli filer1 volume_replication_task_create lvarc_vol_00000 lvarc_vol_00000 failover_iscsi_target0 80
$ ./dss_cli filer1 task --start VREP failover_iscsi_target0
$ ./dss_cli filer1 volume_replication_task_status failover_iscsi_target0

Create the iSCSI targets on both systems.

$ ./dss_cli filer1 iscsi_target_create target0
$ ./dss_cli filer2 iscsi_target_create target0

Assign the created volume to the just created iSCSI target on both systems. The server will report back with a randomly generated SCSI id for the LUN. Make sure to pass this one when assigning the volume on the secondary system. These ids need to be the same.

$ ./dss_cli filer1 iscsi_target_assign target0 lvarc_vol_00000
lvarc_vol_00000:target0:0:wt:Dgp5VLni08UGb5W5
$ ./dss_cli filer2 iscsi_target_assign target0 lvarc_vol_00000 -s Dgp5VLni08UGb5W5
lvarc_vol_00000:target0:0:wt:Dgp5VLni08UGb5W5

Add the replication task to the list of active failover tasks and make sure that failover services are started.

$ ./dss_cli filer1 failover_task failover_iscsi_target0 enable
$ ./dss_cli filer1 failover --start

Contact

Find me as "ixs" on the usual IRC networks. (IRCnet, EFnet, oftc, freenode)