REX-Ray

Openly serious about storage

---

REX-Ray delivers persistent storage access for container runtimes, such as Docker and Mesos, and provides an easy interface for enabling advanced storage functionality across common storage, virtualization and cloud platforms. For example, here's how to list the volumes for a VirtualBox VM running Linux with REX-Ray:

$ rexray volume --service virtualbox
- attachments:
  - instanceID:
      id: e71578b0-1bfb-4fa5-bcd5-4ae982fd4a9b
      driver: virtualbox
    status: /Users/akutz/VirtualBox/libStorage/libStorage.vmdk
    volumeID: 1b819454-a280-4cff-aff5-141f4e8fd154
  name: libStorage.vmdk
  size: 64
  status: /Users/akutz/VirtualBox/libStorage/libStorage.vmdk
  id: 1b819454-a280-4cff-aff5-141f4e8fd154
  type: ""

Overview

REX-Ray is a storage orchestration tool that provides a set of common commands for managing multiple storage platforms. Built on top of the libStorage framework, REX-Ray connects enables persistent storage for container runtimes such as Docker and Mesos.

Storage Provider Support

The following storage providers and platforms are supported by REX-Ray.

Provider	Storage Platform(s)
EMC	ScaleIO, Isilon
Oracle VirtualBox	Virtual Media

Support for the following storage providers will be reintroduced in upcoming releases:

Provider	Storage Platform(s)
Amazon EC2	EBS
Google Compute Engine	Disk
Open Stack	Cinder
Rackspace	Cinder
EMC	XtremIO, VMAX

Operating System Support

The following operating systems (OS) are supported by REX-Ray:

OS	Command Line	Service
Ubuntu 12+	Yes	Yes
Debian 6+	Yes	Yes
RedHat	Yes	Yes
CentOS 6+	Yes	Yes
CoreOS	Yes	Yes
TinyLinux (boot2docker)	Yes	Yes
OS X Yosemite+	Yes	No
Windows	No	No

Container Platform Support

REX-Ray currently supports the following container platforms:

Platform	Use
Docker	Volume Driver Plugin
Mesos	Volume Driver Isolator module
Mesos + Docker	Volume Driver Plugin

Getting Started

This section will help in getting REX-Ray up and running quickly.

Installing REX-Ray

The following command will download the most recent, stable build of REX-Ray and install it to /usr/bin/rexray on Linux systems. REX-Ray will be registered as either a SystemD or SystemV service depending upon the OS.

$ curl -sSL https://dl.bintray.com/emccode/rexray/install | sh

Refer to the User Guide's installation topic for instructions on building REX-Ray from source, installing specific versions, and more.

Configuring REX-Ray

REX-Ray requires a configuration file for storing details used to communicate with storage providers. This can include authentication credentials and driver-specific configuration options. Refer to the libStorage Storage Providers documentation for sample configurations of all supported storage platforms. Additionally, look at core properties & logging for advanced configurations.

Create a configuration file on the host at /etc/rexray/config.yml. Here is a simple example for using Oracle VirtualBox:

libstorage:
  service: virtualbox

Refer to the VirtualBox documentation for additional VirtualBox configuration options.

From here, REX-Ray can now be used as a command line tool. View the commands available:

$ rexray --help

To verify the configuration file, use REX-Ray to list the volumes:

$ rexray volume
- attachments:
  - instanceID:
      id: e71578b0-1bfb-4fa5-bcd5-4ae982fd4a9b
      driver: virtualbox
    status: /Users/akutz/VirtualBox/libStorage/libStorage.vmdk
    volumeID: 1b819454-a280-4cff-aff5-141f4e8fd154
  name: libStorage.vmdk
  size: 64
  status: /Users/akutz/VirtualBox/libStorage/libStorage.vmdk
  id: 1b819454-a280-4cff-aff5-141f4e8fd154
  type: ""

If there is an error, use the -l debug flag and consult debugging instructions.

Start REX-Ray as a Service

Container platforms rely on REX-Ray to be running as a service to function properly. For instance, Docker communicates to the REX-Ray Volume Driver via a UNIX socket file.

$ rexray service start

Hello REX-Ray

In the grand tradition of technical documentation, the first true end-to-end example of REX-Ray is called Hello REX-Ray. It showcases a two-node deployment with the first node configured as a REX-Ray/libStorage server and the second node as merely a client. Both nodes have Docker (1.11+) installed and configured to leverage REX-Ray for persistent storage.

The below example does have a few requirements:

• VirtualBox 5.0+
• Vagrant 1.8+
• Ruby 2.0+

Start REX-Ray Vagrant Environment

Before bringing the Vagrant environment online, please ensure it is accomplished in a clean directory:

$ cd $(mktemp -d)

Inside the newly created, temporary directory, download the REX-Ray Vagrantfile:

$ curl -fsSLO https://raw.githubusercontent.com/emccode/rexray/master/Vagrantfile

Now it is time to bring the REX-Ray environment online:

$ vagrant up

The above command should result in output similar to this Gist.

Once the command has been completed successfully there will be two VMs online named node0 and node1. Both nodes are running Docker and REX-Ray with node0 configured to act as a libStorage server.

Now that the environment is online it is time to showcase Docker leveraging REX-Ray to create persistent storage as well as illustrating REX-Ray'a distributed deployment capabilities.

Node 0

First, SSH into node0

$ vagrant ssh node0

From node0 use Docker with REX-Ray to create a new volume named hellopersistence:

vagrant@node0:~$ docker volume create --driver rexray --opt size=1 \
                 --name hellopersistence

After the volume is created, mount it to the host and container using the --volume-driver and -v flag in the docker run command:

vagrant@node0:~$ docker run -tid --volume-driver=rexray \
                 -v hellopersistence:/mystore \
                 --name temp01 busybox

Create a new file named myfile on the file system backed by the persistent volume using docker exec:

vagrant@node0:~$ docker exec temp01 touch /mystore/myfile

Verify the file was successfully created by listing the contents of the persistent volume:

vagrant@node0:~$ docker exec temp01 ls /mystore

Remove the container that was used to write the data to the persistent volume:

vagrant@node0:~$ docker rm -f temp01

Finally, exit the SSH session to node0:

vagrant@node0:~$ exit

Node 1

It's time to connect to node1 and use the volume hellopersistence that was created in the previous section from node0.

Use the vagrant command to SSH into node1:

$ vagrant ssh node1

Next, create a new container that mounts the existing volume, hellopersistence:

vagrant@node1:~$ docker run -tid --volume-driver=rexray \
                 -v hellopersistence:/mystore \
                 --name temp01 busybox

The next command validates the file myfile created from node0 in the previous section has persisted inside the volume across machines:

vagrant@node1:~$ docker exec temp01 ls /mystore

Finally, exit the SSH session to node1:

vagrant@node1:~$ exit

Cleaning Up

Be sure to kill the VirtualBox web server with a quick killall vboxwebsrv and to tear down the Vagrant environment with vagrant destroy. Omitting these commands will leave the web service and REX-Ray Vagrant nodes online and consume additional system resources.

Congratulations

REX-Ray has been used to provide persistence for stateless containers! Examples using MongoDB, Postgres, and more with persistent storage can be found at Application Examples.

Getting Help

Having issues? No worries, let's figure it out together.

Debug

The -l debug flag can be appended to any command in order to get verbose output. The following command will list all of the volumes visible to REX-Ray with debug logging enabled:

$ rexray volume -l debug

For an example of the full output from the above command, please refer to this Gist.

GitHub and Slack

If a little extra help is needed, please don't hesitate to use GitHub issues or join the active conversation on the EMC {code} Community Slack Team in the #project-rexray channel