MinIO is a drop-in open-source replacement for Amazon S3 (Simple Storage Service) for backing up files, as a storage back-end for tools such as a container registry, or even to host static websites.

MinIO describes itself as:

The 100% Open Source, Enterprise-Grade, Amazon S3 Compatible Object Storage

For this tutorial, we will use Civo to host an instance, which will provide a public IP address, where other servers or your project can connect. You will have the option to create a user per server, one user to all servers, or a user per project, such as Django, to upload all your media and static assets.

Prerequisites

By following this guide, you will set up your own version of an Amazon S3-compatible storage service that can be configured to be accessed by multiple users using MinIO. You can host your MinIO server on any host that you have administrator access to, such as instances on Civo.

We recommend having the following prerequisites:

Getting started with MinIO

Provisioning your Instance

  1. Log into your Civo dashboard
  2. Create a Medium Instance and call it minio-cloud
  3. Select Debian 9 for the Operating System, add your SSH key for login
  4. Make sure you select the option for a public IP
  5. Keep the default firewall

Your Instance will be up in around 45 seconds. Once the instance is up, we can start.

Installing MinIO

Connect via ssh to the IP address of the instance. To prepare for installing the MinIO server, first of all, we want to create a directory to store all data created by users. We will be using /srv/minio/data for our data store. In the instance terminal, run:

$ sudo mkdir -p /srv/minio/data
$ wget https://dl.min.io/server/minio/release/linux-amd64/minio
$ chmod +x minio
$ sudo mv ./minio /usr/local/bin/minio

Then start the MinIO server:

$ sudo minio server /srv/minio/data &

You will see your access key and secret key printed on the console. These are going to be required later on, so take note of them. You can exit from the server for now by pressing Ctrl-C. Now we will create a new user called minio-user and give it permissions to the data-store:

$ sudo useradd minio-user
$ sudo chown minio-user -R /srv/minio/data

After this we need to create a file /etc/default/minio, with the content of this file as:

MINIO_VOLUMES="/srv/minio/data"
MINIO_OPTS=""
MINIO_ROOT_USER=YOUR-MINIO-ROOT-USER
MINIO_ROOT_PASSWORD=YOUR-MINIO-ROOT-PASSWORD

The MINIO_ROOT_USER and MINIO_ROOT_PASSWORD are the values you took note of above. These will be the credentials to manage the server.

We will use systemd to automatically start the MinIO server when the instance starts, to make sure it is automatically available: First, install curl (or check it is installed):

$ sudo apt install curl -y

Then, get the MinIO service file and place it in the correct directory:

$ cd /etc/systemd/system/ && sudo curl -O https://raw.githubusercontent.com/minio/minio-service/master/linux-systemd/minio.service

The content of the minio.service is this:

[Unit]
Description=MinIO
Documentation=https://docs.min.io
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/minio
AssertFileNotEmpty=/etc/default/minio

[Service]
Type=notify

WorkingDirectory=/usr/local/

User=minio-user
Group=minio-user
ProtectProc=invisible

EnvironmentFile=/etc/default/minio
ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /etc/default/minio\"; exit 1; fi"
ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES

# Let systemd restart this service always
Restart=always

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=1048576

# Specifies the maximum number of threads this process can create
TasksMax=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=infinity
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

# Built for ${project.name}-${project.version} (${project.name})

Now we can enable the MinIO service:

$ systemctl enable minio.service

Turning on TLS for Minio

By default, MinIO does not ship with TLS enabled. You can turn on this encryption by following this guide: How to secure access to MinIO server with TLS

Installing MinIO Client

MinIO Client (mc) provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff etc. You can follow this link https://min.io/docs/minio/kubernetes/upstream/ and download the client for the Debian instance or your local machine:

wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin/mc

Configuring the MinIO Client

Now we need to add a host to the client. mc stores all its configuration information in the file ~/.mc/config.json. We are going to add a new host, our instance, to the MinIO mc running on that instance. First, we need to export the public IP address of the instance, in this example 91.211.152.61, which you can easily see on your Civo dashboard or using the CLI tool:

On the instance where you downloaded mc run:

export MINIO_IP="91.211.152.61" # use your instance IP

After we can configure the mc client:

$ sudo mc config host add minio-cloud http://$MINIO_IP:9000 <YOUR-ACCESS-KEY> <YOUR-SECRET-KEY>

The output will be something like this:

$ mc: Configuration written to `/root/.mc/config.json`. Please update your access credentials.
$ mc: Successfully created `/root/.mc/share`.
$ mc: Initialized share uploads `/root/.mc/share/uploads.json` file.
$ mc: Initialized share downloads `/root/.mc/share/downloads.json` file.
$ Added `minio-cloud` successfully.

Now we will create the first storage bucket for the user. In my case I use my username alejandrojnm for the name of the bucket:

$ sudo mc mb minio-cloud/alejandrojnm
Bucket created successfully `minio-cloud/alejandrojnm`.

After this, we need to create a file to put all management policies for the user. I have called this file user.json, and you can save it in any place - but remember where you stored it! The contents of the file will be:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:PutBucketPolicy",
        "s3:GetBucketPolicy",
        "s3:DeleteBucketPolicy",
        "s3:ListAllMyBuckets",
        "s3:ListBucket"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::alejandrojnm"
      ],
      "Sid": ""
    },
    {
      "Action": [
        "s3:AbortMultipartUpload",
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:ListMultipartUploadParts",
        "s3:PutObject"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::alejandrojnm/*"
      ],
      "Sid": ""
    }
  ]
}

Now we can add the policy to our MinIO server and we do it using:

$ sudo mc admin policy create minio-cloud user /path-to-the-file/user.json

To check that everything is working, you can run this command:

$ sudo mc admin policy info minio-cloud user

Next, you we need to create the user by adding:

$ sudo mc admin user add minio-cloud alejandrojnm mypassword

Then we need to apply the policy we created a few steps back:

$ sudo mc admin policy attach minio-cloud user --user alejandrojnm

Note: You can find more details on policy attach here: https://min.io/docs/minio/linux/reference/minio-mc-admin/mc-admin-policy-attach.html#command-mc.admin.policy.attach

To see if everything is in order we can run this command:

$ sudo mc admin user list minio-cloud --json

In return, we should see something like this in the result:

{
 "status": "success",
 "accessKey": "alejandrojnm",
 "policyName": "user",
 "userStatus": "enabled"
}

Installing mc on other server(s)

Now we'll configure the client with this new user, which is the same as we’ve done so far in "Installing" and "Configuring the MinIO mc Client", but on a new server such as on your local machine.

You could also use the user and password in programs such as Transmit, or even in web apps like Restic for running back-ups.

For now, we'll add a new host to the MinIO client. On the new host, add:

$ sudo mc config host add minio-cloud http://$MINIO_IP:9000 alejandrojnm mypassword

Now, if you visit the web UI of MinIO (navigating your browser to the IP address of the instance on port 9000), you will only see your bucket. If you use any client or mc you will only see your bucket. You can run this to check:

$ sudo mc ls minio-cloud/alejandrojnm

This way, you could have many different private users with buckets between them, which could be used for company projects, as well as keep project files separate and private. By adding multiple people to the same bucket, each user will have access to the same files.

This process of adding policy and user creation can also be automated using the MinIO API, which is available for many languages:

Troubleshooting Tips

Here are a few common issues that you might encounter during the tutorial:

  • Permission Denied: This could occur when trying to execute a command or access a file/directory. Ensure that the user you're logged in as has the appropriate permissions. Using sudo before a command might help.
  • Command Not Found: If you receive this error, it may be that the software you're trying to use isn't installed or isn't in your PATH. Verify the software is installed, and your PATH is correctly configured.
  • Unable to Connect to Server: This could be due to network issues or because the server isn't running. Check your network connection and that the server is up and running.
  • File Not Found: If a command refers to a file that doesn't exist, you'll see this error. Check the file path and ensure the file exists at that location.

Need help?

Civo prides itself on being a cloud platform for developers, run by developers who can provide technical support and expert help via Intercom and the Civo community slack.

Further resources

Interested in learning more about this topic? Check out some of these resources: