In today’s digital age, the demand for uploading media files has skyrocketed across various platforms. But, have you ever wondered where or how this media is stored? They all go through a process of file uploading. Civo Object Stores provides scalable cloud storage solutions that enable users to securely store and retrieve files remotely. This flexibility makes them ideal for applications ranging from streaming music services to e-learning platforms.

This tutorial will walk you through the following steps:

  1. Setting up a Node.js backend with Express.js to handle file uploads.
  2. Integrating file upload functionality with civo object stores using the Civo API.
  3. Building a responsive frontend using React to facilitate file uploads.
  4. Testing the file upload application.

Cloud Storage Service providers offer online storage solutions that allow users to store and retrieve their files remotely, providing flexibility and scalability.

Prerequisite

As well as a basic understanding of Kubernetes and JavaScript, you should have the following in place before starting this tutorial:

Creating Object Store Credentials

With access to your Civo account and CLI, we will run the following commands to create our Civo object-store credentials:

$ civo objectstore credential create demo-credential

To retrieve our object store credential details:

$ civo objectstore credential ls
+-----------------+----------------------+--------+
| Name            | Access Key           | Status |
+-----------------+----------------------+--------+
| demo-credential | KQHD3XHBP8O9PG2YF48T | ready  |
+-----------------+----------------------+--------+

With the access key returned, we can get our object store secret key by running the command below

$ civo objectstore credential secret -a <access-key<
Your secret key is: 9OW1dRuveD5Kk8B4YgvUHrD5iQxt77C6ZA5xOD4O

Creating an Object Store on Civo

You can set up your credentials as environmental variables following the command below:

export OWNER_ACCESS_KEY=your_access_key_id

Once you have created an object store credentials using the previous steps, you will be able to create an object store. To do this using the Civo CLI, input:

civo objectstore create <name> --owner-access-key ${OWNER_ACCESS_KEY}
  • Change the <name> parameter to your choice name.
  • The optional parameter included is --owner-access-key ${OWNER_ACCESS_KEY}, the object store credential referenced by id to make the owner of the store to be created.
If you want to follow the dashboard route, you can follow the steps contained in this guideline to help.

Application Core and Uploading

To get started with the front-end, we will use vite, as our build tool and for scaffolding our front-end. Vite is an open source build tool that aims to provide a faster and leaner development experience for modern web projects.

Run the script below to scaffold:

npm create vite@latest

Following the on-screen setup, you can use a similar setup to the one below:

✔ Project name: … civo-pet-store
✔ Select a variant: › JavaScript

Scaffolding project in .../civo-pet-store...

Done. Now run:

  cd civo-pet-store
  npm install
  npm run dev

Run the commands above to change your directory to the newly created project and install.

We are using ChakraUI as our styling library. Copy the command below to install ChakraUI and its dependencies:

npm i @chakra-ui/react@^2.8.2 @emotion/react@^11.11.3 @emotion/styled framer-motion@^11.0.3

We have everything set up for our front-end application. Open it in your editor and navigate to App.css.

Replace the content with the styling below:

#root {
  max-width: 1280px;
  margin: 0 auto;
  padding: 2rem;
  text-align: center;
}

.logo {
  height: 6em;
  padding: 1.5em;
  will-change: filter;
  transition: filter 300ms;
}
.logo:hover {
  filter: drop-shadow(0 0 2em #646cffaa);
}
.logo.react:hover {
  filter: drop-shadow(0 0 2em #61dafbaa);
}

@keyframes logo-spin {
  from {
    transform: rotate(0deg);
  }
  to {
    transform: rotate(360deg);
  }
}

@media (prefers-reduced-motion: no-preference) {
  a:nth-of-type(2) .logo {
    animation: logo-spin infinite 20s linear;
  }
}

.card {
  padding: 2em;
}

.read-the-docs {
  color: #888;
}

In your assets folder, you need a background image, you can download the image here. Add the image to the assets folder. Ensure the image name is background-image.jpg.

Create a components folder, create a file PetCard.jsx and paste the codeblock below:

import {
  Box,
  Center,
} from '@chakra-ui/react';

export default function PetCard({ image, name }) {
  return (
    <Center py={6}>
      <Box
        w={'400px'}
        // w={'full'}
        bg={"wheat"}
        boxShadow={'2xl'}
        rounded={'md'}
        p={6}
        overflow={'hidden'}>
        <Box h={'210px'} bg={'gray.100'} mt={-6} mx={-6} mb={6} pos={'relative'}>
          <img
            src={image}
            alt="Example"
          />
        </Box>
      </Box>
    </Center>
  )
}

Create another file, UploadImage.jsx and paste the codeblock below into it:

import {
  Modal,
  ModalOverlay,
  ModalContent,
  ModalHeader,
  ModalFooter,
  ModalBody,
  ModalCloseButton,
  Button,
  Flex,
  Box,
  FormControl,
  FormLabel,
  Input,
  Stack,
  useColorModeValue,
} from '@chakra-ui/react'
import { useState } from 'react';


function UploadImage({ isOpen, onClose} ) {

  const [formData, setFormData] = useState({
      image: null,
    });

    const handleInputChange = (event) => {
      const { id, value, files } = event.target;
      const updatedFormData = {
        ...formData,
        [id]: (files ? files[0] : null),
      };
      setFormData(updatedFormData);
    };

    const handleUpload = () => {
      console.log(formData.image);
    }

  return (
    <>  
      <Modal isOpen={isOpen} onClose={onClose}>
        <ModalOverlay />
        <ModalContent>
          <ModalHeader>Add new pet</ModalHeader>
          <ModalCloseButton />
          <ModalBody>
          <Flex
    align={'center'}
    justify={'center'}
    bg={useColorModeValue('gray.50', 'gray.800')}>
    <Stack spacing={8} mx={'auto'} maxW={'lg'} py={1} px={6}>
      <Box
        rounded={'lg'}
        bg={useColorModeValue('white', 'gray.700')}
        >
        <Stack spacing={4}>
          <FormControl id="image">
            <FormLabel>Image</FormLabel>
            <Input type="file" onChange={handleInputChange} />
          </FormControl>
        </Stack>
      </Box>
    </Stack>
  </Flex>
          </ModalBody>

          <ModalFooter>
            <Button colorScheme='red'mr={3} onClick={onClose}>
              Close
            </Button>
            <Button colorScheme='blue' onClick={handleUpload} variant='outline'>Upload Image</Button>
          </ModalFooter>
        </ModalContent>
      </Modal>
    </>
  )
}

export default UploadImage;

Now you can replace the content of App.jsx in your src folder to match the below:

import { Stack, Box, Flex, Button, Text, VStack, useBreakpointValue , useDisclosure, Heading} from '@chakra-ui/react'
import bgDefault from './assets/background-image.jpg'
import UploadImage from './components/UploadImage'
import PetCard from './components/PetCard'

export default function App() {
  const { isOpen, onOpen, onClose } = useDisclosure()

  return (
    <Box>
      <Flex
      w={'full'}
      h={'100vh'}
      backgroundImage={bgDefault}
      backgroundSize={'cover'}
      backgroundPosition={'center center'}>
      <VStack
        w={'full'}
        justify={'center'}
        px={useBreakpointValue({ base: 4, md: 8 })}
        bgGradient={'linear(to-r, blackAlpha.600, transparent)'}>
        <Stack maxW={'2xl'} align={'center'} spacing={6}>
          <Heading color={'white'}>CIVO File Upload</Heading>
          <Text
            color={'white'}
            fontWeight={700}
            lineHeight={1.2}
            fontSize="20px">
            Photo by <a href="https://unsplash.com/@tranmautritam?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Tran Mau Tri Tam ✪</a> on <a href="https://unsplash.com/photos/grey-tabby-cat-beside-short-coat-brown-and-white-dog-7QjU_u2vGDs?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>
          </Text>
          <Stack direction={'row'}>
            <Button
              bg={'blue.400'}
              rounded={'full'}
              color={'white'}
              _hover={{ bg: 'blue.500' }}
              onClick={onOpen}
            >
              Upload new image
            </Button>
          </Stack>
        </Stack>
      </VStack>
      <UploadImage isOpen={isOpen} onClose={onClose} />
    </Flex>

    <Flex p={4} mt="2rem" align="center" justify="start" flexWrap="wrap" gap="10">
        <PetCard image={bgDefault} name="Kitten" />
        <PetCard image={bgDefault} name="Kitten" />
    </Flex>

    </Box>
  )
}

Voila, we are all set on the frontend!

If you can run all the commands to this point, you should be able to see the image below on your screen ↓

Application Core and Uploading

If the image does not appear, ensure the following checklist is completed:

  1. All libraries are installed.
  2. Ensure you have copied all previous code blocks in the correct order without skipping any steps.
  3. Check your browser console for any other error.

To set up our backend environment, create a civo-file-upload directory and run the npm init command, run the command below to create the civo-file-upload. 

mkdir civo-file-upload
cd civo-file-upload
npm init

After npm init you will be prompted, you can follow the below:

Press ^C at any time to quit.
package name: (civo-file-upload) 
version: (1.0.0) 
description: CIVO node file upload
entry point: (index.js) 
test command: 
git repository: 
keywords: 
author: 
license: (ISC)

After completing the steps above, your directory should have only the package.json file.

In the directory, we will also install express, @aws-sdk/client-s3, multer, cors, nodemon, which are important packages for our application setup.For our setup, we are using v4.19.2 of express, v3.567.0 of @aws-sdk/client-s3 and "multer v1.4.5-lts.1. Run the command below:

npm install @aws-sdk/client-s3@3.567.0 cors@2.8.5 express@4.19.2 multer@1.4.5-lts.1 nodemon@3.0.3

Create an index.js file with the code below:

    const express = require('express')
    const app = express()
    const port = 3000
    const cors = require('cors')

    app.use(cors())
    app.get('/', (req, res) => res.json({ message: 'Hello World!' }))
    app.listen(port, () => console.log(`This is the beginning of the Node File Upload App`))

Run the command node index.js, and your backend server will be up and running.

Afterward, proceed to your Civo object store to get your credentials (access key and secret key) and endpoint.

Your Alt Text

Once we have obtained the credentials, we can set it up as environmental variables in our project. Then, proceed to add the code block below into the index.js file.

const { S3Client, ListObjectsCommand, PutObjectCommand } = require('@aws-sdk/client-s3');
const cors = require('cors')
const multer = require('multer');
const upload = multer({ dest: 'pics/' });
const { readFileSync, unlinkSync } = require("fs");

const s3Client = new S3Client({
  credentials: {
    accessKeyId: 'your-civo-access-key',
    secretAccessKey: 'your-civo-secret-key',
  },
  endpoint: 'your-civo-objectstore-endpoint',
  region: 'your-region',
  forcePathStyle: true, // this allows the sdk follow the endpoint structure and not append the bucket name
});

Multer helps in handling multipart/form-data, it serves as a Node middleware that is primarily used for uploading files. Multer accepts an options object, the most basic of which is the dest property, which tells Multer where to upload the files.

Now that you have set up your Civo credentials, you can upload a file to the object store. Using the code below, you can upload a file to the object store you created:

const uploadFileToBucket = async ({ bucketName, file }) => {
    try {
      const getFileExt = file.originalname.split('.').pop();
  
      const filePath = file.path;
      const fileContent = readFileSync(filePath);
      const upload = await s3Client.send(
        new PutObjectCommand({
          Bucket: bucketName,
          Body: fileContent,
          Key: file.filename + '.' + getFileExt,
          ACL: 'public-read',
        }),
      );
  
      unlinkSync(file.path);
      return upload;
    } catch (error) {
      console.error('Error uploading file to S3:', error);
      throw error;
    }
  };

The function above accepts 2 parameters. bucketName is your Civo object store name, where you intend for your uploaded images to be stored. File is the image file, passed from the frontend or via your API client. The file object gives us various methods which are needed for a successful upload. From this, we can get the image extension and the image content.

Earlier, we specified a dest configuration in our multer initialization. When you upload an image, multer will temporarily store it in the dest folder specified and read the file content from the temporarily stored image.

To upload to the object store, we used the AWS SDK’s S3 API to create a new object and upload the file data to it.

unlinkSync is a method that helps clean up the temporary file after the upload process is completed.

Now, we can create an upload route in our index.js that makes use of the uploadFileToBucket we created earlier. You can add this code below:

app.put('/upload', upload.single('image'), async (req, res) => {
    try {
      if (!req.file) {
        res.status(400).send('File not uploaded');
        return;
      }

      const upload = await uploadFileToBucket({ bucketName: 'your-civo-object-store', file: req.file  });
      res.send(upload);
    } catch (error) {
      console.error('Error uploading file:', error);
      res.status(500).send('Error uploading file');
    }
  });

Remember to replace the your-civo-object-store with the appropriate name as set up on your Civo dashboard.

Now that we've set up our upload endpoint, we will add an endpoint to get all images uploaded to the object store. This will also allow us to preview everything on the frontend. You can add the code below:

app.get('/all', async (req, res) => {
    try {
      const command = new ListObjectsCommand({ Bucket: 'your-civo-object-store' });
      const { Contents } = await s3Client.send(command);
      res.send(Contents);
    } catch (err) {
      console.error(err);
      res.status(500).send('Error retrieving objects');
    }
  });

Our index.js is complete and should be similar to the one below:

const express = require('express')
const app = express()
const port = 3000
const cors = require('cors')
app.use(cors())

const { S3Client, ListObjectsCommand, PutObjectCommand } = require('@aws-sdk/client-s3');
const multer = require('multer');
const upload = multer({ dest: 'pics/' });
const { readFileSync, unlinkSync } = require("fs");

const s3Client = new S3Client({
    credentials: {
      accessKeyId: 'your-civo-access-key',
      secretAccessKey: 'your-civo-secret-key',
    },
    endpoint: 'your-civo-objectstore-endpoint',
    region: 'your-region',
    forcePathStyle: true, // this allows the sdk follow the endpoint structure and not append the bucket name
  });

const uploadFileToBucket = async ({ bucketName, file }) => {
    const getFileExt = file.originalname.split('.').pop();
    
    const filePath = file.path;
    const fileContent = readFileSync(filePath);
    const upload = await s3Client.send(
      new PutObjectCommand({
        Bucket: bucketName,
        Body: fileContent,
        Key: file.filename + '.' + getFileExt,
        ACL: 'public-read',
      }),
    )
  
    unlinkSync(file.path);
    return upload;
  };

app.put('/upload', upload.single('image'), async (req, res) => {
    try {
      if (!req.file) {
        res.status(400).send('File not uploaded');
        return;
      }

      const upload = await uploadFileToBucket({ bucketName: 'your-civo-object-store', file: req.file  });
      res.send(upload);
    } catch (error) {
      console.error('Error uploading file:', error);
      res.status(500).send('Error uploading file');
    }
  });

app.get('/all', async (req, res) => {
    try {
      const command = new ListObjectsCommand({ Bucket: 'your-civo-object-store' });
      const { Contents } = await s3Client.send(command);
      res.send(Contents);
    } catch (err) {
      console.error(err);
      res.status(500).send('Error retrieving objects');
    }
  });

app.get('/', (req, res) => res.json({ message: 'Hello World!' }))

app.listen(port, () => console.log(`This is the beginning of the Node File Upload App`))

Now, let's ensure we can upload from our frontend application. Navigate to components/UploadImage.jsx and replace the handleUpload function with the code below:

const handleUpload = () => {
    const apiFormData = new FormData();
    apiFormData.append('image', formData.image);

    fetch(`your-backend-base-url/upload`, {
      method: 'PUT',
      body: apiFormData,
    })
      .then((res) => {
        if(res.status == 200) {
          onClose();
        }
      })
}

To be able to view all uploaded images, navigate to the App.jsx page and add the code below. Ensure it is above the return statement.

const [photos, setPhotos] = useState([]);
useEffect(() => {
  fetch(`your-backend-base-url/all`)
  .then((res) => {
    return res.json();
  })
  .then((data) => {
    setPhotos(data);
  });
}, []);

Remember to change your-backend-base-url to the endpoint exposed by the backend. Following best practices, you can store it in your .env.

This helps to get all the uploaded images from our object store and store using react useState.

Now we can change our placeholder images by navigating to this section of App.tsx 

 <Flex p={4} mt="2rem" align="center" justify="start" flexWrap="wrap" gap="10">
        <PetCard image={bgDefault} name="Kitten" />
        <PetCard image={bgDefault} name="Kitten" />
    </Flex>

Replace with:


<Flex p={4} mt="2rem" align="center" justify="start" flexWrap="wrap" gap="10">
        {photos.map((photo, index) => 
        <PetCard key={index} image={`your-object-storeo-endpoint/your-bucket-name/`+photo?.Key} name="Kitten" />        
      )}
    </Flex>

Remember to change your your-object-store-endpoint and your-bucket-name appropriately.

If you follow up on this point, you should be able to see your uploaded images successfully. Congratulations!

If the image does not appear, ensure the following checklist is completed:

  1. Ensure you have copied all previous code blocks in the correct order without skipping any steps.
  2. Check your browser console for other errors.
  3. Ensure your civo credentials are included appropriately on the backend.

Clean Up (Optional Step)

To remove all credentials and delete the object stores created via the CLI earlier in the tutorial, we will run this command:

Object store deletion will destroy all objects (files) within the store and cannot be recovered. 
civo objectstore delete <store-name>

Change the <store-name> parameter to the initial objectstore name we created.

After deleting the objectstore, you can also delete the credentials. The objectstores with the credential must be deleted before the credential can be deleted successfully.

civo objectstore credential delete <credential-name>
Note: <credential-name> is a placeholder for the objectstore credential created earlier and needs to be replaced by the name used during credential create

You have now removed the credentials and object stores used during this tutorial process.

Summary

Object stores have helped scale applications across different domains, and are not limited to web applications alone. It helps prevent the storage of dependencies in databases, and their references can be passed to the database instead.

For the expected end, you can check out this repository for the frontend and the backend.

If you want to learn more about object storage, check out some of these further resources: