Dhyan Tech!!
🚀 Building a File Management System with Node.js, Express, Multer, and MongoDB 📂

🚀 Building a File Management System with Node.js, Express, Multer, and MongoDB 📂

Dhyan Amit Shah's photo
Dhyan Amit Shah
·

17 min read

Table of contents

Introduction

Welcome, developers! In this post, we'll dive into creating a robust File Management System using Node.js and Express, storing our data in MongoDB. Let's explore the key technologies and steps to build this project.

Technologies Used

Let's lay the foundation with key technologies:

  1. Node.js: Our runtime environment, ensuring efficient server-side execution.

  2. Express: The go-to framework for building scalable and modular backend applications.

  3. MongoDB: A NoSQL database for storing file-related information securely.

  4. Multer: A middleware for handling multipart/form-data, perfect for managing file uploads.

  5. Helmet: A security middleware adding crucial HTTP headers to enhance application security.

These technologies form the backbone of our project. Now, let's dive into the project setup.

Project Setup

Setting up your Node.js project is the first step:

npm init -y
npm install express mongoose multer helmet jsonwebtoken bcrypt validator 
npm install express-async-handler passport passport-jwt dotenv

Here, we initialize our project and install the necessary dependencies: Express for server-side logic, Mongoose for MongoDB integration, Multer for efficient file handling, and Helmet for securing our application.

With our project scaffolded and dependencies installed, we're ready to explore the backend architecture.

Backend Architecture

Now that our project is set up, let's explore the architecture of our backend. Understanding how each technology contributes to the project is crucial.

Server:

const express = require('express')
const mongoose = require('mongoose')
const helmet = require('helmet')

require('dotenv').config()

const app = express()
const userRoute = require('./Routes/userRoutes')
const imageRoute = require('./Routes/imageRoutes')

app.use(express.json())
app.use(helmet())

app.use((req, res, next) => {
    console.log(req.path, req.method)
    next()
})

app.get("/api/", (req,res) => {
    res.send("<h1> Hello Helmet!</h1>")
})

app.use('/api/user', userRoute)
app.use('/api/image', imageRoute)

mongoose.connect(process.env.MONG_URI)
    .then(() => {
        app.listen((process.env.PORT), () => {
            console.log('Connected to DB and listening to port', process.env.PORT)
        })
    })
    .catch((error) => {
        console.log(error)
    })

This is the beginning of our application, MongoDB connection is done here, and the initial endpoints are also initialised here.

Model-Schema:

MongoDB serves as our data storage solution. We define a schema for images:

const mongoose = require('mongoose')

const imageSchema = mongoose.Schema({
    filename: {
        type: String,
        required: true
    },
    filepath: {
        type: String,
        required: true
    },
    userEmail: {
        type: String,
        required: false
    }
})

const Image = mongoose.model('Image', imageSchema);

module.exports = Image;

  1. Dependencies:

    • Import the mongoose package for MongoDB interaction.

  2. Defining Image Schema:

    • Create a Mongoose schema for handling image data.

    • Includes fields like filename (string, required), filepath (string, required), and optional userEmail (string).

  3. Model Creation:

    • Use Mongoose to create a model named 'Image' based on the defined schema.

  4. Schema Fields:

    • filename:

      • Type: String

      • Requirement: Must be provided (required: true)

    • filepath:

      • Type: String

      • Requirement: Must be provided (required: true)

    • userEmail:

      • Type: String

      • Requirement: Optional (required: false)

  5. Usage:

    • Developers can use this schema to store information about images, such as their filenames and file paths.

    • The optional userEmail field provides flexibility for associating images with specific users if needed.

  6. Model Export:

    • Export the Mongoose model named 'Image' for use in other parts of the application.

Now, Schema for user authentication:

const mongoose = require('mongoose')
const validator = require('validator')
const bcrypt = require('bcrypt')
const { match } = require('assert')
const { exists } = require('fs')

const Schema = mongoose.Schema

const userSchema = new Schema({
    Email: {
        type: String,
        required: true,
        unique: true
    },
    Password: {
        type: String,
        required: true,
        minlength: 8
    } 
}, {timestamps: true})

userSchema.statics.signup = async function (Email, Password) {

    if(!Email || !Password){
        throw Error('All credentials must be filled.')
    }

    if(!validator.isEmail(Email)){
        throw Error('Please enter a valid email ID.')
    }

    if(!validator.isStrongPassword(Password)){
        throw Error('Password is not strong enough.')
    }

    const exists = await this.findOne({Email})

    if(exists) {
        throw Error('Email is already in use.')        
    }

    const salt = await bcrypt.genSalt(10)
    const hash = await bcrypt.hash(Password, salt)

    const user = await this.create({Email, Password: hash})

    return user
}

userSchema.statics.login = async function (Email, Password) {

    if(!Email || !Password) {
        throw Error('All fields must be filled.')
    }

    const user = await this.findOne({Email})

    if(!user){
        throw Error('Incorrect Email')
    }

    const match = await bcrypt.compare(Password, user.Password)

    if(!match) {
        throw Error('Incoorect Password, please provide valid credentials.')
    }

    return user
}

module.exports = mongoose.model('user', userSchema)

  1. Dependencies:

    • Import necessary packages: mongoose for MongoDB interaction, validator for input validation, and bcrypt for password hashing.

  2. Defining User Schema:

    • Use Mongoose to create a schema with an email (string, required, unique) and password (string, required, minlength: 8).

    • {timestamps: true} adds createdAt and updatedAt fields to track document changes.

  3. Signup Method:

    • userSchema.statics.signup handles user registration.

    • Validates input for empty credentials, valid email, and a strong password.

    • Check if the email is already in use.

    • Hashes the password using bcrypt and creates a new user.

  4. Login Method:

    • userSchema.statics.login manages user login.

    • Validates input for empty fields.

    • Finds the user by email and checks if the email is correct.

    • Compares the provided password with the hashed password stored in the database.

  5. Password Hashing:

    • Bcrypt is used for secure password hashing.

    • Generates a salt with bcrypt.genSalt and hashes the password with bcrypt.hash.

  6. Error Handling:

    • Throws custom errors for various scenarios, like empty fields, invalid email, weak password, existing email during signup, incorrect email during login, and mismatched passwords.

  7. Exporting the Model:

    • Export the Mongoose model named 'user' for use in other parts of the application.

  8. Usage:

    • Developers can utilize this schema for building a user authentication system in a Node.js application.

    • Integrating this schema ensures secure password storage and robust user registration/login processes.

Image Upload:

Create a folder named uploads, and create an upload.js file:

const multer = require('multer')

const storage = multer.diskStorage({
    destination: (req, file, cb) => {
        cb(null, "uploads/")
    },
    filename: (req, file, cb) => {
        cb(null, Date.now() + file.originalname)
    }
})

const upload = multer({ storage: storage})

module.exports = {upload}

  1. Dependencies:

    • Import the multer package for handling file uploads in Node.js applications.

  2. Multer Storage Configuration:

    • Create a disk storage engine using multer.diskStorage.

    • destination Function:

      • Determines the destination folder for uploaded files.

      • In this case, files are stored in the "uploads/" directory.

    • filename Function:

      • Generates a unique filename for each uploaded file.

      • Combines the current timestamp with the original filename to avoid conflicts.

  3. Multer Configuration:

    • Create a Multer instance (upload) with the defined storage configuration.

  4. Usage of Multer:

    • Developers can now use the upload middleware in their routes to handle file uploads.

    • Multer will automatically store the uploaded files in the specified "uploads/" directory with unique filenames.

  5. Model Export:

    • Export the configured Multer instance for use in other parts of the application.

By configuring Multer with this setup, developers can easily manage file uploads, define storage locations, and ensure unique filenames to avoid overwriting existing files. The exported upload instance can be integrated into routes where file uploads are expected, simplifying the implementation of file upload functionality in a Node.js application.

Controllers:

We will have 2 different controller files, imageController and userController.

imageController:

const expressHandler = require('express-async-handler')
const Image = require('../Models/imageSchema')
const mongoose = require('mongoose')
const path = require('path');
const fs = require('fs');

const getImages = expressHandler(async (req, res) => {
    try {
        // Extract the user's email from the token payload
        const userEmail = req.user.email;

        // Find all images associated with the authenticated user's email
        const images = await Image.find({ userEmail: userEmail });

        res.status(200).json(images);
    } catch (error) {
        console.log(error);
        res.status(500).json({ error: "Internal server error" });
    }
});

const postImage = expressHandler(async (req, res) => {
    try {
        if (!req.file) {
            return res.status(500).json({ Error: "No such file found" });
        }

        // Extract the user's email from the token payload
        const userEmail = req.user.email;

        // Associate the image with the authenticated user's email
        const imageFile = new Image({
            filename: req.file.filename,
            filepath: 'uploads/' + req.file.filename,
            userEmail: userEmail  // Assuming userEmail can be nullable in your schema
        });

        const savedImage = await imageFile.save();

        res.status(200).json(savedImage);
    } catch (error) {
        console.log(error);
        res.status(500).json({ error: "Internal server error" });
    }
});

const updateImage = expressHandler(async (req, res) => {
    try {
        const { id } = req.params;
        const { newFilename } = req.body;

        // Log the received ID and newFilename for debugging
        console.log("Received ID:", id);
        console.log("Received newFilename:", newFilename);

        // Check if newFilename is undefined
        if (newFilename === undefined) {
            return res.status(400).json({ error: "newFilename is missing in the request body" });
        }

        // Check if ID is valid
        if (!mongoose.Types.ObjectId.isValid(id)) {
            return res.status(400).json({ error: "Invalid image ID" });
        }

        const userEmail = req.user.email;

        // Find and update the image based on the user's email and ID
        const updatedImage = await Image.findOneAndUpdate(
            { _id: id, userEmail: userEmail }, // Check if the user owns the image
            { filename: newFilename },
            { new: true }
        );

        if (!updatedImage) {
            return res.status(404).json({ error: "Image not found" });
        }

        // Assuming your file is stored in the 'uploads' folder
        const oldFilePath = path.join(__dirname, '..', updatedImage.filepath);
        const newFilePath = path.join(__dirname, '..', 'uploads', newFilename);

        // Update the file extension based on the original file extension (e.g., .png)
        const originalExtension = path.extname(updatedImage.filepath);
        const newExtension = path.extname(newFilename);
        if (originalExtension !== newExtension) {
            fs.renameSync(oldFilePath, newFilePath + originalExtension);
        } else {
            fs.renameSync(oldFilePath, newFilePath);
        }

        res.status(200).json(updatedImage);
    } catch (error) {
        console.error(error.message);
        res.status(500).json({ error: "Internal server error" });
    }
});

const deleteImage = expressHandler(async (req, res) => {
    try {
        const { id } = req.params;

        // Log the received ID for debugging
        console.log("Received ID for deletion:", id);

        console.log("User Information:", req.user);
        // Extract the user's email from the token payload
        const userEmail = req.user.email;

        // Check if ID is valid
        if (!mongoose.Types.ObjectId.isValid(id)) {
            return res.status(400).json({ error: "Invalid image ID" });
        }

        const deletedImage = await Image.findOneAndDelete({ _id: id, userEmail: userEmail });

        if (!deletedImage) {
            return res.status(404).json({ error: "Image not found or unauthorized" });
        }

        // Assuming you want to delete the actual file from the server
        const filePath = deletedImage.filepath;
        fs.unlinkSync(filePath);

        res.status(200).json({ message: "Image deleted successfully" });
    } catch (error) {
        console.log(error);
        res.status(500).json({ error: "Internal server error" });
    }
});

module.exports = {
    postImage,
    getImages,
    updateImage,
    deleteImage
}

  1. Import Dependencies:

    • express-async-handler: A utility to handle asynchronous Express route handlers.

    • Image (Assumed to be a Mongoose model): Represents the MongoDB schema for storing image data.

    • mongoose: MongoDB object modelling tool.

    • path and fs: Node.js modules for working with file paths and the file system.

  2. Get Images Handler (getImages):

    • Endpoint: Handles GET requests to retrieve images associated with the authenticated user.

    • Retrieves the user's email from the token payload.

    • Queries the database for images linked to the user's email.

    • Responds with a JSON array of images or a server error if encountered.

  3. Post Image Handler (postImage):

    • Endpoint: Handles POST requests to upload an image.

    • Check if a file is present in the request.

    • Retrieves the user's email from the token payload.

    • Creates a new Image instance with file details and user email.

    • Saves the image to the database.

    • Responds with the saved image details or a server error if encountered.

  4. Update Image Handler (updateImage):

    • Endpoint: Handles PUT requests to update an image's filename.

    • Validates the received image ID and new filename.

    • Check if the user owns the image by matching the email from the token.

    • Updates the image's filename in the database.

    • Renames the corresponding file on the server filesystem.

    • Responds with the updated image details or appropriate error messages.

  5. Delete Image Handler (deleteImage):

    • Endpoint: Handles DELETE requests to delete an image.

    • Validates the received image ID and checks if the user owns the image.

    • Deletes the image from the database and the corresponding file from the server filesystem.

    • Responds with a success message or an error if the image is not found or unauthorized.

  6. Note:

    • The code assumes a certain structure for the Image schema and the file storage (e.g., in the 'uploads' folder).

    • It uses express-async-handler to simplify error handling in asynchronous routes.

    • File operations such as renaming and deleting involve interactions with the file system using the fs module.

    • Errors are logged to the console, and generic error responses are sent to the client.

These handlers provide basic CRUD (Create, Read, Update, Delete) functionality for managing images in a server, including file uploads and deletions. They are designed to work in an Express.js application with token-based user authentication.

userController:

const { default: mongoose } = require('mongoose')
const jwt = require('jsonwebtoken')

const User = require('../Models/userModel')

const createToken = (_id) => {
    return jwt.sign({_id}, process.env.SECRET, {expiresIn: '3d'})
}

const loginUser = async (req,res) => {
    const {Email, Password} = req.body

    try{
        const user = await User.login(Email, Password)

        const token = createToken(user._id)

        res.status(200).json({Email, token})
    }
    catch(error) {
        res.status(400).json({error: error.message})
    }
}


    const signupUser = async (req,res) => {
        const {Email, Password} = req.body

        try{
            const user = await User.signup(Email, Password)

            // create a token

            const token = createToken(user._id)

            res.status(200).json({Email, token})

        } catch(error) {
            res.status(400).json({error: error.message})
        }
    }


    module.exports = {
        loginUser,
        signupUser,
    }

  1. Import Dependencies:

    • mongoose: Assumes the default export from the 'mongoose' package.

    • jwt: JSON Web Token library for creating and verifying tokens.

    • User: Importing the user model (assumed to be a Mongoose model).

  2. Create Token Function (createToken):

    • Takes a user ID (_id) as an argument.

    • Uses jwt.sign to create a JWT token with the user ID, a secret key (process.env.SECRET), and an expiration time of 3 days ({expiresIn: '3d'}).

    • Returns the generated token.

  3. Login User Handler (loginUser):

    • Endpoint: Handles POST requests for user login.

    • Retrieves Email and Password from the request body.

    • Calls the User.login method (assumed to be a static method on the user model) to authenticate the user.

    • If successful, creates a JWT token using createToken with the user's ID.

    • Responds with the user's email and the generated token in the JSON response.

    • If an error occurs during login, responds with a 400 status and the error message.

  4. Signup User Handler (signupUser):

    • Endpoint: Handles POST requests for user signup.

    • Retrieves Email and Password from the request body.

    • Calls the User.signup method (assumed to be a static method on the user model) to create a new user.

    • If successful, creates a JWT token using createToken with the newly created user's ID.

    • Responds with the user's email and the generated token in the JSON response.

    • If an error occurs during signup, responds with a 400 status and the error message.

  5. Note:

    • The code assumes the existence of the User model with static methods login and signup.

    • JWT tokens are created with a secret key stored in the environment variable process.env.SECRET.

    • The createToken function abstracts the token creation process for reuse in both login and signup handlers.

    • Errors are caught, and corresponding error messages are sent in the response for better client feedback.

These handlers provide basic user authentication functionality, generating JWT tokens upon successful login or signup. The tokens can be used for subsequent authenticated requests to the server. The use of JWTs enhances security and enables stateless authentication in a Node.js application.

Token-Verification:

const jwt = require('jsonwebtoken')

const verifyTokenMiddleware = (req, res, next) => {
    try {
        // Extract the token from the request headers or wherever you store it
        const token = req.headers.authorization.split(' ')[1];

        // Verify and decode the token
        const decodedToken = jwt.verify(token, process.env.SECRET);

        // Attach the decoded token to the request object for further use
        req.user = decodedToken;

        next(); // Move to the next middleware or route handler
    } catch (error) {
        res.status(401).json({ error: 'Unauthorized. Invalid or missing token.' });
    }
};

module.exports = verifyTokenMiddleware;

  • Import Dependencies:

    • jsonwebtoken: The library used for creating and verifying JSON Web Tokens.

  • Verify Token Middleware (verifyTokenMiddleware):

    • A middleware function that can be used in Express.js routes to verify the authenticity of a JWT.

    • Takes three parameters: req (request), res (response), and next (callback to move to the next middleware or route handler).

  • Token Verification Process:

    • Tries to execute the following code block:
    •    // Extract the token from the request headers or wherever you store it
   const token = req.headers.authorization.split(' ')[1];

   // Verify and decode the token
   const decodedToken = jwt.verify(token, process.env.SECRET);

   // Attach the decoded token to the request object for further use
   req.user = decodedToken;

   next(); // Move to the next middleware or route handler

      • Extracts the token from the request headers. Assumes a format like "Bearer <token>" and extracts the token part (split(' ')[1]).

      • Verifies the token using jsonwebtoken.verify with the provided secret (process.env.SECRET).

      • If the verification is successful, attaches the decoded token to the req. user property for later use in subsequent middleware or route handlers.

      • Calls the next() function to pass control to the next middleware or route handler in the chain.

  2. Error Handling:

    • If an error occurs during token verification (e.g., invalid token or missing token), the catch block is executed.

    • Responds with a 401 status (Unauthorized) and a JSON object indicating the error.

  3. Export Middleware:

    • Exports the verifyTokenMiddleware for use in other parts of the application.

  4. Note:

    • This middleware assumes that the JWT is included in the Authorization header using the "Bearer" scheme.

    • The secret used for token verification is fetched from the environment variable process.env.SECRET.

    • The decoded token is attached to the request (req.user) for downstream use in route handlers.

By incorporating this middleware into routes that require authentication, developers can ensure that only requests with valid JWTs proceed to the intended route handlers. This enhances the security of protected routes in a Node.js application.

Routes:

We will be creating 2 different routes, userRoutes and imageRoutes.

userRoutes:

const express = require('express')

const {
    signupUser,
    loginUser
} = require('../Controllers/userControllers')

const router = express.Router()

router.post('/login', loginUser)

router.post('/signup', signupUser)

module.exports = router

  1. Import Dependencies:

    • express: The Express.js web application framework.

  2. Import Controllers:

    • Import the signupUser and loginUser functions from the userControllers module. These functions likely handle the logic for user signup and login, respectively.

  3. Create Express Router:

    • Create an instance of the Express Router using express.Router() and assign it to the variable router.

  4. Define Authentication Endpoints:

    • Define two authentication endpoints using the router.post method:

      • /login Endpoint:

        • Handles POST requests for user login.

        • Calls the loginUser function from the userControllers module.

      • /signup Endpoint:

        • Handles POST requests for user signup.

        • Calls the signupUser function from the userControllers module.

  5. Export the Router:

    • Export the configured Express router to make it available for use in other parts of the application.

  6. Note:

    • The loginUser and signupUser functions are assumed to be implemented in the userControllers module and handle the business logic for user login and signup, respectively.

    • The router sets up the specific URL endpoints (/login and /signup) for handling authentication-related requests.

By using this router, developers can modularize the routing logic for user authentication in their Express.js application. The actual implementation of the authentication logic is expected to be in the corresponding controller functions (loginUser and signupUser).

imageRoutes:

const express = require('express')

const { upload } = require('../upload')

const {
    postImage,
    getImages,
    updateImage,
    deleteImage,
 } = require('../Controllers/imageController')

const verifyTokenMiddleware = require('../Controllers/verifyTokenMiddleware'); 

const router = express.Router()

router.use(verifyTokenMiddleware);

router.post('/add', upload.single("image"), postImage)

router.get('/get/hehe', getImages)

router.patch('/update/:id', updateImage)

router.delete('/delete/:id', deleteImage)

module.exports = router

  1. Import Dependencies:

    • express: The Express.js web application framework.

    • upload: It is assumed to be a module for handling file uploads using Multer (not provided in this snippet).

    • imageController: Import functions for handling image-related operations like posting, getting, updating, and deleting images.

    • verifyTokenMiddleware: Middleware function for verifying JSON Web Tokens (JWTs) to ensure authentication.

  2. Create Express Router:

    • Create an instance of the Express Router using express.Router() and assign it to the variable router.

  3. Use Token Verification Middleware:

    • Use the verifyTokenMiddleware by calling router.use(verifyTokenMiddleware). This ensures that token verification is performed for all routes defined after this middleware.

  4. Define Image Endpoints:

    • Define several image-related endpoints using the router.post, router.get, router.patch, and router.delete methods:

      • /add Endpoint:

        • Handles POST requests for adding a new image.

        • Utilizes the upload middleware (assumed to be configured for handling file uploads using Multer) with upload.single("image").

        • Calls the postImage function from the imageController.

      • /get/hehe Endpoint:

        • Handles GET requests for retrieving images.

        • Calls the getImages function from the imageController.

      • /update/:id Endpoint:

        • Handles PATCH requests for updating the details of a specific image.

        • Calls the updateImage function from the imageController.

      • /delete/:id Endpoint:

        • Handles DELETE requests for deleting a specific image.

        • Calls the deleteImage function from the imageController.

  5. Export the Router:

    • Export the configured Express router to make it available for use in other parts of the application.

  6. Note:

    • The verifyTokenMiddleware ensures that all subsequent routes are protected and require a valid JWT for access.

    • The actual implementation of image-related operations is expected to be in the corresponding controller functions (postImage, getImages, updateImage, deleteImage).

    • The /add route uses the upload middleware to handle file uploads, assuming upload.single("image") is configured for handling single-file uploads with the field name "image."

Testing the Endpoints

Utilize tools like Postman to thoroughly test each API endpoint. Verify that file uploads, updates, and deletions function as intended.

Reflection and Future Enhancements

Reflect on your project journey and highlight key learnings. Consider potential future enhancements such as:

  • User Interface: Build a user-friendly front end to complement the powerful backend.

  • User Roles: Implement different roles for users, allowing for more granular control over file management.

  • Additional Security Measures: Explore other security measures beyond Helmet, such as encryption for stored files.

GitHub Repository

To facilitate collaboration and showcase your project, create a GitHub repository. Share the link in your blog post, allowing others to explore your code, contribute, and learn from your project.

GitHub Repository

Conclusion

Congratulations on reaching this point in our File Management System project! We've covered essential technologies, set up our backend, implemented security measures, and ensured that only authenticated users can access certain functionalities.

Did you find this article valuable?

Support Dhyan Tech!! by becoming a sponsor. Any amount is appreciated!

Node.jsExpressMongoDBJWT