Resumable Chunked File Uploads with Laravel, Next.js, and Redis: A Step-by-Step Guide

2026-10-14

Why Resumable Chunked Uploads Matter for Your AI Agent

Imagine an AI agent that tells you “the dataset is uploaded” after you drag a 2 GB CSV into the web UI. Behind the scenes the browser tried to push the whole file in a single HTTP request. The network hiccups, the request times out, and the agent proudly reports success while the file never reached the server. The next step—training a model—fails because the data is missing.

The pattern that saves you from this false confidence is simple: break every upload into small, independent chunks, store the progress in a fast key‑value store, and let the client retry only the missing pieces. When the agent finally says “upload complete,” you can be sure the file really exists on the back‑end.

Architecture at a Glance

Front‑end: Next.js with TypeScript and Tailwind CSS. Handles slicing, parallel uploads, and UI feedback.
Back‑end: Laravel 10 API routes that accept chunks, write them to temporary storage, and assemble the final file.
State store: Redis holds a short‑lived record for each upload (total chunks, which chunks arrived, file metadata).
Storage: Laravel’s Storage facade (local, S3, etc.) stores the finished file.

Step 1 – Front‑End Chunking Logic

The browser can read a File object in slices. We expose a tiny TypeScript hook that turns a File into a series of Blob chunks, each 1 MiB by default. The hook also generates a UUID that identifies the upload across requests.

TypeScript Hook

import { useState } from 'react';
import { v4 as uuidv4 } from 'uuid';

export interface ChunkUploadStatus {
  uploaded: number[];
  total: number;
  id: string;
}

export function useChunkedUploader(
  file: File,
  chunkSize = 1 * 1024 * 1024 // 1 MiB
) {
  const [status, setStatus] = useState<ChunkUploadStatus>({
    uploaded: [],
    total: Math.ceil(file.size / chunkSize),
    id: uuidv4(),
  });

  async function uploadChunk(index: number) {
    const start = index * chunkSize;
    const end = Math.min(start + chunkSize, file.size);
    const blob = file.slice(start, end);
    const form = new FormData();
    form.append('upload_id', status.id);
    form.append('chunk_index', index.toString());
    form.append('chunk', blob, file.name);

    await fetch('/api/upload/chunk', {
      method: 'POST',
      body: form,
    });

    setStatus(prev => ({
      ...prev,
      uploaded: [...prev.uploaded, index],
    }));
  }

  async function start() {
    const promises = [];
    for (let i = 0; i < status.total; i++) {
      promises.push(uploadChunk(i));
    }
    await Promise.all(promises);
    // Notify the server that all chunks are sent
    await fetch('/api/upload/complete', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ upload_id: status.id, filename: file.name }),
    });
  }

  return { status, start };
}

The hook does three things:

Creates a unique upload_id that both client and server use to coordinate.
Sends each chunk in parallel (you can throttle the concurrency for bandwidth‑limited environments).
Calls a “complete” endpoint once all chunks have been acknowledged.

Step 2 – Laravel Endpoint for a Single Chunk

On the server we expose a POST /api/upload/chunk route. It validates the incoming multipart request, writes the chunk to a temporary folder, and updates a Redis hash that tracks which chunks have arrived.

Route Definition (routes/api.php)

use App\Http\Controllers\ChunkUploadController;

Route::post('/upload/chunk', [ChunkUploadController::class, 'store']);
Route::post('/upload/complete', [ChunkUploadController::class, 'finalize']);

Controller Skeleton

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use Illuminate\Support\Facades\Redis;
use Illuminate\Support\Facades\Storage;

class ChunkUploadController extends Controller
{
    public function store(Request $request)
    {
        $request->validate([
            'upload_id'   => 'required|string',
            'chunk_index' => 'required|integer|min:0',
            'chunk'       => 'required|file',
        ]);

        $uploadId   = $request->input('upload_id');
        $index      = $request->input('chunk_index');
        $chunkFile  = $request->file('chunk');

        // Store chunk in a temporary path: storage/app/tmp/{uploadId}/{index}
        $tmpPath = "tmp/{$uploadId}";
        Storage::makeDirectory($tmpPath);
        $chunkPath = "{$tmpPath}/{$index}";
        Storage::put($chunkPath, file_get_contents($chunkFile));

        // Record receipt in Redis
        Redis::hset("upload:{$uploadId}", "chunk:{$index}", 1);
        // Optionally set an expiration so abandoned uploads disappear
        Redis::expire("upload:{$uploadId}", 3600);

        return response()->json(['status' => 'ok']);
    }

    // ... finalize method below ...
}

The Redis key upload:{uploadId} becomes a hash where each field chunk:{index} is set to 1 when the corresponding piece lands on disk. Because hset is idempotent, retrying a failed chunk does not corrupt state.

Step 3 – Tracking Progress in Redis

Redis is perfect for this job: it lives in memory, offers atomic hash operations, and expires keys automatically. The hash stores:

filename – original name (set on the “complete” request).
total_chunks – calculated by the client and sent once.
chunk:{n} – flag for each received piece.

Storing Metadata on Completion Request

public function finalize(Request $request)
{
    $request->validate([
        'upload_id' => 'required|string',
        'filename'  => 'required|string',
    ]);

    $uploadId = $request->input('upload_id');
    $filename = $request->input('filename');

    // Pull the hash to see how many chunks we have
    $hash = Redis::hgetall("upload:{$uploadId}");
    $totalChunks = (int) ($hash['total_chunks'] ?? 0);
    $received = 0;
    foreach ($hash as $field => $value) {
        if (str_starts_with($field, 'chunk:') && $value == 1) {
            $received++;
        }
    }

    if ($received !== $totalChunks) {
        return response()->json([
            'status'  => 'incomplete',
            'missing' => $totalChunks - $received,
        ], 409);
    }

    // All chunks present – assemble the file
    $this->assembleFile($uploadId, $filename, $totalChunks);

    // Clean up Redis entry
    Redis::del("upload:{$uploadId}");

    return response()->json(['status' => 'assembled']);
}

Step 4 – Assembling the Final File

The assembly step streams each chunk back into the destination file. Streaming avoids loading the whole dataset into PHP memory, which is crucial for multi‑gigabyte uploads.

Assembly Helper

protected function assembleFile(string $uploadId, string $filename, int $totalChunks)
{
    $destPath = "uploads/{$filename}";
    // Open a writeable stream using Laravel's storage driver
    $destStream = Storage::writeStream($destPath);
    if ($destStream === false) {
        throw new \Exception('Failed to open destination stream');
    }

    for ($i = 0; $i < $totalChunks; $i++) {
        $chunkPath = "tmp/{$uploadId}/{$i}";
        $chunkStream = Storage::readStream($chunkPath);
        stream_copy_to_stream($chunkStream, $destStream);
        fclose($chunkStream);
        // Remove the chunk file after it has been copied
        Storage::delete($chunkPath);
    }

    fclose($destStream);
    // Remove the now‑empty temporary directory
    Storage::deleteDirectory("tmp/{$uploadId}");
}

After this runs, the agent can safely call the next API (e.g., “train model”) because the file is guaranteed to exist in storage/app/uploads (or an S3 bucket if you configured Laravel accordingly).

Step 5 – Resuming After a Failure

If the network drops after 30 % of the chunks have been sent, the client can ask the server which pieces are still missing. This is a tiny GET endpoint that reads the Redis hash and returns an array of missing indexes.

Missing‑Chunk Endpoint

public function missingChunks(Request $request)
{
    $uploadId = $request->query('upload_id');
    $hash = Redis::hgetall("upload:{$uploadId}");
    $total = (int) ($hash['total_chunks'] ?? 0);
    $missing = [];

    for ($i = 0; $i < $total; $i++) {
        if (!isset($hash["chunk:{$i}"])) {
            $missing[] = $i;
        }
    }

    return response()->json(['missing' => $missing]);
}

The front‑end hook can call this endpoint on start‑up, filter out already‑uploaded chunks, and continue without re‑sending the whole file. Because the Redis hash expires after an hour, abandoned uploads clean up automatically.

Step 6 – UI Feedback with Tailwind CSS

Users need to see progress and know when a retry is happening. The following Tailwind component renders a progress bar that updates as status.uploaded.length grows.

Progress Bar Component

function UploadProgress({status}) {
  const percent = Math.round((status.uploaded.length / status.total) * 100);
  return (
    <div className="w-full max-w-xl mx-auto mt-6">
      <div className="flex justify-between mb-1">
        <span className="text-sm font-medium text-gray-700">{status.uploaded.length}/{status.total} chunks</span>
        <span className="text-sm font-medium text-gray-700">{percent}%</span>
      </div>
      <div className="w-full bg-gray-200 rounded-full h-4">
        <div
          className="bg-blue-600 h-4 rounded-full transition-all duration-300"
          style={{ width: `${percent}%` }}
        />
      </div>
    </div>
  );
}

When the “complete” request returns status: assembled, you can replace the bar with a green checkmark and enable the next UI step (e.g., “Start training”).

Step 7 – Security and Validation

Even though the upload flow is chunked, you still need to protect it:

Authentication: All endpoints are behind Laravel’s auth:sanctum middleware. The front‑end automatically includes the bearer token.
File type whitelisting: Validate MIME type and extension on the “complete” request, not per‑chunk, to avoid false negatives.
Size limits: Enforce a maximum total size (e.g., 5 GiB) by checking total_chunks * chunk_size on the first chunk.
Rate limiting: Laravel’s throttle middleware prevents a rogue agent from hammering the server with tiny chunks.

Step 8 – Quick Test with cURL

Before wiring the UI, verify the back‑end works with a simple script. The following Bash snippet uploads a 5 MiB test file in 1‑MiB chunks.

#!/usr/bin/env bash
FILE="test.bin"
CHUNK=1048576 # 1MiB
UPLOAD_ID=$(uuidgen)
TOTAL=$(( $(stat -c%s "$FILE") / CHUNK + 1 ))

# Send each chunk
for i in $(seq 0 $((TOTAL-1))); do
  dd if="$FILE" bs=$CHUNK skip=$i count=1 2>/dev/null | \
    curl -X POST http://localhost/api/upload/chunk \
      -F "upload_id=$UPLOAD_ID" \
      -F "chunk_index=$i" \
      -F "chunk=@-;filename=$FILE"
done

# Tell Laravel we are done
curl -X POST http://localhost/api/upload/complete \
  -H "Content-Type: application/json" \
  -d "{\"upload_id\":\"$UPLOAD_ID\",\"filename\":\"$FILE\"}"

If the final response is {"status":"assembled"}, the whole pipeline works and your agent can safely proceed.

Takeaway Rule

Always make large uploads idempotent, chunk‑based, and state‑driven with a fast store. By isolating each piece, you give the client the ability to retry only what failed, and you give the server a cheap way to know when the whole file is present. The rest of the guide shows how Laravel, Redis, Next.js, Tailwind, and TypeScript combine to implement that rule without a heavyweight orchestrator.

What to Try Next

If you’ve followed the steps, your agent now has a reliable “file is ready” signal. The next logical piece is to hook the upload completion event into a background job (Laravel Queues) that starts model training, logs progress, and reports back to the UI. That extension keeps the same resumable philosophy—jobs can be retried, state lives in Redis, and the agent never sees a half‑finished operation.