Laravel Queue Workers, Scheduling and systemd in Production

A Laravel application can return a successful HTTP response while its invoices, email, imports, or webhooks remain stuck in a queue. Production operations therefore need to watch the worker and scheduler as independent services. A working SMTP test proves connectivity, not that the job carrying a customer email reached a worker or completed.

This guide assumes a Laravel application on a Linux host with systemd, PHP CLI, and an existing queue backend. Paths and service names are examples; use the queue connection and queue names configured by your application. A queue worker may execute financial or provisioning jobs, so do not blindly retry failed jobs or clear a backlog without reviewing idempotency.

1. Identify the queue connection and the job types

Start with application configuration, not a generic worker command. Laravel can use database, Redis, SQS, or another supported backend. Determine which queue each job uses; a worker listening only to default will not process jobs sent to emails or provisioning.

php artisan about
php artisan queue:failed
php artisan schedule:list

Run these commands from the deployed application directory under the intended service account. Avoid printing .env in terminal recordings or support tickets. Inspect pending jobs by backend and classify them: email, payments, provisioning, and notifications have different retry risks.

2. Run a worker under supervision

queue:work is a long-running process. Laravel recommends a process monitor so the worker is restarted after failure and after deployments. A minimal systemd unit might look like this; replace the account, path, queue names, and limits with reviewed values.

[Unit]
Description=Example Laravel queue worker
After=network-online.target

[Service]
Type=simple
User=app-user
WorkingDirectory=/var/www/example-app
ExecStart=/usr/bin/php artisan queue:work --queue=emails,default --sleep=3 --tries=3 --timeout=90
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Set the worker timeout shorter than the backend's retry-after/visibility timeout to avoid two workers processing the same job concurrently. The precise values depend on queue driver and longest expected job. Restart policies should be bounded by monitoring so a crash loop is visible, not silently repeated forever.

3. Keep one scheduler mechanism

Laravel's scheduler needs a system cron entry or a timer that invokes schedule:run each minute. Do not install both mechanisms on the same application unless you intentionally account for duplicate executions. A dedicated systemd timer can work well when unit state is monitored. Also use per-task overlap protection for expensive scheduled jobs where the application supports it.

php artisan schedule:list
systemctl list-timers --all --no-pager
crontab -l

Check the service account's cron, system crontabs, and systemd timers. Running schedule:run manually proves a single invocation works but does not prove the minute-by-minute mechanism survives a reboot.

4. Deploy without dropping queued work

Long-running workers hold the old application code in memory. After a successful release, Laravel's queue:restart signals workers to exit gracefully; the supervisor starts them with the new code. Verify that the signal reaches the cache backend used by the application.

php artisan queue:restart
systemctl status example-worker --no-pager
journalctl -u example-worker --since '15 minutes ago' --no-pager

Do not restart workers halfway through a schema change that the old or new code cannot handle. Review whether queued payloads from the old release remain compatible with the new code. Where jobs create external resources or charge customers, design idempotency before relying on retries.

5. Monitor delivery, backlog, and failure

Track queue depth, oldest job age, worker heartbeat, failure count, and delivery latency. For email OTP, follow the specific chain: OTP generated → record committed → mail job dispatched → worker picked it up → SMTP accepted → customer mailbox received it. mail_sent in an application log may mean the SMTP server accepted the message, not that the recipient read it.

Use queue:failed for investigation, then inspect job type and external side effects before retrying. Review the exception and payload with access controls; job payloads may contain personal or sensitive data. Never paste full failed-job payloads into public issue trackers.

Troubleshooting and common errors

• Jobs pending but worker is active: confirm the worker is listening to the actual queue and connection.
• Same job runs twice: compare timeout, retry-after, worker crashes, and idempotency keys.
• Scheduler list is correct but jobs never run: check the cron/timer user, working directory, PHP binary, and service logs.
• Email test succeeds but OTP does not arrive: trace the notification dispatch, queue, worker, SMTP response, and mail failure log separately.
• Worker fails after reboot: inspect WorkingDirectory, service dependencies, permissions, and the first boot journal rather than only restarting it.

Security considerations

Run workers with the least filesystem and database privileges required by their jobs. Keep secrets in the application's secure configuration, not the systemd unit file or command line. Limit who can retry payment or provisioning jobs. Record operator actions when replaying or discarding failed work.

FAQ

Can I replace the queue worker with synchronous jobs?

Synchronous execution can hide a broken worker but increases request latency and couples delivery to the HTTP response. Fix the actual queue path for jobs that are intended to be asynchronous.

Do I need both cron and a systemd timer?

No. Choose one scheduler trigger and verify it runs exactly once per minute under the correct account.

Is queue:retry all safe?

Not for every system. A failed job may have already called an external payment or provisioning API. Review and retry only jobs whose side effects are understood and idempotent.

Continue reading

Pair this with Linux server health monitoring and backup and restore planning. Ask East4Serv to review worker and scheduler reliability for your deployment.

References: Laravel queues and Laravel task scheduling.