[BUG] Logs are not sent in Laravel queues

[antoinelame]

Description

Logs from an asynchronous job in a Laravel queue are not sent to CloudWatch until you shutdown the queue process.

This is because php artisan queue:work does not close the process. Queue workers do not "reboot" the framework before processing each job.

How to reproduce

1. Create a new job:

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Log;

class TestJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function handle()
    {
        Log::info('The job has been executed');
    }
}

2. Create a new route:

use App\Jobs\TestJob;

Route::get('dispatch-job', function () {
    TestJob::dispatch();
});

3. Run the queue:

php artisan queue:work

Note: make sure that in your .env you do not use the sync queue driver.

4. Call GET /dispatch-job

You will receive the logs in CloudWatch only when you'll interrupt the queue:work command. Not before.

Related GitHub issues

• Buffer gets log in Laravel Queues #7

[mugisham]

Running php artisan queue:listen. seems to work but this is a less than ideal solution. This closes after each job and sends the logs, is there a way it can keep working without closing after each job as done by php artisan queue:work

[ninjacoder96]

any update on this? we are encountering this as well.

[olivier1208]

Finally found THE trick
Change the buffer size to 1 !!
That's it !

[mugisham]

Finally found THE trick

Change the buffer size to 1 !!

That's it !

What do you mean?

[olivier1208]

$handler = new CloudWatch($client, $groupName, $streamName, $retentionDays, 10000, $tags);
to
$handler = new CloudWatch($client, $groupName, $streamName, $retentionDays, 1, $tags);

If you look at the protected function write in Cloudwatch.php in the package you'll see the following

if (count($this->buffer) >= $this->batchSize) { $this->flushBuffer(); }

That's why

[mugisham]

Does this affect performance sending a log at a time

[mugisham]

https://stackoverflow.com/questions/46931742/cloudwatch-agent-batch-size-equal-to-1-is-it-a-bad-idea