Running Laravel in Amazon Elastic Beanstalk

Laravel PHP framework has been the framework of my choice for the past 12 months – it’s elegant, trendy and full-featured. Official documentation is comprehensive so deploying both web application and workers is a piece of cake, but what if you want to run it in AWS Elastic Beanstalk environment?

This tutorial will not only help you run Laravel’s scheduled tasks and queued jobs in AWS worker environment, but also will prove once again how easy it is to extend Laravel standard functionality.

Elastic Beanstalk: peculiarities

Amazon offers a special type of EB environment for workers – a ‘worker’ environment. While AWS can run both scheduled tasks and queued jobs, the flow is slightly different from the standard Laravel flow:

Laravel queue flow

In the standard flow, Laravel pushes jobs to the queue and another (or the same) instance of Laravel polls for new jobs from the queue. Scheduled tasks are executed by Laravel’s own scheduler that is invoked every minute from the UNIX cron tab.

When it comes to AWS EB, we cannot install our cron tabs directly and we cannot (or at least, we should not) pop from the queue directly:

AWS EB worker flow

Instead, a daemon maintained by AWS will send POST requests to our instances informing us of scheduled tasks subject to run and new queue messages. While this may not sound terribly complicated, Laravel (as of 5.2) doesn’t support neither out of box – scheduled tasks are expected to be run in the console and queues are supposed to be processed by queue workers.

Implementation

Scheduler

Let’s start with scheduled tasks. We want to be able to run whatever is executed by php artisan schedule:run from a webhook. We don’t want to create separate hooks for each scheduled task because:

  • We want to rely on Laravel’s own scheduler – it’s easier to read, doesn’t require devops skills, stays in the application (will be part of the repo) and not outside of it;
  • Our local and development environments may not be AWS-powered, and we don’t want to have two separate processes for scheduled tasks for AWS and non-AWS deployments;
  • Too many webhooks is messy, we don’t want many methods that will only be used by AWS.

This is our final controller method that runs all scheduled tasks, it’s very similar to Laravel’s own ScheduleRunCommand::class:


/**
 * @param Container $laravel
 * @param Kernel $kernel
 * @param Schedule $schedule
 * @return array
 */
public function schedule(Container $laravel, Kernel $kernel, Schedule $schedule)
{
    $events = $schedule->dueEvents($laravel);
    $eventsRan = 0;
    $messages = [];
    foreach ($events as $event) {
        if (! $event->filtersPass($laravel)) {
            continue;
        }
        $messages[] = 'Running: '.$event->getSummaryForDisplay();
        $event->run($laravel);
        ++$eventsRan;
    }
    if (count($events) === 0 || $eventsRan === 0) {
        $messages[] = 'No scheduled commands are ready to run.';
    }
    return $this->response($messages);
}

The following line is very important. As we know, Laravel will try to resolve any dependancies we list in the argument list, but in this case there will be an extra side effect that we need:


public function schedule(Container $laravel, Kernel $kernel, Schedule $schedule)

Our web process is using web kernel that doesn’t load scheduled console commands, but here we asked for the console kernel (Illuminate\Contracts\Console\Kernel) that Laravel will have to instantiate for us. While console kernel is being ‘made’, scheduled tasks are parsed and finally Laravel web app can be aware of them. Then Laravel will resolve Schedule for us, now that we have a list of scheduled tasks – Scheduler can work properly.

Important note: swap Kernel with Schedule in the argument list and method will stop working. Kernel has to be resolved first, because of it’s side-effect.

The rest of the method is quite straight forward and resembles ScheduleRunCommand a lot. We would love to re-use the existing class but it couldn’t be overridden and we didn’t want to use default console output formatter.

Queues

We tried to keep any overhead at minimum so we didn’t create any new Queue or Connection classes – it’s just one custom job class that is passed to framework’s default worker.

The final method looks like this:


/**
 * @param Request $request
 * @param Worker $worker
 * @param Container $laravel
 * @return array
 */
public function queue(Request $request, Worker $worker, Container $laravel)
{
    $this->validateHeaders($request);
    $body = $this->validateBody($request, $laravel);
    $job = new AwsJob($laravel, $request->header('X-Aws-Sqsd-Queue'), [
        'Body' => $body,
        'MessageId' => $request->header('X-Aws-Sqsd-Msgid'),
        'ReceiptHandle' => false,
        'Attributes' => [
            'ApproximateReceiveCount' => $request->header('X-Aws-Sqsd-Receive-Count')
        ]
    ]);
    try {
        $worker->process(
            $request->header('X-Aws-Sqsd-Queue'), $job, 0, 0
        );
    } catch (\Exception $e) {
        return $this->response([
            'Couldn\'t process ' . $job->getJobId()
        ], 500);
    }
    return $this->response([
        'Processed ' . $job->getJobId()
    ]);
}

Basically, all we did is extract SQS meta data from the HTTP headers and inject it into job, so it’s kind of HTTP to SQS adapter. We don’t need to delete the job or mark it as failed – everything is done by AWS. If we don’t return HTTP status code of 200 (let’s say we caught an exception) AWS will do the rest.

That’s it! Now we just need to add a couple of routes (just two routes for any number of scheduled tasks or queues!) and we are good to go.

AWS installation

Don’t forget to subscribe your AWS worker to corresponding SQS queues or SNS topics.

In order for your scheduler to get called every minute, provide a cron configuration file with your AWS application – you can either include it in your repo or add it somewhere at the last stage. Anyway, file should be called cron.yaml, should reside in the root folder of the application and should look like:


version: 1
cron:
 - name: "schedule"
   url: "/worker/schedule"
   schedule: "* * * * *"

Conclusion

Laravel once again proved to be a flexible, easy-to-extend framework.

Link to the full source code, an actual working package with Laravel and Lumen service providers: https://github.com/dusterio/laravel-aws-worker

Post by Denis Mysenko

Born in the snows of Siberia