Building scalable IT system for delivery from US to Russia: Drupal, Symfony2 and Yii2 compared

I was not posting to the blog for a long time, and finally it’s time to share my experience with new project. This post will also cover some badly structured thoughts about PHP frameworks :)

As part of Qwintry team, we’ve built a great b2b product in logistics sphere - Qwintry Logistics - delivery from US to Russia - and it was built on a new PHP framework, so it was a big deal for us - lots of new experience, and it’s great that it was not just experience but a business success as well :)

Qwintry Logistics (I will use QWL further in text to avoid typing these words again) is an IT system which provides our b2b customers (US stores and freight forwarders) a new way for a high-quality and affordable delivery to Russia, with simple API for integration. We are not directly competing with monsters like USPS or UPS for cross-border delivery, but we are doing very similar thing here - delivery from stores/warehouses to customer door (or pickup point - which in many ways can be more convenient for the end customer than courier delivery to door).

Under the hood the system connects Qwintry warehouses (pickup hubs, if we use right terminology) in Oregon and Delaware, multiple US companies for trucking freights to airports, IATA agents and companies that book air freight for us, customs brokers (to do customs clearance in Russia), and multiple delivery companies in Russia/Belarus/Kazakhstan. All these multiple partners are connected into single workflow, and we get beautiful tracking for each individual package as a result, without investing billions of dollars to build our own infrastructure (that’s where we differ from UPS and USPS):

Tracking for package from US to Russia

At the same time, the IT system and some know-hows and experience in automation of fulfillment and logistics, that we got while working on Drupal-powered Qwintry b2c website (freight-forwarding business with 100k+ registered customers) allow us to get the amazing quality and speed while keeping costs low and pricing matrix attractive.

Framework selection: Drupal8, Symfony2, Yii2

When we decided to start building the QWL website, our team had plenty of experience in Drupal 5/6/7 and some dirty hands in Drupal8, and decent experience in Symfony2 (several big projects on air), with my mind still being restless due to the fact that none of these frameworks completely met following requirements for the future QWL system:

  1. being blazing fast for authorized users and API calls
  2. configuration in code, no configuration in db
  3. ability to write less code, with the code being beautiful, clean, and fun to write
  4. being simple PHP platform to quickly hire developers without skyrocketing budgets

Why not Symfony2?

While a lot of members of our team was (and still are) in love with sf2 - I was not, since writing sf2 code was never a fun for me - never even remotely looking like fun. Yes, I’m lazy, and I’m probably not this perfect developer which sf2 is built for :)
I realized that the level of sf2 complexity and over-engineering is too big for me - very clearly - when I noticed that I kept forgetting what was the business (real) task I was trying to solve - just after five minutes of staring into a bunch of routing or configuration files in our not-too-complex project, or into Factory to generate some basic form. I just dived into the code and tried to solve issues that sf2 was throwing in my face most of time. Not business issues, but issues related to level of perfect abstraction Symfony2 provides me. I think that’s a fundamental difference between me and sf2 - is that I want to solve real business issues quickly without diving too much into this game of absolute extend-ability and clean dependency-injectionability this great and 100% testable framework provides :)
My opinion can change some day, but right now I believe that most projects around me and processes around me are not that complex, but they still have tight deadlines and the quicker I can provide real value to the business with my code - the better.

Why not Drupal?

At the same time all the problems of Drupal -

  • big db structure, where writing some raw db queries is a pain - you do ten joins to get basic data from 10 fields - (don’t even try to explain that to financial analytics who are great in what they do, but have very basic SQL knowledge and read-only access to db),
  • bad performance for authenticated users and very bad performance in node_load/node_save,
  • ugly code you need to write (well, comparing to Wordpress it’s a great code, but lets face it - it is not good comparing to modern frameworks with good ORM/AR),
  • configuration in db

- were still there, and we were very well aware about all these issues since Qwintry.com is a big project with 30+ custom modules created by our team - starting from Endicia/USPS integration and ending with our own coupons engine, full-fledged order picking system UI, and highly customized referral program - it’s all there, and we’ve been maintaining it for several years with impressive number of incoming packages and orders, so every line of code that could fail - failed at least several times :)

In big projects during relatively long maintenance periods you quickly realize that this Drupal code has issues:

<?php
$order
->field_paid[LANGUAGE_NONE][0]['value'] = 1;
node_save($order);
qwintry_log($order->nid, 'Order was paid');
?>

Because sometimes node_save with single changed field still fail due to innodb locks or whatever, and you at least need to wrap it into try { } catch blocks like this:

<?php
$order
->field_paid[LANGUAGE_NONE][0]['value'] = 1;
try {
 
node_save($order);
 
qwintry_log($order->nid, 'Order was paid');
} catch (
Exception $e) {
 
drupal_set_message('Critical warning for operator');
 
qwintry_log($order->nid, 'Exception thrown during invoicing');
}
?>

Why am I posting this example? Because when your projects grow that big so you have everyday problems like that (and you write a lot of custom code for new features, so your custom code base size is comparable to framework size) - it may be a good sign that you probably need some lower level framework (lower than Drupal), where transactions are there for all objects, working mostly automatically, and where it is a lot cheaper in terms of cpu and speed to save objects to db in ‘traditional’ way of framework - so you don’t do ugly db_insert or db_query(“INSERT INTO {table}”) instead of node_save when you need to process just 30-40 objects at a time.

I was even considering plain PHP, but it would be stupid in 2014 to invent own wheel :)
Another idea was to use Laravel, and I’m pretty sure it would be a viable option - never heard bad things about it - not too complex, but still elegant and fast.
But, instead of that, we decided to try Yii2 which was in alpha quality by the time when we started - and it turned out to be a great framework.

Why Yii2?

Historically, Yii is the most popular PHP framework in Russia, and probably in China (no links to chinese IT websites, sorry) - since the author of the framework, Qiang Xue, is Chinese (lives in US). It is not hugely popular in US, as far as I know.

habrahabr.ru vote for PHP framework in 2013:

Big popularity is not a sign that the framework is good (or bad) - but it usually means you can find developers easier and faster. Finding good Drupal developers is a big issue. Finding sf2 developers is easier, but still can be a problem. Finding Yii developers is easier than finding sf2 developers :) You have to be careful while testing them since the learning curve is not that big in Yii - so there can be quite a number of bad programmers among these guys - those PHP guys that Ruby and Python devs make jokes of :)

I have a lot of friends in Russia who have been using Yii1 for years, but during these years I’ve been successfully doing Drupal projects and we honestly never had a lot of projects where custom code base can be bigger than framework+contributed modules code base, and good speed under highload was not a big factor, so for that type of projects it would be stupid (and too expensive) to use Yii.
I still think that for those kind of projects (where you don’t write big - and 2-3 small custom modules+customized theme is not big - amounts of custom code) Drupal is a perfect engine. You basically develop most of stuff at the speed of prototyping, you have access to huge amount of great contributed modules and to drush - it’s just amazing how quick the complex web development can be nowadays.
But now, when I look back and think of projects that we’ve implemented in Drupal and which had huge amount of development hours invested in it (thousands of hours on code writing) - I think that Drupal was not the perfect solution for these - it gives more issues than value.

So, I used Yii1 in my sandboxes for some time. Than, when we finally needed some PHP framework for serious code writing - we started to use Symfony2 (read my blog post about this selection here) - the Yii1 already felt outdated, while Yii2 was not even remotely ready for usage. And the Symfony2 was exciting to try, giving us the new modern tools like Composer and Twig.

The Yii core devs decided to rewrite second version of their framework almost from scratch, just like Fabien did with Symfony2. It was a long-lasting project, and there was a risk to loose all the community that will switch to fresh sf2, Laravel and Phalcon and never come back. But now I think it was worth it (and I don’t think Fabien regret his sf2 rewrite, as well). The beta of Yii2 was released in April of 2014 (and it was a raw beta). We started working with Yii2 and QWL project in March of 2014 so we had several funny issues related to alpha quality of code - but all of them were resolved with later releases.

So, in QWL, we needed great performance, and the code was mostly custom (a bunch of external APIs to integrate; a huge bunch of docs to generate from db objects - mostly for customs brokers; and a need for good own API) - so it was obvious that it’s not where Drupal shines.
And it was a great moment to try Yii2 since it was close to production quality already.
New kid on the block, with all the goodness of modern PHP development: composer, namespaces, autoloading, great ActiveRecord, prepared for REST APIs building. All that was implemented by slim and very simple framework core, and it works with a great real speed.

What is a real speed? I use that term to distinguish between speed of cached page performance and speed of page performance without cache.
In Symfony2 (which is not fast at all, comparing to Yii) you can’t even disable all the levels of caching, so I had a number of issues with that - when your routing annotations are cached, when your Yaml configuration is cached, when your Doctrine objects and schemas are cached, and when your Twig templates are cached, and when your DI LazyServiceMapPass is probably cached - and you can’t disable all these cache layers completely in development environment due to the complexity of framework features - it’s easy to get lost when debugging something, and if you delete all the cache - the framework will be surprisingly slow while warming up all that.

In Yii2 you can disable all caching, and the dev environment with all the debug panels will still be fast! You can (and you should) cache everything in production to be super-fast, but it’s really amazing how fast it is in development mode. It was one of the reasons to choose this framework.
Another reasons:

1) Configuration of components is dead simple, and is written in PHP:

if you see something like

<?php
'sms' => [
    
'class' => 'app\components\sms\SMSC',
    
'user' => 'username',
    
'key' => 'xxx'
],
?>

in website configuration file - you know that you can access this object through Yii::$app->sms and it’s implementation is a class like this:

namespace \app\components\sms;

class SMSC extends Component {
  public $user;
  public $key;
  /* some methods here */
}

so configuration params are transformed into class properties by default, and you are not required to write code for additional processing to transform configuration directives to something the class can use. Yes, it can be restrictive, and Symfony2 developers can give a bunch of examples where such configuration approach is lacking flexibility and may be it is not exactly self-documenting, but in most cases that’s what you need - in most cases you don’t want to know a heck about DI service compiler passes, and you don’t want to create additional XML with service definition (which will be cached, remember? good luck with typos in these xmls) - you just want to create this component and start using it right away.

2) Most of decisions are made by core developers

The Yii2 core feels monolithic compared to sf2 (so core developers made own Logger, own ActiveRecord, and great RBAC permission system is also in core) - but it felt good when I was doing real work, comparing to alternatives with bigger amount of features (using Doctrine+ACL).
I like the fact that Bootstrap3 is there in core of Yii2 and there are widgets to do less typing (e.g. \yii\bootstrap\Modal for modals). I have had experience injecting Bootstrap3 into Drupal7 and I can’t say it’s a big pleasure to do.

At the same time, the culture of contribution in Yii is weak comparing to Drupal world.
You don’t use a lot of contributed modules when you build Yii project - you mostly rely on the core and your own code. Of course, there are a lot of contributed modules but their number is ridiculously low comparing to number of Drupal user-contributed modules - at least, that was my impression. The level of extendability is significantly lower in Yii2 comparing to Drupal, but it didn’t bug me a lot since I know how complex and slow infinite extendability can get. Yii2 is still in many ways better extendable than Drupal - for example, it’s very easy to use your own user and session processing, and customize the registration forms - and this flexibility is more important in big projects than a way to inject something in theme hooks or change menu routing details via hook_menu_alter (Drupal shines here).

Now, in Yii2, most of contributed modules are hosted on GitHub (and Packagist - for composer).

3) ActiveRecord is a lot less typing than ORM like Doctrine.

Active Record is a concept that somehow mixes the object representing each table row and a super-object that can be used to retrieve specified table rows - in single class. Sorry for this terrible explanation, but here is a simple example:

Drupal 7

$shipment = node_load(db_query("SELECT entity_id FROM {field_data_field_tracking} WHERE field_tracking_value=:tracking", [':tracking' => $tracking])->fetchField()); // yes, I know about db_select, but using it in a bit more complex cases looks even uglier than that
$shipment->field_is_sent[LANGUAGE_NONE][0]['value'] = 1;
node_save($shipment);

Yii2

$shipment = \app\models\Shipment::findOne(['tracking' => $tracking]);
$shipment->is_sent = 1;
$shipment->save();

Symfony2

$em = $this->getDoctrine()->getManager(); // if you're in controller
$shipment = $em->getRepository('PixeljetsShipmentBundle:Shipment')->findOneBy(['tracking' => $tracking]);
$shipment->setIsSent(1);
$em->flush();

In ORM (Doctrine) the super-object whose job is to fetch rows from db is EntityRepository, while each db row is represented by Entity object, it means you need separate class for Repository and separate class for Entity, so in the code above the hierarchy of ORM is:
ShipmentRepository -> Shipment

In yii2 ActiveRecord this super-object concept is mixed into ActiveRecord class which also holds event handlers related to each-db-row objects and stuff like that.
Obviously, Active Record is created by developers who are lazy and hate typing and creating a bunch of classes, while ORM is more academic, loosely coupled, supporting SOLID, and beautiful concept, which works better for enterprise development.
But the resulting code of retrieving objects (and this is the code that you write ten times per day) of ActiveRecord is more elegant and short.

The underlying code of ActiveRecord is probably less than 10% of Doctrine code size and you can understand how it works in one day. Then try to explore the Doctrine code some day (I only did a quick overview, and yes, no surprise it’s big and complex)

Some beautiful ideas of Yii2 AR I think were borrowed from Eloquent (Laravel AR), for example the relations:

<?php
class Shipment extends ActiveRecord {
  function
getAuthor() {
    return
$this->hasOne(User::className(), ['id' => 'user_id']);
  }
}
?>

now you can get owner of shipment:

<?php
$shipment
= Shipment::findOne($id);
echo
$shipment->author->username; // separate lazy sql query against users table executed here
?>

or, you can do it via join:

<?php
$shipment
= Shipment::find()->where(['id' => $id])->joinWith('author')->one(); // select * from shipment left join user ...
echo $shipment->author->username; // no sql query here!
?>

Isn’t it beautiful and expressive?
While I always had problems remembering the arguments and names of Drupal zoo of db functions (in D7 it got even worse with EntityFieldQuery and db_select - I don’t really want to repeat myself so read more about this in my previous post) - it really lacked simplicity, straight-forwardness and “it works for everything in the system” impression, - and while I didn’t like Doctrine DQL and had bad experience extending this DQL syntax to add some condition to my search query - Yii2 simplicity was like a miracle for me.
You can fallback to raw sql in conditions when you need, still getting AR objects as a result:

<?php
// get 20 shipments which are older than 1 month or which author email is <a href="mailto:test@test.com">test@test.com</a>
$q = Shipment::find()->joinWith('author')->where(['author.email' => 'test@test.com']);
$q->orWhere('shipment.create_time < :time', [':time' => time() - 60*60*24*30]);
$shipments = $q->limit(20)->all();

foreach (
$shipments as $shipment) {
  echo
$shipment->tracking;
}
?>

By the way, Yii2 supports transactions out of the box, and you can save objects reliably and be sure that you don’t leave your db in a bad state.

Other things to note in Yii2:

Bower

- jQuery is in core but it is not glued into the code - it is downloaded as a Bower package, and the idea of Bower manager support via Composer looks very nice (minor issue: it seems to slow down the process of composer update process).
It means that Yii2 modules can specify dependencies to bower packages. (Bower is a package manager for frontend libraries, mostly for javascript libraries - built by Twitter and seems to gain some momentum now)

DB is primary, no schema generation from code

- DB is primary, while the code is secondary. Very important difference comparing to Symfony2 ORM and Drupal schema: if you remove your database completely, in Symfony2 you can get full schema of db from your ORM models, in Drupal you can do the similar thing, in Yii you can’t do that easily - the AR code ‘learns’ about your db structure on the fly. You need some good db admin software to do db design - but I think that SQLyog interface is superior to writing Drupal schemas by hand or using Symfony2 console utility. The Doctrine2 ability to generate ALTER queries from changed models still looks very cool :)
In Yii2, to do migrations, you go to db admin, alter the table using its ui, and copy&paste the resulting ALTER to migration file, that is later pushed to git. In db there is a separate table to remember migration states. This approach is pretty simple and gets things done.

Template layer

- the core template engine is plain PHP - the only thing I don’t like here is that it’s easy to forget to escape some user data, and it’s too much typing to escape everything:

<?= \yii\helpers\Html::encode($var) ?>

I don’t like this, so I made a shortcut function:

function e($var) {
  return \yii\helpers\Html::encode($var);
}

and now in templates I use

<?= e($var) ?>

- still not as beautiful as {{var}} in Twig, and not OOP, but I can live with it (for now).

BTW, there is a Twig integration for Yii2 - maintained by Yii core devs. Not tried it yet.

The big difference between Drupal templating and Yii2 templating is that Yii2 templates mean more typing and copy&pasting when you start - since you need to add real template (view) for each form, and for each page, and for each model CRUD, while in Drupal when you create new node type or simple form you don’t need to take care of creating any template, you need to think of it only if your need some fancy html structure.
At the same time, I found it easier to manage html structure of complex form in Yii2 than in Drupal7 (which has a great theme layer - but you can write a book on how you do specific things here - and that is a killer for newbies).

<div class="row">
        <div class="col-sm-3">       
        <?= $form->field($model, 'broker_status')->dropDownList(Shipment::getAvailableBrokerStatuses(), ['prompt' => '']) ?>
        </div>
        <div class="col-sm-4">       
        <?= $form->field($model, 'broker_message')->textarea() ?>
        </div>
</div>

to do this basic styling in Drupal you have to fight with form_alter (add #field_prefix to mimic the required structure) or override template completely which means you need to add separate theme callback - and while it is not a rocket science - lazy developers may feel a bit upset when designer says that he needs this element floating there and he can’t do it just via CSS, using default Drupal form html structure.
In Yii2 you go this extra-mile at the start, using Gii which generates CRUD for models from db table, and then you have precise control over the html, it is easy to modify and support - and all the the templates in the system work the same way - and you know exactly in which file some html structure of registration form located. In Drupal, during maintenance period of some complex project, you always need some time to figure out how this specific div was generated - was it some form_alter from contributed module adding it, or was it your div from your custom theme?

In Symfony2, the Twig engine is great, and I can’t say anything bad about it - it is similar to Yii2 approach. I’m afraid the Twig won’t save Drupal8 theme layer from complexity unless devs are ready for big functionality degradation (because Drupal flexibility is complex, and if you convert all these theme_ callbacks to .twig files - you will also get performance degradation that will slow down Drupal even further - and cache cleanup will be a bigger pain than it is now.. but we’ll see the result when Drupal8 is released)

Forms

In Yii2 forms are usually created on the base of some model.
That is a nice concept - ActiveRecord and form models have validation rules described in same syntax - with rules() and scenarios() methods inside model class. Note that the amount of typing is minimal, and you can pass anonymous function for custom validation rule - and it’s again no redundant typing, and quick result.

Client side validation is automatically generated from validation rules described in model class.

The nice ‘side effect’ of same validation rules used not just for forms, but for base Model class is that you can launch your validation for some ActiveRecord object simply by calling

<?php
if (!$shipment->validate()) {
$errors = $shipment->getErrors();
}
?>

Menu routing

When you need to create another route in Drupal7, you need to implement hook_menu() and append your new path to the registry of routes:

<?php
function devel_generate_menu() {
 
$items = array();

 
// Admin user pages
 
$items['admin/generate'] = array(
   
'title' => 'Generate items',
   
'description' => 'Populate your database with dummy items.',
   
'position' => 'left',
   
'page callback' => 'system_admin_menu_block_page',
   
'access arguments' => array('administer site configuration'),
   
'file' => 'system.admin.inc',
   
'file path' => drupal_get_path('module', 'system'),
  );
  return
$items;
}
?>

in Symfony2 you do the similar thing but there are plain routes there, no such metadata like menu title. So, you need to create xml file like this:

<?xml version="1.0" encoding="UTF-8" ?>

<routes xmlns="http://symfony.com/schema/routing"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://symfony.com/schema/routing http://symfony.com/schema/routing/routing-1.0.xsd">
    <route id="fos_user_registration_register" pattern="/">
        <default key="_controller">FOSUserBundle:Registration:register</default>
    </route>
</routes>

or, you can deal with annotations and specify routes via annotations in controller:

<?php
class PageController extends Controller
{
   
/**
     * @Route("/pages/{id}.{_format}", name="pages_view"})
     */
   
public function viewAction($id)
    {
   
// action code here
   
}
}
?>

I loved the idea of annotations at the first sight, but in practice, in development it turned out to be a pain due to caching issues (when something specified in annotations do not work, you have to cleanup cache every time you change your annotation).
The fact that you can specify routes with different ways across the same project is also frustrating for me.

By the way, in Drupal8 the way to specify menu routes is pretty much the same as #1 way in Symfony2 - via [modulename].routing.yml file.

and now the typical Yii2 routing:

<?php
class PageController extends Controller
{
   public function
actionView($id) {
  
// action code here
  
}
}
?>

that’s right - you don’t need to write anything to get the route into the routes registry, it’s just the convention that action* methods in controller are turned into routes. In this specific case the ?id=xxx query parameter is also required and system throws 403 error if it is not there. Of course, you need to add access callbacks to restrict access but you can do it later when you need it, via separate conventional methods in the controller - it is done by attaching behaviors.
In development you don’t even need to cleanup cache so new action starts working!
This whole idea that routes are not described separately can be shocking, but it works pretty well and it’s a breeze when you add a lot of ajax callbacks with separate paths.

You also get CLI interface (which is easy to extend by putting controller in special folder), access control, various logger types (logging to files, db, syslog, email with granular setup is available), basic web ui for rapid html and model generation (called gii), mongodb support, memcache and redis support for caching (file caching is default), sphinx support.

Implementation of specific features:

QWL project required simple API, and due to tight deadlines we decided to use non-REST approach for first version of api.
You can check how the api looks like in our docs.
This is where validate() method of ActiveRecord worked great and it allowed us to create less duplicating code for API functionality - basically the API methods use almost exactly same POST data and apply the same validation rules that are used in control panel when customer uses web browser to create the package - the errors are just rendered in different way and returned in JSON.

Some notes regarding API authentication: in Yii2 this part is pretty well thought through and there is a separate handbook page for this. In QWL, when customer uses the web panel - he is logged in through sessions and cookies, and during the api call the current user object is created on different authenticator mechanism - header tokens in our case - and this didn’t require a lot of developer time to implement.

Another interesting task was creating a dashboard where operator can take a quick look at the packages that require special attention (if they are sitting for too long in pickup point, or they have active issue flag, or some status in tracking is incorrect and unexpected, etc). That is a very important feature that allows us to monitor the health status for each part of logistics chain.

We have pretty big and advanced dashboard for operators on Qwintry b2c website (Drupal powered), and it is implemented using Panels and Views. It has 15+ panes with misc stats and to make it open in adequate time that human can wait (2 seconds) we had to move most of panes to ajax loader (that was a custom hack for Panels) - otherwise the loading times could be 10+ seconds.

In QWL, for now, the dashboard is simpler than that - but it’s still amazing that 4 panes with 80 packages (which are converted to real ActiveRecord objects, while in Drupal Views rows are not even converted to nodes - it’s just arrays of data - for better performance) are opened in less than 200ms, that means we don’t need to think of Redis caching or other performance tricks like ajax loading now - we spend this time on development of new features instead.

We also implemented flexible file storage for documents - when package is created - all the files are put to local filesystem for faster access and preview, and when the shipment is marked as “delivered” - all the files are moved to S3 for long term storage, to save local file space.
We’ve created File AR model with ‘storage’ field and when the ‘storage’ field is changed to s3 - files are copied to s3 - during afterSave hook of the model. We’ve used AWS sdk for php for this, and very thin Yii2 component so s3 is available as a service.

We’ve also integrated littlesms.ru api for SMS notifications - and it was also available as a service throughout the system so sms are sent like this:

<?php
Yii
::$app->sms->sendMessage($phone, $message)
?>

and in project config it is:
<?php
'sms' => [
           
'class' => 'app\components\sms\LittleSMS',
           
'user' => 'user111',
           
'key' => 'pw'
       
],
?>

when littlesms.ru was discontinued we switched to SMSC api, but we didn’t change any code using the sms service, we just created new (very simple) component on the base of SMSC code and changed the system config:

<?php
'sms' => [
           
'class' => 'app\components\sms\SMSC',
           
'user' => 'userSMSX',
           
'key' => 'pw2'
       
],
?>

both components had same method sendMessage (it could be the same Interface with sendMessage method or even an AbstractClass that both classes inherit) so the change was seamless for the remaining code.

Another cool feature of QWL is implementation of AMQP protocol for queues of tasks (unfortunately, still nothing like that is implemented in core).
For example, when we need to generate documents for broker, it can be an archive with 200+ pdf documents.
We use mpdf for pdf generation, and it is not quick at all, unfortunately. So it’s obvious that this is a long running task that needs to be executed asynchronously.

In Drupal, we would use Batch API, but Batch API has some flaws - if you occasionally close or refresh the page - the process won’t be finished. You also can’t process Drupal batch from cron (you need to use Drupal queues in that case - but they are not good for web browser usage). So, we decided to create scalable solution that will work both from web browser and cron execution. My teammates with bigger Yii experience used some code from older Yii1 projects here, with adaptation to Yii2.
We use RabbitMQ and ActiveRecord model which keeps the progress information.
We also have PHP daemon (which is based on Yii CLI controller) for workers - and monit utility for daemon monitoring. We also use custom Yii2 widget which shows bootstrap3 progressbar which is updated by javascript that uses ajax to check the task progress status.
The result looks like this:


We use this task queue when sending sms and email messages, generating pdf documents, generating document archives, and in other places.

The QWL also has a registry of courier companies and a set of methods so final delivery company is selected on the base of multiple parameters (pickup point selected by customer, address city, box size, box weight, etc) - but that’s a topic for separate post. Pickup points map contains merged locations from multiple courier companies.

Conclusion on framework selection

Yii2 is simple, and if you have good PHP background - it will be easy to learn.
When I was learning it, and looking into how things are done here, I got this great feeling that authors of Yii2 did a lot of stuff exactly the same way I would do it in a “perfect framework” that I would create.
Many times my thought was “These guys are lazy just like me” :) this great balance of simplicity and flexibility, the balance between having fun writing code and still good code readability in long term maintenance is hard to achieve.
I’m pretty sure some developers would say that Yii2 is a bit quick&dirty, and not perfectly testable (by the way, there is a Codeception integration in core of Yii2 development package - so it is definitely testable!), and services and configuration management are too simple here, and static calls to get services are ugly - most of it is true, especially comparing to sf2 - and I’m ok with that opinion - it’s just the matter of personal preference, experience, targets, number of developers in the project, and project type. I try to think about business targets most of time when we write our code. I saw lots of great talented developers that dream about 100% test coverage and spend hours discussing which PHP DI component is better - and then they fail meeting the deadlines or just distort the real task so it does not comply to customer needs, but fits into perfect architecture of system instead.
At the same time, nobody wants to get ugly unmaintainable code in repo, me neither.

It’s all about balance :) and for me, Yii2 seems to have good balance.
QWL project is in production, it’s used by customers, and it’s a pleasure to maintain and extend it. We’ll see how it goes.
So, for our team, it was a period of ~7 years of mostly Drupal development (since 2005), and then ~2 years when our team was building Drupal7 projects and Symfony2 projects, and now in 2014-2015 we have Drupal, Symfony2 and Yii2 projects to maintain. Luckily, bigger framework selection in our case does not mean increased complexity of development - it means quite the opposite - the tools now fit the tasks even better and we see better what is the right framework for the future project.

Comments

Thank you for sharing this!
What if you didn't use PHP but other lang/frameworks?

(I am a noob but fast learning Drupaler, and still never touched PHP, neither studied CS - still, delivered my first project by myself for 30k$)

21 December, 2014

Hi! this is a question answered earlier in comments to a previous post :)
And congratulations to you and your project!

21 December, 2014

Hi This is good example you provided to developers. It atleast gives good understanding when to use drupal, YII or any other framework.I appreciated the way you compare all three important php platforms.

22 December, 2014

Thank you Anton for sharing this with so much details! Great job!

"I realized that the level of sf2 complexity and over-engineering is too big for me"
This is very similar to what I feel, at least doing my first step in Drupal8 after previous Drupal version.

22 December, 2014

Nice article but you should also consider Laravel as it is very robust, scalable, fast and similar in syntax to yii2.

25 December, 2014

Would you be able to share the AMQP code you used for Yii2?

02 January, 2015

I think it’s a good idea, but it will require some time to make a module.
Have you tried https://github.com/sprytechies/yii2-resque ?
It is based on https://github.com/chrisboulton/php-resque and I never tried it but it can be a good way to start :)

05 January, 2015

Thanks for the comparison write up. It is good to see an up to date comparison of the frameworks. I have been doing Yii for some time and I have enjoyed it because of its close relations to pure PHP code constructs and its speed of processing and scaling it can do.

04 January, 2015

I worked with Yii2 since it will be started. I used all versions and i want to say it the best framework. DAO and AR patterns wery usefull. Views has a biutifull interface.
I glad to meet you with Yii2.

sorry for my bad English

06 January, 2015

Why didn't you choose Ruby On Rails?

10 December, 2015

Please share he AMQP code you used for Yii2? or i offer to collaborate to build the module

08 January, 2016

Post new comment

Private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options

Note for potential spammers: all links in your comment will not be indexed by search engines.

Anton Sidashin

Anton Sidashin senior developer, Pixeljets co-founder

I'm a web developer specializing in PHP and Javascript, and Drupal, of course. I'm building Drupal projects since 2005, and I was working as full-time senior engineer in CS-Cart for a while, building revolutionary e-commerce software. In my free time, I enjoy playing soccer, building my body in gym, and playing guitar.

Drupal.org ID: restyler
Drupal association member