Tips Posts

Retrieve count of nested relationship data in Laravel

Recently in one of our client's projects, we want to load the count of relation in laravel. But we do not want to retrieve original records.

For example,

We have the following Models,

  1. Category
  2. Products
  3. Orders

For that, we have categories, products, orders, order_items table. Where in the order_items table, we got the following fields

  • order_id
  • product_id
  • quantity

So the requirement was, In the Products table, we want to display the total number of orders placed with that item regardless of the quantity in each order. All we need is a number of orders where the product is purchased.

1st way: Query via Relationship

$products = Product::all(); 
$productsArr = $products->map(function (Product $product) 
{     
     $productObj = $product->toArray();     
     $productObj['orders_count'] = $product->orders()->count();     
     return $productObj; 
   }
);

But the problem with this approach was, we are firing queries to the database for every single product. so if I'm retrieving 100 Products from the database then it will fire 100 additional queries to the database. Imagine if we have thousands of products.

2nd way: Eager Load Relationship and Calculate Count

$products = Product::with('orders')->get(); 
$productsArr = $products->map(function (Product $product) 
{
     $productObj = $product->toArray();
     $productObj['orders_count'] = $product->orders->count();
     return $productObj; 
   }
);

so this way, we are only firing two queries to the database. But the problem here is, we are loading all the Orders of each product which we don't need at all. so it will consume lots of memory since we are loading lots of orders. so imaging if we retrieve 100 products, and each product has 10 orders, then we are loading 1000 Orders into memory without any need.

3rd way: Use withCount function

The third powerful approach of using withCount function in Laravel. so we refactored our code like,

$products = Product::withCount('orders')->get();
$productsArr = $products->map(function (Product $product) 
{     
    $productObj = $product->toArray();
    $productObj['orders_count'] = $product ->getAttribute('orders_count');     
    return $productObj; 
    }
);

In this approach, we are firing two queries but no Order models are loaded into memory.

4th Bonus: Using in a nested relationship while multiple eager loading

You can even use it with nested relationships. Imagine a case, where you want to retrieve categories along with its products with orders count.

    $categories = Category::with(['products' => function ($query)
    {
        $query->withCount('orders');
    },
        'someOtherEagerLoading1',
        'someOtherEagerLoading2'
    ])->get();
    $categoriesArr = $categories->map(function (Category $category)
    {
        $categoryObj = $category->toArray();
        $categoryObj['products'] = $category->products->map(function (Product $product)
        {
            $productObj = $product->toArray();
            $productObj['orders_count'] = $product ->getAttribute('orders_count');
            return $productObj;
        });
        return $categoryObj;
    });

Hope this will help you to retrieve the count of relationship data without retrieving actual relation data.

September 18, 20202 minutesMitul GolakiyaMitul Golakiya
Using Common StatusTrait in Laravel in multiple models

Recently, we were working on a laravel app where we have a status column in multiple models. We have multiple processes going on for which we have different jobs.

Initially job status will be "Pending", then each job will take one record, change the status to "Running" and process that record, and update the status to either "Completed" or "Failed".

We have constants for each status. Something like below,

static $STATUS_PENDING = 0; 
static $STATUS_RUNNING = 1; 
static $STATUS_COMPLETED = 2; 
static $STATUS_FAILED = 3;

And the problem is, we need to go and define the status in every model that we have (around 10+ models).

Then we have functions to update status and check the status in each model like,

public function markRunning($saveRecord = true) 
{     
    $this->status = static::$STATUS_RUNNING;
    if ($saveRecord) {
        $this->save();
    }
}

public function isRunning()
{
    return $this->status === static::$STATUS_RUNNING;
}

And above functions existed for each 4 status. So what we did is, we created a common StatusTrait which can be used across multiple models.

Here is the code of StatusTrait that you can find for that.

Then in each model, we use this trait. For e.g.,

class SavePdf extends Model 
{     
    use StatusTrait;
    .....
}

And then can use any method of trait in all the models,

... 
$savePdf = new SavePdf(); 
$savePdf->markRunning(); 
...

Or we can check if the status of the model is running or not. For e.g.,

... 
if ($savePdf->isRunning()) 
{     
    // logic here 
} 
...

This is how we have saved tons of writing code and optimized the code. Another advantage is, we can just update the value of any status from one single place.

You can also check this kind of pattern and do something like this.

August 26, 20201 minuteMitul GolakiyaMitul Golakiya
Use Laravel Debugbar's measure function to optimize application performance

Laravel Debugbar is a great package to debug Laravel applications while developing. But it's not just limited to debugging. You can use it to optimize the performance of your app a lot as well. Like,

  • number of models loaded in a memory
  • number of queries fired with timing
  • memory used and more.

In short, we can have a complete view of what's going on in each request.

But the less known and used feature it gives is the Timeline tab. Where you can see how much time is taken for each request. And more than that, how much time Laravel took to boot up and how much time our other code has taken. Check the below screenshot.

Timeline

Recently we came to the case, where one of our consultation clients' CRM application was taking too much time on the first load. Their developers were not able to spot a problem. They checked queries and other stuff which looked completely good. so we were not sure where the time had been spent by the application.

That's where debugbar came to rescue us. We used its measure function facility by which we can measure the time spent in each of the functions wherever we want to use. It gives simply two functions startMeasure and stopMeasure to measure the time spent between these two statements.

so we can put startMeasure in the starting of function and put stopMeasure at the end of the function which will render something like this in the timeline tab.

public function searchClients($department) 
{     
    \Debugbar::startMeasure("searchClients'');
    // logic here

    \Debugbar::stopMeasure("searchClients");

    return $result;
}

Once we put this, we get a time that < code > searchClients is taking. Check the screenshot below,

Timeline

Hope this can help you to figure out what piece of code is taking the time and you can optimize it.

Happy Optimizing :)

August 22, 20201 minuteMitul GolakiyaMitul Golakiya
Efficient and Fast Data Import with Laravel Jobs & Queues

Recently Spatie released a brand new package for multi-tenancy called laravel-multitenancy.

It comes with great support to work out of the box with sub-domains like,

https://zluck.infychat.com

https://infyom.infychat.com

https://vasundhara.infychat.com

It identifies the tenant based on the sub-domain and sets a database runtime for your tenant-specific models.

Recently, we used it in one of our clients for Snow Removal CRM. Here we have two models,

  1. Freemium Model - with no sub-domain (application will work on main domain only)
  2. Premium Model - where the tenant will get its subdomain

And a user can convert his account from Freemium to Premium at any point in time by just subscribing to the plan.

So what we want is, on the backend, we want to have a separate database for each tenant, but the application should run on main as well as a sub-domain.

So what we want to have is the ability to extend/customize the tenant detection mechanism. And Spatie does a very good job there where you can customize the logic.

You can create your own TenantFinder class and configure it in the config file of the package. And there is very good documentation for that here:https://docs.spatie.be/laravel-multitenancy/v1/installation/determining-current-tenant/

To do that, what we did is, we have a field called tenant_id in our users table. All of our users are stored in the main database since we may have user access across the tenant.

And when any user does a login, we listen for the event Illuminate\Auth\Events\Login which is documented in Laravel Docs over here.

When a user does a login, our listener will be called and will create a cookie called tenant on the browser with the tenant id of the user. So our listener will look like,

<?php
namespace App\Listeners;

use App\Models\User;
use Cookie;
use Illuminate\Auth\Events\Login;

class LoginListener
{
    public function handle(Login $event)
    {
        /* @var User $authUser /
        $authUser = $event->user;

        Cookie::forget('tenant');
        Cookie::queue(Cookie::forever('tenant', encrypt($authUser->tenant_id)));
    }
}

Also, we encrypt the cookie on our end, so we do not want Laravel to encrypt it again, so we added the tenant cookie into except array of EncryptCookies middleware as per documentation here. so our middleware looks like,

<?php
namespace App\Http\Middleware;

use Illuminate\Cookie\Middleware\EncryptCookies as Middleware;

class EncryptCookies extends Middleware
{
    /**
      The names of the cookies should not be encrypted.

      @var array
     /
    protected $except = [undefined];
}

Now at the last point, we extended our logic to find a tenant and get it to work on the main domain as well as sub-domain.

We have created our own custom class called InfyChatTenantFinder, which looks like,

<?php
namespace App\TenantFinder;

use App\Models\Account;
use App\Models\SubDomain;
use Illuminate\Http\Request;
use Spatie\Multitenancy\Models\Concerns\UsesTenantModel;
use Spatie\Multitenancy\Models\Tenant;
use Spatie\Multitenancy\TenantFinder\TenantFinder;

class InfyChatTenantFinder extends TenantFinder
{
    use UsesTenantModel;

    public function findForRequest(Request $request): ?Tenant
    {
        $host = $request->getHost();

        list($subDomain) = explode('.', $host, 2);

        // Get Tenant by subdomain if it's on subdomain
        if (!in_array($subDomain, ["www", "admin", "infychat"])) {
            return $this->getTenantModel()::whereDomain($host)->first();
        }

        // Get Tenant from user's account id if it's main domain
        if (in_array($subDomain, ["www", "infychat"])) {

            if ($request->hasCookie('tenant')) {
                $accountId = $request->cookie('tenant');
                $accountId = decrypt($accountId);
                $account = $this->getTenantModel()::find($accountId);

                if (!empty($account)) {
                    return $account;
                }

                \Cookie::forget('tenant');
            }
        }
        return null;
    }
}

So basically, first we check if the sub-domain is there, then find a tenant from the sub-domain.

If the domain is the main domain then get the tenant id from the cookie and return the account (tenant) model.

So this is how you can customize the logic the way you want to have a custom tenant identification system.

August 14, 20202 minutesMitul GolakiyaMitul Golakiya
How to remove public path from URL in Laravel Application

While hosting on the Laravel project on cPanel, the traditional problem that lots of developers get is, /public path is appended to the URL. Because in most cases, we put a project directly into the public_html folder, so public_html is our root for the website and that's where our laravel application is also placed.

But to run the Laravel application, we need to point our domain root to the public folder of the laravel. It is possible to do it with cPanel but you need to go through some steps which are not known by most of the people and also the tedious process. So to make it simple, what you can do is, there is a way we can do it via the .htaccess file in our root folder.

We can copy the .htaccess file from our public folder and then make modifications to it to work with the direct root folder and route every request to the public folder.

Here is the final .htaccess file,

<IfModule mod_rewrite.c>     <IfModule mod_negotiation.c>         Options -MultiViews -Indexes     </IfModule>
    RewriteEngine On

    # Handle Authorization MemberHeader
    RewriteCond %{HTTP:Authorization} .
    RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]

    # Redirect Trailing Slashes If Not A Folder...
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_URI} (.+)/$
    RewriteRule ^ %1 [L,R=301]

    # Remove public URL from the path
    RewriteCond %{REQUEST_URI} !^/public/
    RewriteRule ^(.*)$ /public/$1 [L,QSA]

    # Handle Front Controller...
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^ index.php [L]

By adding the above file to the root folder we can use laravel projects without the public path. Check out the following two lines:

  RewriteCond %{REQUEST_URI} !^/public/   RewriteRule ^(.*)$ /public/$1 [L,QSA]

These two lines make magic and our application will work without public path in URL.

August 01, 20201 minuteMitul GolakiyaMitul Golakiya
Custom path generation in Spatie Media Library while multi-tenant

Recently we use Spatie laravel-multitenancy package in our of our client's CRM project along with spatie/laravel-medialibrary.

The default behavior of the media library package is, it generates a folder for each media with its ID in our configured disk, like a public folder or s3 or whatever disk we configured.

This works really great when you are dealing with a single-tenant application. But while using multi-tenant, you got a media table in each of your tenant databases. so this pattern simply doesn't work. Because then you will end up having multiple media files under the same folder from different tenants.

So what we want to do is, instead of having a structure like,

public -- media 
---- 1 
------ file.jpg 
---- 2 
------ file.jpg 
...

What we want to achieve is, we want to have a folder structure where media of every tenant will be in a separate folder with tenant unique id,

public -- media 
---- abcd1234 
// tenant1 Id 
------ 1 
-------- file.jpg 
------ 2 
-------- file.jpg 
---- efgh5678 
// tenant2 Id 
------ 1 
-------- file.jpg 
------ 2 
-------- file.jpg 
...

Spatie Media library is very configurable out of the box where you can write your own media library path generator. That is documented very well over here

So what we did is, we wrote our own Path Generator, which can store files into tenants folder. Here is how it looks like,

<?php
namespace App\MediaLibrary;

use Spatie\MediaLibrary\MediaCollections\Models\Media;
use Spatie\MediaLibrary\Support\PathGenerator\DefaultPathGenerator;

class InfyCRMMediaPathGenerator extends DefaultPathGenerator
{
    /*
      Get a unique base path for the given media.
     /
    protected function getBasePath(Media $media): string
    {
        $currentTenant = app('currentTenant');
        return $currentTenant->unique_id.DIRECTORY_SEPARATOR.$media->getKey();
    }
}

What we did here is, we just simply extended the Spatie\MediaLibrary\Support\PathGenerator\DefaultPathGenerator of Media Library and override the function getBasePath. We attached the prefix of the tenant's unique_id to the path. so instead of 1/file.png, it will return abcd1234/1/file.png.

All you need to make sure is, whenever you are uploading a media, your application should be tenant aware. Otherwise, it will not able to get the current tenant.

Hope this helps while using spatie media library along with spatie multi-tenant.

Even if you are not using a spatie multi-tenant, still you can create your own PathGenerator and use it your own way to have a media structure you want.

July 30, 20202 minutesMitul GolakiyaMitul Golakiya
Spatie Laravel Multi-Tenancy without Sub Domain (customize TenantFinder)

Recently Spatie released a brand new package for multi-tenancy called laravel-multitenancy.

It comes with great support to work out of the box with sub-domains like,

https://zluck.infychat.com

https://infyom.infychat.com

https://vasundhara.infychat.com

It identifies the tenant based on the sub-domain and sets a database runtime for your tenant-specific models.

Recently, we used it in one of our clients for Snow Removal CRM. Here we have two models,

  1. Freemium Model - with no sub-domain (application will work on main domain only)
  2. Premium Model - where the tenant will get its subdomain

And a user can convert his account from Freemium to Premium at any point in time by just subscribing to the plan.

So what we want is, on the backend, we want to have a separate database for each tenant, but the application should run on main as well as a sub-domain.

So what we want to have is the ability to extend/customize the tenant detection mechanism. And Spatie does a very good job there where you can customize the logic.

You can create your own TenantFinder class and configure it in the config file of the package. And there is very good documentation for that here:https://docs.spatie.be/laravel-multitenancy/v1/installation/determining-current-tenant/

To do that, what we did is, we have a field called tenant_id in our users table. All of our users are stored in the main database since we may have user access across the tenant.

And when any user does a login, we listen for the event Illuminate\Auth\Events\Login which is documented in Laravel Docs over here.

When a user does a login, our listener will be called and will create a cookie called tenant on the browser with the tenant id of the user. So our listener will look like,

<?php
namespace App\Listeners;

use App\Models\User;
use Cookie;
use Illuminate\Auth\Events\Login;

class LoginListener
{
    public function handle(Login $event)
    {
        /* @var User $authUser /
        $authUser = $event->user;

        Cookie::forget('tenant');
        Cookie::queue(Cookie::forever('tenant', encrypt($authUser->tenant_id)));
    }
}

Also, we encrypt the cookie on our end, so we do not want Laravel to encrypt it again, so we added the tenant cookie into except array of EncryptCookies middleware as per documentation here. so our middleware looks like,

<?php
namespace App\Http\Middleware;

use Illuminate\Cookie\Middleware\EncryptCookies as Middleware;

class EncryptCookies extends Middleware
{
    /**
      The names of the cookies should not be encrypted.

      @var array
     /
    protected $except = [undefined];
}

Now at the last point, we extended our logic to find a tenant and get it to work on the main domain as well as sub-domain.

We have created our own custom class called InfyChatTenantFinder, which looks like,

<?php
namespace App\TenantFinder;

use App\Models\Account;
use App\Models\SubDomain;
use Illuminate\Http\Request;
use Spatie\Multitenancy\Models\Concerns\UsesTenantModel;
use Spatie\Multitenancy\Models\Tenant;
use Spatie\Multitenancy\TenantFinder\TenantFinder;

class InfyChatTenantFinder extends TenantFinder
{
    use UsesTenantModel;

    public function findForRequest(Request $request): ?Tenant
    {
        $host = $request->getHost();
        list($subDomain) = explode('.', $host, 2);

        // Get Tenant by subdomain if it's on subdomain
        if (!in_array($subDomain, ["www", "admin", "infychat"])) {
            return $this->getTenantModel()::whereDomain($host)->first();
        }

        // Get Tenant from user's account id if it's main domain
        if (in_array($subDomain, ["www", "infychat"])) {

            if ($request->hasCookie('tenant')) {
                $accountId = $request->cookie('tenant');
                $accountId = decrypt($accountId);
                $account = $this->getTenantModel()::find($accountId);

                if (!empty($account)) {
                    return $account;
                }
                \Cookie::forget('tenant');
            }
        }
        return null;
    }
}

So basically, first we check if the sub-domain is there, then find a tenant from the sub-domain.

If the domain is the main domain then get the tenant id from the cookie and return the account (tenant) model.

So this is how you can customize the logic the way you want to have a custom tenant identification system.

July 24, 20203 minutesMitul GolakiyaMitul Golakiya
toBase function in Laravel Eloquent

Sometimes we need to load a large amount of data into memory. Like all the models we have in the database.

For e.g. PDF Printing, Perform some global updates, etc.

So the general practices we use in Laravel is to write the following code,

$users = User::all();

Just imagine I have 10,000 users in the database and when I load all the users in one shot.

But it takes a really high amount of memory to load all the records and prepare Laravel Model class objects. And sometimes we also load them in chunks to save the memory, but in some use cases, chunking can not be the option.

Here is the screenshot of mine when I load 10,000 users into memory with the above code.


10k Models


It's using 37MB memory. Also, imagine the required memory if we are loading some relationships as well with these 10,000 records.

The Eloquent model is a great way to handle operations with lots of features like Mutators, Relationships, and much more.

But we really do not use these features all the time. We simply output or use the direct values which are stored in the table. So ideally, we do not need an eloquent model at all, if we are not going to use these features.

In those cases, Laravel also has a handy function toBase(). By calling this function it will retrieve the data from the database but it will not prepare the Eloquent models, but will just give us raw data and help us to save a ton of memory.

So my revised code will look something like this,

$users = User::toBase()->get();

Check the revised memory screenshot after adding the toBase function.


10k Models toBase


So it almost saves 50% of the memory. It's reduced from 35MB to 20MB and the application also works much much faster, because it doesn't need to spend time in preparing 10,000 Eloquent models.

So if you are not really going to use features of Eloquent and loading a large amount of data, then the toBase function can be really useful.

Here you can find a full video tutorial for the same.

June 21, 20202 minutesMitul GolakiyaMitul Golakiya