Doctrine Helper Bundle is a Symfony 7.x bundle that provides a few functionalities that simplify the development of request handling logics that involve interacting with data going to and coming out of the database via the popular Doctrine ORM library.
Make sure Composer is installed globally, as explained in the installation chapter of the Composer documentation.
Open a command console, enter your project directory and execute:
composer require mlukman/doctrine-helper-bundle
Open a command console, enter your project directory and execute the following command to download the latest stable version of this bundle:
composer require mlukman/doctrine-helper-bundle:1.*
Then, enable the bundle by adding it to the list of registered bundles
in the config/bundles.php
file of your project:
// config/bundles.php
return [
// ...
MLukman\DoctrineHelperBundle\DoctrineHelperBundle::class => ['all' => true],
];
Most of the Doctrine operations require chaining a few function calls, like for example:
$count = $em->getRepository(Comment::class)
->createQueryBuilder('c')
->select('COUNT(1)')
->where('email = :email')
->setParameter('email', $email)
->getQuery()
->getSingleScalarResult();
The DataStore
service provided by this bundle has simplified this operation into:
$count = $ds->count(Comment::class, ['email' => $email]);
The following are operations that DataStore
has simplified:
# | Operation | Pure Doctrine | Using DataStore |
---|---|---|---|
1 | Query the record with a specific ID | $em ->getRepository(Comment::class) ->find(3); |
$ds->queryOne(Comment::class, 3); |
2 | Query latest record of Comment with a specific email | $em ->getRepository(Comment::class) ->findOneBy(['email' => $email], ['date'=>'DESC']); |
$ds->queryOne(Comment::class, ['email' => $email], ['date'=>'DESC']); |
3 | Query up to 10 latest records of Comment with a specific email | $em ->getRepository(Comment::class) ->findBy(['email' => $email], ['date'=>'DESC'], 10); |
$ds->queryMany(Comment::class, ['email' => $email], ['date'=>'DESC'], 10); |
4 | Query using OR conditions | $em ->getRepository(Comment::class) ->createQueryBuilder('c') ->where('e.email = :email') ->orWhere('e.name = :name') ->setParameter('email', $email) ->setParameter('name', $name) ->setMaxResults($limit) ->addOrderBy('e.date', 'DESC') ->getQuery() ->getResult() |
$ds->queryUsingOr(Comment::class, ['email' => $email, 'name' => $name], ['created' => 'DESC'], $limit); |
More operations:
-
Count records:
$ds->count( Comment::class, // entity class ['approved' => false] // (optional) filters );
-
Full-text search:
$ds->fulltextSearch( Comment::class, // entity class ['text'], // array of column names that have fulltext index $keyword, // the keyword to search ['approved' => true] // (optional) filters )->getQuery()->getResult();
-
Get/count previous/next records:
$ds->getPrevious( $current, // the current record used as pivot 'submitDate', // the column to sort the records by ['parentPost'] // matcher fields (filter only records that match these fields with pivot record) ); // all these methods have same parameter signatures but different return values $ds->countPrevious($current, 'submitDate', ['parentPost']); $ds->getNext($current, 'submitDate', ['parentPost']); $ds->countNext($current, 'submitDate', ['parentPost']);
Common request body handling flow is as below.
graph LR
client(Client) -- Raw body --> kernel(Symfony HTTP Kernel)
kernel -- Request object --> controller(Controller)
controller -- Entity object --> orm(Doctrine ORM)
orm -- SQL --> db(Database)
Inside the controller, there would be many lines of code that manually read parameters one by one from the Request object and transfer it into the corresponding Entity object. What if the controller can just call a single line of code to do so?
$specificRequest->populate($specificEntity);
This helper bundle has Service\RequestBodyConverter
which does that!
First, create a PHP class that extendsDTO\RequestBody
, for example:
use MLukman\DoctrineHelperBundle\DTO\RequestBody;
class CommentRequest extends RequestBody
{
public ?string $name;
public ?string $email;
public ?string $comment;
}
For simplicity, just let all its properties public
instead of using setters/getters.
Next, modify the corresponding Entity class to implement DTO\RequestBodyTargetInterface
. For now there is no method to implement and this interface is just for type-detection. However, the Entity class must either have public properties with the same names as the ones inside the DTO\RequestBody
subclass or corresponding setter methods or mixture of both. Otherwise the parameters from the request will be ignored. For example:
use Doctrine\ORM\Mapping as ORM;
use MLukman\DoctrineHelperBundle\DTO\RequestBodyTargetInterface;
#[ORM\Entity]
class Comment implements RequestBodyTargetInterface
{
#[ORM\Id]
#[ORM\GeneratedValue]
#[ORM\Column]
private ?int $id = null;
#[ORM\Column(length: 30)]
public ?string $name = null;
#[ORM\Column(length: 30)]
public ?string $email = null;
#[ORM\Column(type: Types::TEXT, nullable: true)]
private ?string $comment = null;
public function setComment(string $comment): self
{
$this->comment = $comment;
return $this;
}
}
Finally, modify the controller method by doing:
-
Add the
RequestBody
subclass to the list of the method parameters. Make it optional by prepending the class name with?
.#[Route('/comments/post', name: 'app_comments_post')] public function comments_post(?CommentRequest $commentRQI): Response
-
Add the code to use this
RequestBody
subclass (sorry, it's not really a single line of code if you count in the entity instantiation and the condition checking but the transfer of values from the request into the entity is actually that single line).$comment = new Comment(); if ($commentRQI) { $commentRQI->populate($comment); }
Now, any requests coming to that route with POST or PUT bodies, either in JSON format or HTTP form submission, will be converted into the specific method parameter and will be able to be utilized by the controller method.
Note: to troubleshoot any conversion issues, this bundle provides a web profiler panel with label "Request Conversions".
You might be already familiar with ValidatorInterface
used to validate objects whose classes have been annotated with Symfony validations such as @[NotBlank]
. This bundle provides a service that even simplifies the validation process named ObjectValidator
. Calling this service's validate()
method will output an array that has already consolidated all the error messages tagged under respective field names, for example:
#[Route('/comments/post', name: 'app_comments_post')]
public function comments_post(?CommentRequest $commentRQI, ObjectValidator $objectValidator): Response
{
$comment = new Comment();
if ($commentRQI) {
$commentRQI->populate($comment);
$validateErrors = $objectValidator->validate($comment);
if (empty($validateErrors)) {
// Here we can safely save the $comment entity object to database
// Then make sure to redirect to another page or this same page
}
// If we reach here it means $validateErrors has some results
// We can pass this array to Twig template form to show the errors to the user
// For now we just print_r
print_r($validateErrors);
}
}
The output might look like:
[
"quantity" => [
"This value should not be blank.",
"This value should be positive."
],
"reference" => [
"Value must be unique."
]
]
It is also possible to manually construct the validation errors array by calling addValidationError()
method:
$validateResult = [];
$objectValidator->addValidationError($validateResult, "quantity", "This value should not be blank.");
$objectValidator->addValidationError($validateResult, "quantity", "This value should be positive.");
$objectValidator->addValidationError($validateResult, "reference", "Value must be unique.");
That feature might be helpful if you want to chain validations of multiple objects, or if you have custom validations that need to be implemented outside of the objects.
To store images in the database, we can simply use the column type "image" in the ORM\Column
annotation. The corresponding PHP class is ImageWrapper
which basically wraps around the Imagine
library (https://imagine.readthedocs.io/). This bundle performs conversion between the database BLOB
columns and PHP ImageWrapper
objects.
use Doctrine\ORM\Mapping as ORM;
use MLukman\DoctrineHelperBundle\Type\ImageWrapper;
#[ORM\Entity]
class Profile
{
#[ORM\Column(type: "image", nullable: true)]
public ?ImageWrapper $photo = null;
}
The recommended usage of ImageWrapper
class is to use it in conjunction with DTO\RequestBody
to handle form submissions. Specifying a property whose name matches with the upload field's name
will automatically load the image from the form submissions.
use MLukman\DoctrineHelperBundle\DTO\RequestBody;
class ProfileRequest extends RequestBody
{
public ?ImageWrapper $photo = null;
}
If not using DTO\RequestBody
when handling upload form, in the code that handle the file upload should first fetch the Symfony\Component\HttpFoundation\File\UploadedFile
object that refers to that file upload submission and just use the provided static method fromUploadedFile
:
$profile->photo = ImageWrapper::fromUploadedFile($uploadedFile);
Or if the image file is already on the server's file system, you can manually load the image file into an ImageWrapper
object by calling its constructor:
// $image can be either a resource stream from fopen or simply the full path to the image file on the server
$profile->photo = new ImageWrapper($image);
Regardless of which method the ImageWrapper
object is instantiated with, saving the entity object that host the ImageWrapper
property as ORM\Column(type:"image")
to the database will store the image binary into the corresponding BLOB
database column.
Once the image is stored and later fetched from the database, here are a few ways how you can use the image:
-
As a download response from a controller method:
// passing $request is optional and only necessary for enabling client-side caching return $profile->photo->getDownloadResponse($request);
-
As a Base64 string returned to the client either in
<img src />
HTML tag or inside API response JSON:$profile->photo->getBase64DataUrl();
-
As a download link to be embeddable inside
<img src />
in combination with #1 above:// first, this method needs to be called inside controller $profile->photo->setDownloadLink($this->generateProfileImageLink($profile));
{# inside twig template #} <img src="{{ profile.photo.downloadlink }}" /> <img src="{{ profile.photo }}" /> {# this works too #}
-
As a raw binary string for further processing:
$imageBinary = $profile->photo->get();
However, for a simple resizing of the image, ImageWrapper
already provides the resize($maxWidth, $maxHeight, $keepAspectRatio)
method:
$profile->photo->resize(400,300,false)->getBase64DataUrl();
Unlike images where the only data that matters is the imagery information, storing files in the database usually also involves storing the file name and mime type as well in order for the stored file data to be useful later. These additional metadata are often stored in separate database columns, which add up the complexity of the application to combine and split those information upon reading and writing to the database. To simplify this issue, this bundle provides the column type "file" and its corresponding PHP class FileWrapper
.
Using this class in conjunction with DTO\RequestBody
will automatically process file upload field submissions and populate the file metadata, which can be retrieved using getName()
, getMimetype()
, getSize()
and getContent()
.
use MLukman\DoctrineHelperBundle\Type\FileWrapper;
#[ORM\Entity]
class Profile
{
#[ORM\Column(type: "file", nullable: true)]
private ?FileWrapper $resume = null;
}
Similar to ImageWrapper
above, FileWrapper
too provides fromUploadedFile()
, getDownloadResponse()
, setDownloadLink()
and getDownloadLink()
.
If you are not comfortable with the idea of storing files in the database, you can also use alternative column type 'fsfile'. The usage is similar to 'file':
use MLukman\DoctrineHelperBundle\Type\FileWrapper;
#[ORM\Entity]
class Profile
{
#[ORM\Column(type: "fsfile", nullable: true)]
private ?FileWrapper $resume = null;
}
With 'fsfile', only the metadata is stored in the database using JSON format, while the actual file content will be stored in the filesystem at path prefix {appdir}/var/fsfiles/
. On a HA setup, please ensure this path prefix is mounted on a shared storage medium.
Important note: do not convert columns of type 'file' to 'fsfile' and vice versa on entities with existing records as this bundle does not handle the migration of the file contents from the database into the file system and vice versa. The design decision whether to use 'file' or 'fsfile' must be made from the beginning.
This column type automatically encrypts and decrypts the provided string value before storing to the database and after loading from the database. By default, the encryption key used for the encryption and encryption will be derived from the APP_SECRET
environment variable, hence you need to sync this environment variable if you migrate data from one environment to another. However, you can also use a different key by calling the static method \MLukman\DoctrineHelperBundle\Type\EncryptedType::setEncryptionKey($keyString)
during kernel's boot up.
use MLukman\DoctrineHelperBundle\Type\EncryptedValue;
#[ORM\Entity]
class Profile
{
#[ORM\Column(type: "encrypted", nullable: true)]
private ?string $nationalId = null;
}
When our applications handle a long list of a particular entity data, it is often a challenge to present the list of data to the users. Simply listing all of them would not only impact the performance of the applications, it would also become bad experience to the users who need to scroll through all of them. A common technique is to paginate the list into reasonable chunk and provide the users with the UI elements to navigate through the pages. While this bundle does not provide the UI elements in order to be UI framework agnostic, it provides the functionalities to process the pagination parameters from the query parameters, use them to add filtration to database queries and prepare the information that will be required by the Twig template to show the correct pagination control UI elements.
To start with the pagination, the first step is to add a DTO\Paginator
object to the controller's method, for example:
#[Route('/admin/users', name: 'app_admin_users')]
public function users(Paginator $paginator): Response
Next is to prepare a \Doctrine\ORM\QueryBuilder
pre-configured with the base query and any default filters and sorting, for example:
#[Route('/admin/users', name: 'app_admin_users')]
public function users(Paginator $paginator, DataStore $ds): Response
{
$qb = $ds->queryBuilder(User::class, 'u')
->andWhere("u.status = 'Active'")
->addOrderBy('u.registered', 'DESC');
}
Now let's the paginator do the magic by simply calling paginateResults()
method, passing the query builder in the parameter:
#[Route('/admin/users', name: 'app_admin_users')]
public function users(Paginator $paginator, DataStore $ds): Response
{
$qb = $ds->queryBuilder(User::class, 'u')
->andWhere("u.status = 'Active'")
->addOrderBy('u.registered', 'DESC');
return $this->render('admin/users/index.html.twig', [
'users' => $paginator->paginateResults($qb),
'paginator' => $paginator,
]);
}
Finally, we need to prepare the UI elements that will show the pagination control. The $paginator
object that we provide to the Twig template in the example above provides a few method that will provide the necessary info:
Method | Description | |
---|---|---|
getMaxPage() |
The number of pages to navigate between. It is also a good idea to hide the pagination control if this value is 1. | |
getPage() |
The current page number starting from 1 up until getMaxPage() value. This value is processed from the query parameter page , which the pagination control needs to modify upon navigation. |
|
getLimit() |
The number of records in one page. This value is processed from the query parameter limit , which the pagination control needs to modify upon navigation. |
|
getFrom() |
The index of the first record in the current page. For example, on the 2nd page with limit 10, the value would be 11. | |
getTo() |
The index of the last record in the current page. For example, on the 2nd page with limit 10, the value would be 20 (assuming there are 20 or more records in total). |
Another common method of record list filtration is by allowing users to search a specific keyword. This bundle simplifies adding that functionality by providing DTO\SearchQuery
that is to be added to the controller's method, for example:
#[Route('/admin/comments', name: 'app_admin_comments')]
public function comments(SearchQuery $search): Response
Next is to prepare a \Doctrine\ORM\QueryBuilder
pre-configured with the base query and any default filters and sorting, for example:
#[Route('/admin/comments', name: 'app_admin_comments')]
public function comments(SearchQuery $search, DataStore $ds): Response
{
$qb = $ds->queryBuilder(Comment::class, 'c')
->andWhere("c.deleted = 0")
->addOrderBy('c.posted', 'DESC');
}
Next, let the DTO\SearchQuery
do the magic by calling either applyLikeSearch()
or applyFulltextSearch()
passing the query builder as the first parameter and a list of columns with aliases as the second parameter:
#[Route('/admin/comments', name: 'app_admin_comments')]
public function comments(SearchQuery $search, DataStore $ds): Response
{
$qb = $ds->queryBuilder(Comment::class, 'c')
->andWhere("c.deleted = 0")
->addOrderBy('c.posted', 'DESC');
$search->applyFulltextSearch($qb, ['c.message']);
}
Then, pass the query builder result and the SearchQuery
object to Twig template:
#[Route('/admin/comments', name: 'app_admin_comments')]
public function comments(SearchQuery $search, DataStore $ds): Response
{
$qb = $ds->queryBuilder(Comment::class, 'c')
->andWhere("c.deleted = 0")
->addOrderBy('c.posted', 'DESC');
$search->applyFulltextSearch($qb, ['c.message']);
return $this->render('admin/comments/index.html.twig', [
'comments' => $qb->getQuery()->getResult(),
'searchquery' => $search,
]);
}
Finally, prepare a search field in the UI:
<input type="search" name="{{ searchquery.name }}" value="{{ searchquery.keyword }}" />
Another method of record list filtration is by providing users a set of predefined filters such as filter by statuses, categories etc. This bundle provides such functionality via PDO\PreDefinedQueries
. Again, like Paginator
and SearchQuery
, the UI elements for displaying the filters are not provided but it should be simple to do.
To start, add a DTO\PreDefinedQueries
object to the controller's method, for example:
#[Route('/admin/assets', name: 'app_admin_assets')]
public function assets(PreDefinedQueries $status): Response
Next is to prepare a \Doctrine\ORM\QueryBuilder
pre-configured with the base query and any default filters and sorting, for example:
#[Route('/admin/assets', name: 'app_admin_assets')]
public function assets(PreDefinedQueries $status, DataStore $ds): Response
{
$qb = $em->queryBuilder(Asset::class, 'a')
->addOrderBy('a.registered', 'DESC');
}
Next, we need to define for each of the filter option the name and how it modifies the query builder. To do so, call the addQuery()
method, passing the name of the filter option and a callback that accepts the query builder to perform modifications. Call that method for each filter option (chainable) and finally call the apply()
method passing the prepared query builder for example:
#[Route('/admin/assets', name: 'app_admin_assets')]
public function assets(PreDefinedQueries $status, DataStore $ds): Response
{
$qb = $em->queryBuilder(Asset::class, 'a')
->addOrderBy('a.registered', 'DESC');
$status
->addQuery('all', fn(QueryBuilder $qb) => $qb)
->addQuery('active', fn(QueryBuilder $qb) => $qb->andWhere("a.status = 'Active'"))
->addQuery('inactive', fn(QueryBuilder $qb) => $qb->andWhere("a.status IN ('Disposed', 'Consumed')"))
->apply($qb);
}
Then, pass the query builder result and the SearchQuery
object to Twig template:
#[Route('/admin/assets', name: 'app_admin_assets')]
public function assets(PreDefinedQueries $status, DataStore $ds): Response
{
$qb = $em->queryBuilder(Asset::class, 'a')
->addOrderBy('a.registered', 'DESC');
$status
->addQuery('all', fn(QueryBuilder $qb) => $qb)
->addQuery('active', fn(QueryBuilder $qb) => $qb->andWhere("a.status = 'Active'"))
->addQuery('inactive', fn(QueryBuilder $qb) => $qb->andWhere("a.status IN ('Disposed', 'Consumed')"))
->apply($qb);
return $this->render('admin/assets/index.html.twig', [
'assets' => $qb->getQuery()->getResult(),
'filter' => $status,
]);
}
Finally, prepare the UI elements that provides a list of filter options which navigates to specific URLs upon interaction. For example, using a simple <a>
, <ul>
and <li>
:
<ul>
{% for id,url in filter.urls %}
<li>
<a href="{{ url }}">
{% if id == filter.selectedId %}
<strong>{{ id }}</strong>
{% else %}
{{ id }}
{% endif %}
</a>
</li>
{% endfor %}
</ul>
By making entity classes implement \MLukman\DoctrineHelperBundle\Interface\AuditedEntityInterface
and use \MLukman\DoctrineHelperBundle\Trait\AuditedEntityTrait
, all creations and updates of the entity objects will be automatically stored along with other fields.
For creation date and update date fields, those interface & trait combination already covers everything needed from the entity columns and their getters & setters to the event subscribers needed to hook to Doctrine's PrePersist
and PreUpdate
events.
For creator and updater columns, however, those interface & trait combination only provides the event subscribers but the entity columns need to be defined by the implementing entity classes. This design was decided because there is no way for this bundle to know what would be the entity class that will be used in the implementing applications. For that reason, the implementation applications need to implement the remaining two methods of the AuditedEntityInterface
as well as the entity column definitions for $createdBy
and $updatedBy
:
interface AuditedEntityInterface
{
public function setCreatedBy(?\Symfony\Component\Security\Core\User\UserInterface $createdBy);
public function setUpdatedBy(?\Symfony\Component\Security\Core\User\UserInterface $updatedBy);
}
For example, the following MappedSuperclass
is taken from an application where the class that implements UserInterface
is called Login
with a ManyToOne
relation $user
to the User
class:
#[ORM\MappedSuperclass]
abstract class AuditedEntity implements AuditedEntityInterface
{
use AuditedEntityTrait;
#[ORM\ManyToOne]
#[ORM\JoinColumn(onDelete: 'SET NULL')]
protected ?User $createdBy = null;
#[ORM\ManyToOne]
#[ORM\JoinColumn(onDelete: 'SET NULL')]
protected ?User $updatedBy = null;
public function getCreatedBy(): ?User
{
return $this->createdBy;
}
public function getUpdatedBy(): ?User
{
return $this->updatedBy;
}
public function setCreatedBy(?UserInterface $createdBy)
{
if ($createdBy instanceof Login) {
$this->createdBy = $createdBy->getUser();
}
}
public function setUpdatedBy(?UserInterface $updatedBy)
{
if ($updatedBy instanceof Login) {
$this->updatedBy = $updatedBy->getUser();
}
}
}
This bundle provides a few traits that can be used to automatically add ID columns to entities which are generated based on specific patterns.
This trait provides shortcut to make an entity use UUID as the primary key column. It is as simple as adding to the class and that is all you need:
#[ORM\Entity]
class Component
{
use \MLukman\DoctrineHelperBundle\Trait\UuidEntityTrait;
}
This trait provides shortcut to make an entity use timestamps in the format of YYYYMMDDhhmmssµs
(from year until microseconds) as the primary key column. It is as simple as adding to the class and that is all you need:
#[ORM\Entity]
class Component
{
use \MLukman\DoctrineHelperBundle\Trait\TimestampIdEntityTrait;
}
It is possible to add prefix and suffix to the generated timestamps by overriding getIdPrefix()
and getIdSuffix()
methods in each entity class.
This trait provides shortcut to make an entity use a random hexadecimal value of length 8 characters as the primary key column. It is as simple as adding to the class and that is all you need:
#[ORM\Entity]
class Component
{
use \MLukman\DoctrineHelperBundle\Trait\Hex8IdEntityTrait;
}
This trait provides shortcut to make an entity use a random hexadecimal value of length 16 characters as the primary key column. It is as simple as adding to the class and that is all you need:
#[ORM\Entity]
class Component
{
use \MLukman\DoctrineHelperBundle\Trait\Hex16IdEntityTrait;
}
This bundle provides an attribute #[\MLukman\DoctrineHelperBundle\Attribute\Timezonify]
that can be used to annotate entity properties of type DateTime
to activate the following behaviors:
- Convert all datetimes used with such properties into UTC timezone before saving to the database
- Convert the UTC datetimes saved in the database to the application time zone which can be either:
- Set to each user session/request by calling the method
\MLukman\DoctrineHelperBundle\Service\Timezonify::setTimezone(string|DateTimeZone $tz)
method - Default to the time zone specified in the environment variable
APP_TIMEZONE
- Fallback to the default time zone set in the PHP environment which is returned by the native function
date_default_timezone_get()
- Set to each user session/request by calling the method
Sample usage:
#[ORM\Entity]
class Component
{
#[\MLukman\DoctrineHelperBundle\Attribute\Timezonify]
protected \DateTime $createdDate;
}
Note: adding this attribute to entities with existing database records will have unintended effect of the stored datetimes will be skewed. To solve this issue, you would need to manually patch those datetimes to UTC time zone first before adding this attribute,
This bundle will be updated and improved to add more features. Feel free to star and/or watch this repo!
I am open for any ideas to enhance this bundle further. Feel free to raise your feature requests as issues on this GitHub repo.