What?
This is something I have noticed more often recently. On Long running hydroponics ingests where the data is being transformed via twig templates, even on a timeout == 0 (run until finished), the normal run time is less than an hour. And hydroponics shuts down.
Looking at how Drupal's renderInline works, very convenient for us, since it does not require a cache refresh on template changes, it makes every call to generate a Twig template in the Environment unnamed and uncached. Which leads to Twig generating Anonymous classes that persist and accumulate and can't be garbage collected.
Now that Twig in recent 3.x version has a hot Cache https://github.com/twigphp/Twig/pull/4338/changes we can use the chained array loader with a pre-established name (so the logic/classes are persistent/reusable across multiple renderings of the same template under different context) and only trigger a hot cache clear on Stored twig template change.
This is a big refactor (at the Metadata Display Entity level) but will bring huge performance/memory improvement also on Rendered Entity Search API indexing via drush and hydroponics.
$templateName = --> unique name based on the DB source UUID
$templateString = --> from our DB source.
$loader->setTemplate($templateName, $templateString);
$output = $twig->render($templateName, $context);
Currently
public function renderInline($template_string, array $context = []) {
// Prefix all inline templates with a special comment.
$template_string = '{# inline_template_start #}' . $template_string;
return Markup::create($this->createTemplate($template_string)->render($context));
}
Notice how createTemplate($template_string) is not passing a name at all ...
What I am unsure is if I should create better a parallel Twig Environment that uses the same global settings (Except the caching folder) and that is injected once and stays persistent instead of trying to tap into the Drupal one and use same folder/chain loader it has?
This is a mayor development/refactor.
@alliomeria 👀
What?
This is something I have noticed more often recently. On Long running hydroponics ingests where the data is being transformed via twig templates, even on a timeout == 0 (run until finished), the normal run time is less than an hour. And hydroponics shuts down.
Looking at how Drupal's renderInline works, very convenient for us, since it does not require a cache refresh on template changes, it makes every call to generate a Twig template in the Environment unnamed and uncached. Which leads to Twig generating Anonymous classes that persist and accumulate and can't be garbage collected.
Now that Twig in recent 3.x version has a hot Cache https://github.com/twigphp/Twig/pull/4338/changes we can use the chained array loader with a pre-established name (so the logic/classes are persistent/reusable across multiple renderings of the same template under different context) and only trigger a hot cache clear on Stored twig template change.
This is a big refactor (at the Metadata Display Entity level) but will bring huge performance/memory improvement also on Rendered Entity Search API indexing via drush and hydroponics.
$templateName = --> unique name based on the DB source UUID
$templateString = --> from our DB source.
$loader->setTemplate($templateName, $templateString);
$output = $twig->render($templateName, $context);
Currently
Notice how
createTemplate($template_string)is not passing a name at all ...What I am unsure is if I should create better a parallel Twig Environment that uses the same global settings (Except the caching folder) and that is injected once and stays persistent instead of trying to tap into the Drupal one and use same folder/chain loader it has?
This is a mayor development/refactor.
@alliomeria 👀