Bad Performance in Enonic XP


#1

Enonic version: 7.2
OS: WindowServer2019

We are having trouble with our XP-installation.

We can serve pages ok, but when we use jmeter and test with some connections, CPU on our applicationserver is going close to 100% and requests are handled slowly.

Even when we only request pages manually and hit F5 the CPU-usage is increasing fast.
We have added more CPU’s to the VM and that helps a bit, but there must hopefully be some tuning we can do?

We have an Apache-server in front of the applicationserver.

Please help…

Sondre.


#2

This sounds like normal behavior for any instance without caching or optimization. What is your average page rendering time?


#3

Hi,

We are having about 4 sec before “Time to Interactive”
Se attached file

We are running on Windowsplatform with Apache HTTP in front of the application-nodes.

What are other companies using as a proxy?

Sondre.


#4

Hmm… Time to interactive is measured in the browser and can heavily relate to client side activity. I recommend you install and run our Livetrace app to diagnose performance on the server (available on market). This will often also reveal any bottlenecks in your application code.


#5

We have run the Livetrace now and here is som screenshots.
One with only one request, and one with multiple requests.

How to make this site available to more than one user at the time?
We can’t go live with this unresponsive site.

Sondre.


#6

Hi Sondre,

You can click the triangles to the left in order to inspect the individual request further! A developer can then take a look at the most lengthy steps and see if it’s possible improve the performance of that step, either by optimizing the code, implementing cache using lib-cache, identify slow-loading (large) images, or if it’s slow resources: see if resources can be served differently (use minimized resources or use a Content Delivery Network)

Example from enonic.com: In the image below, you see that there is an “add-script” step on the front page which takes 535 milliseconds - half of the page load time. The rest of the components on the page seem healthy, there are hardly any parts that use more than a dozen milliseconds to render.

This should hopefully be a quick start for you to identify what causes your performance problems, at least when it comes to page rendering speed on the server side.

/Bjørnar


#7

Hi,

Thank you for input, espesialy about lib-cache. I don’t know if our developers has used that.
Yes we have seen this, but the main problem is when we got more than one request to the site.

What are other companies using as cachingsoftware?

Sondre.


#8

I know XP sites that run behind Varnish, and XP sites that run behind BigIP, and there are probably many other options out there as well. I think services like CloudFlare also offers cache? I don’t know enough about these products to give you any advice on what to use and when to use it, but perhaps it’s a start.


#9

Thank you.

Can you provide med with contactinformation to to a customer that run Varnish?

Sondre.


#10

Well, since these are companies that I was a front-end developer at a number of years ago, I don’t know how easy it would be to find the right contact person, in a world where many such services are outsourced to other consultant companies. But off the top of my head I know that SSB has been running Varnish for many years, and that Posten/Bring and Gjensidige have been running BigIP - I think the former uses/used Tata Consultancy Services and the latter uses/used EVRY as their hosting provider which is responsible for setting up and managing the BigIP cache. I don’t know who SSB uses to manage Varnish, it’s been five years since I was on a project there.

Usually, my XP sites run just fine without needing any extra front-end cache (>_<)
I’m running sites on Google Cloud and Enonic Cloud.


#11

I don’t think caching is your main problem. You need to optimize your code first. Generally, aim to render pages in less than 500ms!


#12

Thank you for the information. We will have a meeting with the developers today.


#13

Helllo,

We have under 500ms when there is only one user on the site.

The problem is when more than one user access the site.
Responsetime is increasing beyond acceptable values…

Sondre.


#14

How much compute capacity do you have? As in vCPU’s? You cannot compare numbers directly when running on a laptop Compared to a server. Also, are you using SSD’s?


#15

You say that you’re running the XP server in a Virtual Machine. Is it a dedicated VM, or are there other processes running? I’ve heard of people experiencing slowness because they were running inside a VM that shared resources with other processes, and where switching to a dedicated server solved the bottleneck.


#16

We are running on 8vCPU in VmWare on SSD.


#17

The VM is only running Enonic XP. No other programs installed.


#18

We had a session today with the developer, and we traced it down to thyeleaf in livetrace.
Trace i attached here:


What is going on in the time marked with the green squares?

Rest of the items in the trace is loading fast. Where is the time going in the thymesleaf?

Sondre.


#19

Blank spaces is simply when it is processing.


#20

With 8vCPU you should be able to run a minimum of between 4 and 8 parallell rendering processes. If each render takes 2 second, your server will only have a throughout of 2-4 pages/second. If it takes 500ms, 8-16 per second. So, you need to optimize your code.