Handle POST-requests in parts on the page-url

[tom]

Enonic version: 7.1.0
OS: Ubuntu

I would like it to be possible to to have a part in a page. And on a POST-request to that page, handle the POST in a post-function exported from the part, and render something to the page (where the part is included, same as with GET).

Currently nothing is rendered where the part was when we do a POST-request to a page. (See example below)

---

The background for my experimentation around this, is that I wanted to get more back to basics. I want to ship less JavaScript to improve initial load time of my pages.

I basically want to get the same benefits of having parts and pages, with using a form.

I want to do validation serverside in my part, and render out the form again, with the inputfields’ errors indicated.

---

Example:

I have uploaded some test code on Github. For this example the part controller is found here, and the page controller can be found here.

GET Request renders:

POST Request when pressing the “Send name”-button renders:

We can see that no part is rendered on a POST to a page, and the post function in the part is not called.

---

We had a discussion around what is possible and not regarding this topic, as well as some workarounds in this thread on the #developer channel on Slack.

---

Alternative ways to solve this in XP today feels a bit like hacks:

1. Just do everything in a page - The problem is that I want to reuse the part in several pages, and I don’t want to duplicate code.
2. Send the POST to a service, and render the resulting html. - This has the same problems as with page above.
3. Send the POST to the url of the part - Either I have to render the entire page-html in the part, or I would have to use JavaScript/Ajax to place the parts html inside the correct element on the page.

[tsi]

So, I distill this as follows:
You have a form in a part, and you want to be able to “reload” the part after applying a post directly to the part controller?

Short version is this is fully supported. You need to do the following:

1. Create a post to the part controller using “componentUrl()”.
2. In the controller, add an exports.post() that will be executed. Wrap it so it will perform similar rendering as handled by your “get” request
3. Add client-side javaScript in the part that is initially rendered that will handle the replacement of the part DOM based on the post response.

Think that should be it…

[tsi]

NOTE: Remember that you can also check if you are running in standalone or inline mode. Though, using POST you can be sure things are not inline

[Alan]

You haven’t described what exactly you do in the part, but from what I see it doesn’t look like a good system design.

It’s absolutely not a good idea to propagate POST requests to ALL components on the page, simply because you never know which of them need it. If you want a part to perform some business logic which will be triggered on POST, then you should post directly to this part.

If you are posting to the page then you obviously have to handle the POST inside the page to let the page control what to do with the data it received - hence the term “page controller”.

Not sure why you mention duplication of the code. If your page controller needs to execute some business logic stored in the part controller, then you simply export function inside the part controller and then call it from the page controller. Or even have a separate helper class where you store business logic which can be called from anywhere.

/parts/part1/part1.js:

exports.hello = function(req) {
...
}

/parts/part2/part2.js:

exports.hello = function(req) {
...
}

/pages/default/default.js:

var part1 = require('../../parts/part1');
var part2 = require('../../parts/part2');

export.post = function(req) {
      part1.hello(req):
      part2.hello(req):
}

[tom]

@tsi The 3rd “hack” I listed was in reference to using componentUrl(), to post directly to the part. I rejected that solution, because I don’t want to write client side JavaScript for things I should be able to solve server side.

@Alan Regarding posting to all parts on a page being bad system design, there is of course the obvious trap where you intend to post to one part, but all parts that export post() or all() tries to resolve it.

But that is a problem that has been there since html got the <form> tag. And I think the basic use case of having a single form (which usually is the case in my experience), should be easier to implement then it is in Enonic today. You have to jump trough hoops in all the solutions outlined on the Slack-thread and above by @tsi.

But the different solutions does give the developers enough tools to work around the “multiple form problem”, if that is a use case they do have.

---

What I am concerned with here, is going back to basics. To the type of web pages we made 20 years ago.

I have a simple page with a menu and a form. I want the html boilerplate and menus to be in the Thymeleaf-template belonging to the page (so it can be reused for many pages), and I want the form to live in a part, so it can encapsulate the form logic and markup (in a seperate Thymeleaf-template).

To me the “bad system design” is not easily supporting what is the basic use case (single form on the webpage), just because the developer may shoot themselves in the foot.

And if the developer shoots themselves in the foot, then this is very basic html-knowledge, and it’s a learning opportunity.

---

@Alan I was writing about code duplication, but I was mostly referring to the Thymeleaf-templates. I didn’t want to duplicate the html boilerplate and menus every time I need to create a form.

In my exact case the same form was also used in two places, with two different pages wrapping it. That would mean that I would need to create two more page-components for every form I need. And I usually prefer to have fewer pages, and more parts.

I know I can create reusable html-fragments for Thymeleaf. And that might fix some template duplication between all my pages. But again, that feels like a workaround, when my usecase really is something very basic.

---

Lastly it seems to me that it is inconsistent that parts are not rendered on pages for other verbs then GET.

---

PS: This feature request was for POST, but I actually want all the other verbs too.

[tsi]

So, your use-case seems quite strange.
You essentially want “parts” of the page (DOM) to reload, without using Javascript?
The options imho are then:

1. Page refresh (POST to a service that redirects back to the entire page with url params)
2. Iframes

Also, re-rendering forms (interactive content) through server is very bad in terms of UX and accessibility. Slow, resets the DOM and more…

[Alan]

@tom

@Alan Regarding posting to all parts on a page being bad system design, there is of course the obvious trap where you intend to post to one part, but all parts that export post() or all() tries to resolve it.

We cannot guess the intention… There might be hundreds of implementations out there of parts and layouts handling posts to their own URLs and suddenly they will start receiving posts intended for the entire page? Doesn’t feel right.

@tom

But the different solutions does give the developers enough tools to work around the “multiple form problem”, if that is a use case they do have.

What “multiple form problem”?
Let’s say each of your parts has its own form, and you have several of those on the page (meaning several forms on one page), and you want to post to all of them with one submit? I don’t see a problem in this. You use componentUrl of the part in action attribute of each form, and a simple javascript handler for the button which gets all form elements on the page and submits each of them.
Or is it the other way around? One form with multiple parts inside and you want each of the parts to rerender whenever the main form is submitted?

@tom

Lastly it seems to me that it is inconsistent that parts are not rendered on pages for other verbs then GET.

Can you elaborate on this one? You can render a part on POST of course.

[tom]

@tsi

So, your use-case seems quite strange.

It’s how we solved forms before we got Ajax (and how a lot of the web, and cms’ operate today).

@tsi

You essentially want “parts” of the page (DOM) to reload, without using Javascript?

The options imho are then:

1. Page refresh (POST to a service that redirects back to the entire page with url params)

Yes, this is exactly what I want.

@tsi

Also, re-rendering forms (interactive content) through server is very bad in terms of UX and accessibility. Slow, resets the DOM and more…

The impact of the refresh on UX is at best an optimization problem. (I plan to use progressive enhancement to address that).

Lets – for arguments sake – assume that my suggestion is simpler to implement for a page with a single form. Then it’s a trade off between a slightly better UX, or saved developer time. And for many use cases, the latter is preferred. At lest developers should be able to decide which option to choose.

I don’t know if you guys remember, but accessibility is sort of my specialty (I did a talk at an Enonic meetup). Accessibility doesn’t suffer from re-rendering the page as long as the html is correct. In some ways it’s easier to follow best practices when we do a refresh.

Like this tip from the gov.uk design system (which is famous for good a11y):

add ‘Error: ’ to the beginning of the <title> so screen readers read it out as soon as possible

Answering @Alan:

@Alan

There might be hundreds of implementations out there of parts and layouts handling posts to their own URLs

Yes, this would be a breaking change. There are two possible solutions to that:

1. Implement it in the next major version bump
2. Use different callback names then post and put for this functionality.

And yes, I can see the pain in both those paths (but you should do both!)

@Alan

Let’s say each of your parts has its own form, and you have several of those on the page (meaning several forms on one page), and you want to post to all of them with one submit? I don’t see a problem in this. You use componentUrl of the part in action attribute of each form, and a simple javascript handler for the button which gets all form elements on the page and submits each of them.

We agree on this. In my feature request there would be a collision if you have multiple forms trying to submit to the same page. But that is not really a problem, since in those cases, the developer can fall back to what you describe

@Alan

Can you elaborate on this one? You can render a part on POST of course.

Can you try this:

1. Deploy this application
2. Create a template from the page named “first”.
3. Put a part named “myform” inside it.
4. Then open it as a page
5. Press the “Send name”-button
6. See the same result as in the images in the first post above.

If I’m doing something wrong in my code, please let me know.

[tsi]

I finally understood what your problem is

So, the response from the POST was this:

<!DOCTYPE html>
<html lang="no">
<head>
  <meta charset="utf-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>First Example</title>
</head>

<body data-portal-component-type="page" style="background: lightblue;">
  <h1>Public page with POST</h1>
  <main data-portal-region="main">
      <!--# COMPONENT /main/0 -->
  </main>
</body>
</html>

Pay attention to the html comment there, it is the component placeholder.

If you take a look at the Site engine flow, there is a step called “post processor”.

The post processor finds component tags, and renders each component.

By default, post processing is enabled.
See details here: https://developer.enonic.com/docs/xp/stable/framework/http#http-response

However, as an optimization postProcess is by default set to “false” for POSTs. (The documentation unfortunately does not mention this currently).

As such, the solution should be to simply specify postProcess: true in your response, like this:

return {
  postProcess: true,
  body: thymeleaf.render(view, {} 
  ...
}

Now, here comes the caveat, even if I turn this flag on, things still don’t work - this is a bug
We’ll fix it!

Accessibility is a complex topic. But any user that can “see” even just fractions of a page will suffer greatly if a page needs to be fully reloaded before an error is displayed, and for the completely blind, context is also reset. Getting instant feedback on typing/leaving a field is unbeatable IMHO

Now, to the final part. Is this a viable approach in XP?
It can be done, but you have to place extensive logic into your page controller/template, that is aimed at a specific part. You will have to ensure only the right part. You are likely to get a lot of uncontrollable elements in your page and “response” html. Would I do things this way? No. In this case I would rather place the form into the page controller itself.

[tsi]

After some consideration, maybe the postProcess should be true by default also for POST, but only when content-type is text/html… (Does not apply to http services naturally)

To be concluded…

[tom]

Very good! Thanks to @tsi and @Alan for spending your time on this.

Just to be sure, it is the post() method in the part that will be called when postProcess=true in the page controller Response?

---

@tsi

But any user that can “see” even just fractions of a page will suffer greatly if a page needs to be fully reloaded before an error is displayed

@tsi

Getting instant feedback on typing/leaving a field is unbeatable IMHO

Some research actually indicates that validation on form submit is best.

I have not decided if I want “inline validation” or “validation on submit”. But if I choose to add “inline validation”, I want to add it as a “progressive enhancement”. This means that I want my form to work without JavaScript (or if the frontend JavaScript is broken).

Security concerns make also it so that I have to do serverside validation anyway.

---

@tsi

…and for the completely blind, context is also reset.

Yes, but that context change is part of the “deal”, when you have a submit button.

---

Thanks Thomas and Alan!

Let me know if you want to sparr around any of these topics! You can find me on the Enonic Community Slack.

[tom]

@tsi

Now, here comes the caveat, even if I turn this flag on, things still don’t work - this is a bug

We’ll fix it!

Was there an issue created for this bug on Github?

[Alan]

Yes, we have created an issue for this: https://github.com/enonic/xp/issues/7585

[tom]

Hey guys. Was this issue ever resolved and released?

I see #7585 was closed, but I still can’t see my parts when I do a HTTP Post (I’m on Enonic 7.2.1).

I have postProcess = true in my controller response.

[pkw]

I took a quick search in our github. The #7585 is included in 7.2.1.

[marthasimons2]

@tom

We’ll fix it!

Gug fixed??

[asi]

Hi @marthasimons2 this bug is fixed.

[tom]

Hi guys,

Does anyone know if this was ever resolved? I’m running into this issue in a project on XP 7.13.2.

Components are not being rendered on a page on POST-requests, even if I have postProcess = true in my response in the page-controller.

[Alan]

Has it stopped working after migrating from an older version to 7.13.2? Do you know on which version of XP the same/similar code worked for you?

[tom]

@Alan

Has it stopped working after migrating from an older version to 7.13.2?

No, this code is completely new. I only tested it in a 7.13.2-project.

I don’t remember how it was resolved 2 years ago (if I ever got that code working).