Hi! I need to query some content by fields present in objects related to it. But I didn’t found any documentation about how can I do that with enonic. For example:
I have a folder that holds the “original” books objects which means that these objects are filled with information. Also, I have on a tree structure, book shortcuts, that points to the related books. This was our choice to avoid duplicated content.
On this project we have a search that I need to order the results by the book publish year, but shortcuts doesn’t have this property. How can I do that without add an attribute “year” on the shorcuts since I do not want duplicate an information that already exists on my base, like “data.relatedBook.year DESC”?
So the books folder has the content with all the data, but you want to return shortcuts that point to the books and ordered by year? I think this could be solved by doing a content.query() on the books and ordered by year. Then loop through the results and do a content.query on each book result for the shortcuts with data.target = bookResult._id and put those results into an array.
It’s hard to say exactly how to do this without seeing the content structure but you can do as many queries and loops as you need.
2 Likes
Today, the approach we use is the one you suggested. But this is not the right way to do things… If enonic doesn’t support the query on related structures, it had at least work with some kind of JOIN on content queries. I think this is a limitation that may impact hardly big projects, since they cant use this kind of feature nativelly, needing workarounds to have a better work…
Joins Are The path to certain death of performance - @rmy, can you comment on this?
Also, why have you used shortcuts and not tagged content instead?
What you mean with tagged content? I need the shortcuts to use a tree-structured content, which some objects need to be in several locations… and this includes the need to search them on the tree by using attributes from the “original” object. I know that performance can be affected, but I think that some kind of feature can be present on Enonic, if the users use or not, it will be there…
Maybe you could post some more info about this structure? Screenshots of tree + selected content types would be nice
Hi.
Since we are not storing data in a relational database but rather the document-oriented elasticsearch, this is kind of tricky. The best approach for now is to do this in two steps. I know that in ES2.2+ support for terms-lookups are possible (https://www.elastic.co/guide/en/elasticsearch/reference/2.2/query-dsl-terms-query.html), but we are not on ES2 yet.
I have had an idea of a library for doing something similar, Ill give it a short try and see what Im able to put together.
1 Like
Ok, after a quick look, its possible to create something for this, but its quite difficult to make a general lib for this (filtering by related content query is easy, ordering gets complex very quickly).
I think we will have to do an internal discussion on this topic.
2 Likes
My point here is that actually we don’t have a feature of relational DB that are joins, neither the best feature of the document-structured db, that is the possibility to have documents tree-structured. I know performance can be a gap in this world, since it could need more time to mount the objects at runtime to perform a seach in it… What about a new structure on contents that holds the related objects (it may be a option disabled by default, so, only who wants to use it will do), like we have for now the “data”, and “eXtraData” on enonic objects, we could have (when this option is enabled) a related-data, which holds contents related on a objects, keeping the tree structure and reindexing them if a object is updated on its relationship. Elastic search works well with this kind of property, as we can see in the use of x-data mixins. Here is an article that speaks about relationships on ES, which is not far from the real world needs, since we can need at sometime related objects.
https://www.elastic.co/guide/en/elasticsearch/guide/current/relations.html
A example of what I’m trying to talk:
For now, we have a Book-shortcut like this:
{
id: 123abc,
data: { book: 123 }
}
a book like:
{
id: 123,
data: { title: “The three little pigs”}
}
The solution could be like:
{
id: 123abc,
data: {book: 123}
relatedData: {
book (which is the name of the ContentSelector Property): {
id: 123,
data: { title: “The three little pigs” }
}
}
}
I suggest you rather try to work with what is already there. As I asked earlier, would be nice to see more details on your structure. However, I assume now you have created a tree structure, and it contains references to the books. Then you have books with all the data?
If you turn this around, by linking from books to the structure (using the tree structure as metadata), just like tagging your books with tree-structure metadata. Then you should be able to build powerful search queries for books and structures out of the box.
1 Like
Interesting discussion, here are some thoughts from me on this of the things mentioned in the article;
-
Application-side joins
What we recommend at the moment. -
Data denormalization
Its no problem to apply relational data to the parent node in index, the problem is to keep things in sync when you do a change on a content that has an incoming reference. You then have to change the data for the referring contents, which in theory could be “all the world”. Also, if storing this on the actual node, you will run into problems with keeping the integrity of the original node; should it be changed? It certainly have to if changing the actual data; then a new version will suddenly be created. Only viable solution then is to just store this information index-time, but that means that the index-process will be magnitudes slower since you have to follow all relationships (e.g loading nodes). -
Nested objects
That doesnt really fit our information model at all. -
Parent/child relationships
Parent/child-relationships that are mentioned in the ES-article has side-effects that can be very complex. All entries in a parent/child relation has to reside on the same shard; which could case sliding effects on your data if a certain document is referred from a lot of other nodes, which again has relations to other documents etc. Suddenly, all your data will be forcefully placed on a single shards on a single node.
The only real solution I can see atm is to implement some kind of join logic in a lib, e.g maybe in the contentLib, but its really not straight forward to support every usecase. Paging, sorting etc all soon becomes quite complex since we doesnt have a set model to work with. But its an interesting topic to dive into, maybe we will find a solution that can fit most usecases, either in a 3’rd party lib or builtin.
1 Like