Toolbox export don't export content


#1

Enonic version: latest
OS: any

When I do an export + import I expect the content to be exactly the same on the import side. This is not so in my experience.

Content created with lib content.create() does not export content with toolbox export, when display name contains diacritic characters. If content is successfully created it seems strange it can’t be successfully exported with toolbox export. If it can’t be exported it shouldn’t be legal content and not created. It will create inconsistencies, and if there is one thing we developers don’t like that much is inconsistencies. Content created must be allowed to be exported allways, you will create all sorts of problem if export don’t allow created content to be exported.

The export function provided is not at all reliable as of now and your user base should be adviced with a large warning box that the export function must not be used for backup if you have content created from libs and not from user interface.

In my use case I have 700 items created by content.create() but about 400 available in the export. Which is not so good when I need the backup to be restored from an export.


#2

There is a known bug for pathname containing : on windows. : will be a forbidden character in 7.0.

Also, please check that you are importing/exporting from the same branch that you are creating content!


#3

It’s the export that is the issue - the exportet content don’t contain the items that is content. The import works fine, well I guess because all the “bad” content is gone :smiley:


#4

This was content containing norwegian characters like ÆØÅ. If you create content with content.create() this content will not be exported, so it’s not restricted to colon @tsi


#5

Hmm… Never heard about this before?
Creating content from UI or API all goes through the same API, so there is no difference here.

Can you confirm your are exporting from the same branch you are creating content in? Remember default branch is master if nothing is specififed.


#6

All content is on draft since it has been edited. There is a difference when you create content in user interface, and that is that the path will be fixed such that it don’t contain norwegian characters - not so when content is created from lib. Typical error messages I get in toolbox export is
"java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /content/mysite/kategorier/?stfold Museene"
Path created is /kategorier/Østfold Museene" which you don’t get to do in user interface - this is auto corrected.


#7

Try writing øæå directly in content studio path, should not be a problem!
Can it be that you are somehow not using UTF charset when creating these items?


#8

Well I tried to change to (lowercase) “ø” directly in path (Mac) and this didn’t work. Changed path to “Ostfold Museene” and the error message disappeard and I also verified content on file system (export). Changed path back to “Østfold Museene” and the error message reappeared. So writing æøå in path seems indeed to create the problem.


#9

I Just created and exported content with øæå on my mac without any issues whatsoever. Here is the result of the export as seen in finder:
28

This is strange indeed?


#10

Yes, this is very strange - I did exactly the same: Created a new folder “Østfold folder”, manually modified path, and I get
"java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /content/mysite/kategorier/?stfold-folder"
I have xp 6.15.4 on OSX 10.13.6 (High Sierra)
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
UPDATE: Oh this is on my local - export is performed on a Centos machine. I can try export from my mac instead - might work better


#11

Retested on my mac and the export works fine. So this problem is related to the production Centos machine!


#12

Ah… I’m afraid your underlying OS may be configured to use something else than UTF-8 as system locale.
Has been quite a while since I heard about anyone having that problem, and then I believe the local operations department had deliberatly changed it from the default.

So, solution is to fix your system locale to utf-8, and things should work fine!


#13

You are exactly right :smiley: restarted xp with proper utf-8 settings and now the export workes like a charm!

Thank you so much :smiley:

(this was a default GCP server, so obvously the default setting from Google or perhaps Centos isn’t that reasonable)


#14

Final notes - on a default GCP centos server you should add the following lines to /etc/environment to avoid this problem:
LANG=nb_NO.utf-8
LC_ALL=nb_NO.utf-8
And thanks again to Thomas for pointing out the problem and resolving the issue!


#15