Requesting large amount of data from an API in XML format

Help
1

Enonic version: 7.1.3
OS: Ubuntu 19.10

I’ll be importing data from a certain API, which provides an XML in the response. My main goal is to created objects in Content Studio based on the response sent back to me.
One important thing to note is that the response sent back from the API is quite large, the XML has around 160K lines.
Does anybody know a good strategy for doing this, in terms of fetching data from the API? What I mean is, is it okay for me to do this straight on Enonic by using http-client and then converting the XML to JSON or, since we’re dealing with a very large amount of data, will I be better off handling all this directly in JAVA (request + conversion)?

2

Converting the full XML document to JSON is likely to require a lot of server memory, so I think that for this situation it might be wise to try a solution in Java.

There was a similar thread a few years ago, which wasn’t the exact same scenario, but the comments there may still be relevant: Working with large amounts of data/content/content-types

I’ve browsed through @rmy 's GitHub repo and found a slightly newer version of the app discussed in the thread above, where CSV data is imported using Java: https://github.com/runarmyklebust/xploader
…but since it’s not on market and undocumented, it’s probably in an experimental state.