[Fix]: Solr remote url and properties settings #36

GridexX · 2023-07-14T10:05:00Z

Context

Setting an external instance of Solr isn't working.
As describing in the documentation, you must change the INDEX_HOST and INDEX_PORT for that.

It should then update the xwiki.properties file with the following content:

solr.type=remote
solr.remote.baseURL=http://$INDEX_HOST:$INDEX_PORT/solr

Test

After some tests, it appears that the properties baseURL is not added in the file.

Step to reproduce this issue

Launch the docker image with variables settled:

 docker run --rm -p 8080:8080 --name xwiki -e INDEX_HOST=solr -e INDEX_PORT=7000 xwiki:lts

Retrieve the solr properties inside the xwiki.properties file:

docker exec xwiki cat /usr/local/tomcat/webapps/ROOT/WEB-INF/xwiki.properties | grep solr

Which output the following:

solr.type=remote
#-# The default is the subfolder "store/solr" inside folder defined by the property "environment.permanentDirectory".
# solr.embedded.home=/var/local/xwiki/store/solr
# solr.remote.baseURL=http://localhost:8983/solr
# solr.remote.corePrefix=xwiki
# solr.indexer.batch.size=50
# solr.indexer.batch.maxLength=10000
# solr.indexer.queue.capacity=100000
# solr.synchronizeAtStartup=false
#-# Indicates which wiki synchronization to perform when the "solr.synchronizeAtStartup" property is set to true.
# solr.synchronizeAtStartupMode=FARM
# eventstream.store = solr

As we can see, the baseURL is not the expected one (http://localhost:8983/solr). We should have the following value, based on what was filled in the environment variables:
solr.remote.baseURL=http://solr:7000/solr

ℹ️ This problem concerns both LTS and Stable version, every docker images are impacted.

GridexX · 2023-07-14T10:18:49Z

Changes

Rewrite the old solr.remote.url property to solr.remote.baseURL. From the warning message:
[solr.remote.url] property in xwiki.properties file is deprecated, use [solr.remote.baseURL] instead
Update the xwiki_replace function to add an entry if it was missing in a file

Test

To ensure the fix is working, you can reproduce the following steps:

Download the docker-entrypoint.sh
Create the Dockerfile with the new entrypoint by copy the content:

FROM xwiki:14

COPY docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh

ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["xwiki"]

Build and run the container:

docker build -t gridexx/xwiki .
docker run --rm -p 8080:8080 --name xwiki -e INDEX_HOST=solr -e INDEX_PORT=7000 gridexx/xwiki

Launch and retrieve the config file:

docker exec xwiki cat /usr/local/tomcat/webapps/ROOT/WEB-INF/xwiki.properties | grep solr

Here is the output:

solr.type=remote
#-# The default is the subfolder "store/solr" inside folder defined by the property "environment.permanentDirectory".
# solr.embedded.home=/var/local/xwiki/store/solr
solr.remote.baseURL=http://solr:7000/solr/xwiki
...

As we can see, we get the expected properties filled

vmassol · 2023-07-17T13:59:48Z

Thanks @GridexX I see several open issues related to Solr, see https://jira.xwiki.org/issues/?jql=text%20~%20%22solr%22%20and%20resolution%20%3D%20Unresolved%20and%20project%20%3D%20%22XWiki%20Docker%20images%22%20

Is this PR trying to fix https://jira.xwiki.org/browse/XDOCKER-183 ?

Note that the way we work is by having a jira issue first and then doing a PR to fix it (and referencing the issue in the commit). See https://dev.xwiki.org/xwiki/bin/view/Community/DevelopmentPractices#HRule:AlwaysputaJIRAissuereferenceincommitmessages

Also, should the documentation at https://github.com/xwiki/xwiki-docker/blob/master/README.md#using-an-external-solr-service be updated too or not?

I've not tested your changes yet but I've noticed that you haven't updated the xwiki_replace_or_add function in docker-entrypoint.sh file located in the template directory. This is needed as it's the one that gets instantiated for the various environments when building with Gradle.

Thanks!

improve replace properties function change solr remote url Signed-off-by: GridexX <[email protected]>

GridexX · 2023-07-17T14:30:34Z

Hello @vmassol, thanks a lot for the details!
I need this fix in order to make the integration of Remote Solr inside Kubernetes possible.
It is (unintentional) the JIRA issue I was trying to fix 😄

The documentation doesn't need to be updated, as I have fixed what was written and didn't work previously.

According to what you said, I have modified my PR to include the JR issue number in the commit message and also modified the docker-entrypoint.sh 👍

Feel free to test it when you will have time. Have a nice day !

jiriks74 · 2025-06-10T14:06:28Z

Hi,
thx for this. I've just started setting up my own XWiki instance and I ran into severe issues with Solr.

The script extracts the jar file to a wrong locacion, the lastest jar is 12.2.1 which crashes on Solr 9 (I'll see whether I'll get around to making a Jira account)

Logs

2025-06-10 13:42:32.911 ERROR (coreLoadExecutor-13-thread-1) [c: s: r: x: t:] o.a.s.c.CoreContainer SolrCore failed to load on startup => org.apache.solr.common.SolrException: Unable to create core [xwiki]
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1778)
org.apache.solr.common.SolrException: Unable to create core [xwiki]
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1778) ~[?:?]
	at org.apache.solr.core.CoreContainer.lambda$loadInternal$12(CoreContainer.java:1080) ~[?:?]
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212) ~[metrics-core-4.2.26.jar:4.2.26]
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:380) ~[?:?]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
	at java.base/java.lang.Thread.run(Unknown Source) [?:?]
Caused by: org.apache.solr.common.SolrException:  Error loading class 'solr.XSLTResponseWriter'
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1229) ~[?:?]
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1058) ~[?:?]
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1745) ~[?:?]
	... 6 more
Caused by: org.apache.solr.common.SolrException:  Error loading class 'solr.XSLTResponseWriter'
	at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:553) ~[?:?]
	at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:474) ~[?:?]
	at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:940) ~[?:?]
	at org.apache.solr.core.PluginBag.createPlugin(PluginBag.java:165) ~[?:?]
	at org.apache.solr.core.PluginBag.init(PluginBag.java:335) ~[?:?]
	at org.apache.solr.core.PluginBag.init(PluginBag.java:323) ~[?:?]
	at org.apache.solr.core.SolrCore.initWriters(SolrCore.java:3086) ~[?:?]
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1137) ~[?:?]
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1058) ~[?:?]
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1745) ~[?:?]
	... 6 more
Caused by: java.lang.ClassNotFoundException: solr.XSLTResponseWriter
	at java.base/java.net.URLClassLoader.findClass(Unknown Source) ~[?:?]
	at java.base/java.lang.ClassLoader.loadClass(Unknown Source) ~[?:?]
	at java.base/java.net.FactoryURLClassLoader.loadClass(Unknown Source) ~[?:?]
	at java.base/java.lang.ClassLoader.loadClass(Unknown Source) ~[?:?]
	at java.base/java.lang.Class.forName0(Native Method) ~[?:?]
	at java.base/java.lang.Class.forName(Unknown Source) ~[?:?]
	at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:537) ~[?:?]
	at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:474) ~[?:?]
	at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:940) ~[?:?]
	at org.apache.solr.core.PluginBag.createPlugin(PluginBag.java:165) ~[?:?]
	at org.apache.solr.core.PluginBag.init(PluginBag.java:335) ~[?:?]
	at org.apache.solr.core.PluginBag.init(PluginBag.java:323) ~[?:?]
	at org.apache.solr.core.SolrCore.initWriters(SolrCore.java:3086) ~[?:?]
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1137) ~[?:?]
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1058) ~[?:?]
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1745) ~[?:?]
	... 6 more

and now I see that the config options aren't even applied. I'll see whether the built in Solr works but from what I can see (including the age of this PR) the support for a separate Solr instance has ended.

jiriks74 · 2025-06-10T15:44:49Z

Got it to connect to my solr instance by modifying the entrypoint:

docker-compose.yaml

  web:
    image: "xwiki:17-postgres-tomcat"
    entrypoint: "/docker-entrypoint.sh"
    command: "xwiki"
    depends_on:
      - db
      - index
    ports:
      - "8181:8080"
    environment:
      - DB_USER=...
      - DB_PASSWORD=...
      - DB_DATABASE=...
      - DB_HOST=...
      - INDEX_HOST=index
    volumes:
    ...

/docker-entrypoint.sh

#!/usr/bin/env bash

echo "My arguments are: $@"

HOST=${INDEX_HOST:-localhost}
PORT=${INDEX_PORT:-8983}

echo "Modifying tomcat config for remote Solr..."
sed -i "s|# solr.remote.baseURL=http://localhost:8983/solr|solr.remote.baseURL=http://${HOST}:${PORT}/solr|" /usr/local/tomcat/webapps/ROOT/WEB-INF/xwiki.properties

RESULT="$(grep 'http://index:8983/solr' /usr/local/tomcat/webapps/ROOT/WEB-INF/xwiki.properties)"

if [[ -n "$RESULT" ]]; then
  echo "Done!"
else
  echo "Error setting config! Check the script!"
  exit 100
fi

echo "Starting XWiki..."
/usr/local/bin/docker-entrypoint.sh "$@"

Now I have to make Solr work...

jiriks74 · 2025-06-10T21:41:17Z

~~I think~~ I figured out the Solr config with the help of #37:

docker-compose.yaml

  index:
    image: "solr-extra:9" # We have to install unzip to the solr:9 image
                          # Has to be a image rebuild since solr runs under an unpriviledged user
    environment:
      - XWIKI_VERSION=17.4.0 # https://maven.xwiki.org/releases/org/xwiki/platform/xwiki-platform-search-solr-api/
    volumes:
      ...

solr-init.sh

#!/bin/bash
set -euo pipefail

XWIKI_VERSION="${XWIKI_VERSION:-17.4.0}"

echo "Setting up Solr for XWiki version ${XWIKI_VERSION}"

echo "Downloading XWiki configuration files for Solr..."
curl -fsS "https://maven.xwiki.org/releases/org/xwiki/platform/xwiki-platform-search-solr-api/${XWIKI_VERSION}/xwiki-platform-search-solr-api-${XWIKI_VERSION}.jar" \
  -o /tmp/xwiki-platform-search-solr-api.jar
echo "Success!"

echo "Extracting configuration files into place..."
rm -rf /tmp/xwiki
unzip -oqq /tmp/xwiki-platform-search-solr-api.jar -d /tmp/xwiki/

# See /usr/local/xwiki/data/store/solr in the XWiki (web) container for Solr core names
# Don't forget to add a prefix set in XWiki (web) tomcat config (default: xwiki)
#   - The tomcat config file location in the XWiki (web) container:
#     /usr/local/tomcat/webapps/ROOT/WEB-INF/xwiki.properties
unzip -oqq /tmp/xwiki/xwiki-platform-search-solr-server-core-search.zip -d /var/solr/data/xwiki_search_9 && \
  unzip -oqq /tmp/xwiki/xwiki-platform-search-solr-server-core-minimal.zip -d /var/solr/data/xwiki_events_9 && \
  unzip -oqq /tmp/xwiki/xwiki-platform-search-solr-server-core-minimal.zip -d /var/solr/data/xwiki_extension_index_9 && \
  unzip -oqq /tmp/xwiki/xwiki-platform-search-solr-server-core-minimal.zip -d /var/solr/data/xwiki_ratings_9

cp -R /opt/solr/contrib/analysis-extras/lib /var/solr/data/xwiki_search_9 && \
  cp -R /opt/solr/contrib/analysis-extras/lib /var/solr/data/xwiki_events_9 && \
  cp -R /opt/solr/contrib/analysis-extras/lib /var/solr/data/xwiki_extension_index_9 && \
  cp -R /opt/solr/contrib/analysis-extras/lib /var/solr/data/xwiki_ratings_9
echo "Success!"

echo "Setup complete."

Now I have to figure out whether the search crashes because the index isn't done yet (even though the queue goes from something to zero when reindex is triggered) or if there's some other issue.

jiriks74 · 2025-06-10T21:43:45Z

~~Ok, there's and issue:~~

2025-06-10 21:43:02,816 [XWiki Solr index thread] ERROR o.x.s.s.i.DefaultSolrIndexer   - Failed to process entry [INDEX Object_property xwiki:XWiki.Realtime.WysiwygEditor^XWiki.UIExtensionClass[2].scope] 
java.lang.NullPointerException: null

EDIT: Fixed it. See updated comments above for corrected setup scripts.

Comment links:

GridexX force-pushed the fix/replace-properties-function branch from a2f074e to 3ffa03f Compare July 14, 2023 10:06

GridexX force-pushed the fix/replace-properties-function branch from 3ffa03f to 8812f1e Compare July 17, 2023 14:24

XDOCKER-183: enable remote Solr with variables

a91dcb8

improve replace properties function change solr remote url Signed-off-by: GridexX <[email protected]>

GridexX force-pushed the fix/replace-properties-function branch from 8812f1e to a91dcb8 Compare July 17, 2023 14:25

GridexX mentioned this pull request Jul 21, 2023

XDOCKER-282: Update the process to use an external Solr instance #37

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Fix]: Solr remote url and properties settings #36

[Fix]: Solr remote url and properties settings #36

Uh oh!

GridexX commented Jul 14, 2023

Uh oh!

GridexX commented Jul 14, 2023 •

edited

Loading

Uh oh!

vmassol commented Jul 17, 2023

Uh oh!

GridexX commented Jul 17, 2023 •

edited

Loading

Uh oh!

jiriks74 commented Jun 10, 2025

Uh oh!

jiriks74 commented Jun 10, 2025

Uh oh!

jiriks74 commented Jun 10, 2025 •

edited

Loading

Uh oh!

jiriks74 commented Jun 10, 2025 •

edited

Loading

Uh oh!

Uh oh!

[Fix]: Solr remote url and properties settings #36

Are you sure you want to change the base?

[Fix]: Solr remote url and properties settings #36

Uh oh!

Conversation

GridexX commented Jul 14, 2023

Context

Test

Step to reproduce this issue

Uh oh!

GridexX commented Jul 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Test

Uh oh!

vmassol commented Jul 17, 2023

Uh oh!

GridexX commented Jul 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jiriks74 commented Jun 10, 2025

Uh oh!

jiriks74 commented Jun 10, 2025

Uh oh!

jiriks74 commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jiriks74 commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

GridexX commented Jul 14, 2023 •

edited

Loading

GridexX commented Jul 17, 2023 •

edited

Loading

jiriks74 commented Jun 10, 2025 •

edited

Loading

jiriks74 commented Jun 10, 2025 •

edited

Loading