what are you trying to achieve
Install a 3rd Party Plugin in a Docker Setup
what have you tried so far to achieve this
Searched the documentation for clues
====
Dear all,
after having created a docker setup as documented here: Install the Server | SonarQube Docs i now would like to endulge myself in the possibly hazardous procedure of manually installing a 3rd Party Plugin.
The documentation here: Install a Plugin | SonarQube Docs does describe the steps needed for a ānormalā plugin installation (i guess)
Am i assuming correctly that i could handle the step of copying the plugin jars inside a custom Dockerfile?
I already created such a Dockerfile to inject our custom certs that were needed to talk to the LDAPS.
So i would add more COPY commands to that Dockerfile to copy the jars into the target directories, right?
(please excuse my ping to @Tobias_Trabelsi here, i hope i am assuming correctly that he might be quite knowledgeable concerning these container things? )
cheers
Daniel
//edit:
P.S.: Maybe it might be helpful to extend the documentation by mentioning how to handle the (manual) installation of 3rd party plugins in a docker based setup
But i think these things might be of use for Tobias, too. Give and take.
I already mentioned the install docs you qoted in my OP, so obviously i was not able to make the distinction about my question clear enough.
Handling the volumes is not easy (and not part of Install a Plugin | SonarQube Docs) ā¦ to get a glimpse of the steps you have to research please take a look at
(most of the answers needed to solve my question are seemingly taken care of there ā¦ i quote it here not only because of that, but also to enable you to realize how much of a SEP-Field you make this if you do not integrate helpful hints into your Install a Plugin | SonarQube Docs)
Here's a one-liner that copies myfile.txt from current directory to my_volume:
docker run --rm -v $PWD:/source -v my_volume:/dest -w /source alpine cp myfile.txt /dest
One cannot easily write to a volume, one has to create a container to manipulate the volume
And if i understand the documentation here correctly, contents of a directory that a volume mounts to get copied into that volume ā¦ so in the end, maybe my way of prepopulating the plugins in my own Dockerfile might be a pragmatic solution.
I am now even more under the assumption that āthe sonar (preferred ) wayā of interacting with a volume to manually install a plugin in a docker setup is worth being documented in Install a Plugin | SonarQube Docs to help the next person thinking about this.
there are a few things here, so let me try to explain.
To install 3rd party plugins, you should in theory be able to use the marketplace. with the named volume mounted to the plugin install directory these installations should persist in case of a change of an image etc.
If you want to do it programatically you can define a additional container that downloads the jars of the plugins you want to have in the named volume (up to you).
if you want to use a local filesystem mount from your docker host, this is also possible since 8.5 where we embed the language analyzer in SQ directly. So since then you can use something like this:
i am writing compose here, but of cause you can use the respected docker run command as well. The only thing that you need to take care about, would be that the SQ user inside the container has the correct permissions to the docker host path.
Hi Tobias, thank you for taking the time to create your answer!
If i may, i would like to go over your reply, so that you might be able to correct any mistakes in my perception
The docker-compose.yml you quoted is the Docker Compose .yml file example from Install the Server | SonarQube Docs with the following one line changed, right?
- /path/to/your/plugins:/opt/sonarqube/extensions instead of
- sonarqube_extensions:/opt/sonarqube/extensions
which makes the extensions bind-mounted instead of volume mounted. If i understand correctly, then:
Using the bind-mount instead of the volume would embed the directories content into the container and the volume would stay empty (probably? )
to get the plugin(s) into the volume i [c|w]ould
run the docker-compose with volumes and install plugins via marketplace (works for CE, but not for DE/EE/DCE because there plugins now have to be installed manually, if i understand correctly) so that they get stored in the mounted volume
create a dummy-container and in that process could inject them into the volume
create my own Dockerfile/image based on āFROM sonarqube:$tagā where i COPY all plugin-jars to the right directories (because on first mount, any contents of the directory get copied into the volume)
Anything obviously wrong in that summary?
cheers
Daniel
P.S.: I still would like to suggest that an addition to Install a Plugin | SonarQube Docs would be beneficial to everyone trying to install plugins while using docker. (And even more when not on CE, because then you have to install them manually now, if i understand it correctly)
yep but this is not bad in this case. this used to be a problem before 8.5 but now when the filesystem permissions are correct, SQ will create the required folders.
no the installation via the marketplace is working with CE/DE and EE. only on DCE this is not possible and needs to be done manually. how do you come to this conclusion?
This is also why we donāt document this directly. there are only edge cases where you need to install plugins manually. the majority of the use cases are just click install on the web UI and restart sonarqube (also from the web ui).
Actually that is my understanding of Log in with Atlassian account ā¦ to quote from there (the whole MMF is interesting, i am just highlighting to focus on my current assumption):
If i combine the consequences of this, i assume something has to be performed manually/automatically to make the plugins available to a volume that SQ in the container uses. (if i bind-mount, the process is more clear. i understand that.)
Only in the CE i can just fire up a compose with volumes only and install via Marketplace that then persists inside the volume, if understood correctly. I would be glad if i missed something there.
i stand corrected. sorry i was working on something else and did not notice the change. i checked and your understanding is correct.
with this in mind i now better understand where your are coming from and that there is a lack of documentation here. thank you for pointing that out.
can you manage now to install a plugin manually or do you need additional help with the procedure?
thank you for reminding me of this open wound ā¦ No, actually it now is a rather patched up wound.
Yes, i indeed have found a way to set up my docker compose + a Dockerfile with integrated plugin. Getting that was actually rather easy, setting it up with a fitting docker-compose was more complicated but i managed to get something ready i can work with.
For the Dockerfile i assume following conditions fulfilled in the directory where the Dockerfile resides:
Custom ssh certificate keystore in a file named ācacerts.customcertstoreā
Additionally the environment variables for the JDK_KEYSTORE_PASSWORD* entries need to be set/supplied (or you replace those inside the Dockerfile with real passwords, not advisable but more easy to test with)
File: Dockerfile
FROM sonarqube:8.9.0-community
COPY /cacerts.customcertstore /tmp/cacerts
ARG JDK_KEYSTORE_PASSWORD_SRC
ARG JDK_KEYSTORE_PASSWORD_DEST
RUN keytool -importkeystore -srckeystore /tmp/cacerts -destkeystore ${JAVA_HOME}/lib/security/cacerts -srcstorepass $JDK_KEYSTORE_PASSWORD_SRC -deststorepass $JDK_KEYSTORE_PASSWORD_DEST
COPY /plugins/sonar-softvis3d-plugin-1.2.1.jar /opt/sonarqube/extensions/plugins/
RUN chmod -R o+x /opt/sonarqube/extensions/plugins/
RUN chown -R sonarqube:sonarqube /opt/sonarqube/extensions/plugins/
RUN echo "hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4" > /etc/nsswitch.conf
RUN echo "sonar.telemetry.enable=false" >> /opt/sonarqube/conf/sonar.properties
RUN echo "http.nonProxyHosts=firsthostname|192.168.1.11" >> /opt/sonarqube/conf/sonar.properties
cheers
Daniel
P.S.: This setup is definitely not optimized concerning best practices. It is a quick and dirty creation which fulfilled my needs at the time. You can and should adapt it to create something more efficient.
P. P.S.: Tobias @Tobias_Trabelsi , i would be glad if you might be able to add any insight into what might be (dis-)advisable concerning this example. It is just a quick shot, which worked for me. Would definitely be nice to gather some suggestions, if any available. (besides obvious best practices like e.g. ādont put each RUN in its own lineā )
oh, there you go: full idea for the setup goes as follows:
create a directory that contains the following docker-compose.yml + the .env file.
by redirecting to the .env file you can run multiple stages on the same machine as the data will be managed in the respective volume-names that are managed by docker on its own
Thank you so much, Daniel, I really appreciate the effort that youāve put into this project. I hope that SonarQube will update their documentation to cover this since they removed the ability to update the DE and other paid editions.
sonarqube will start up and you need to ack the warning about 3rd party plugins and be good to go.
If you donāt want to utilize the file system of the docker host, you can write a init container in your compose as well. This would be an example that uses the init capability that was added to compose with 3.7 (donāt quote me on thatā¦ somewhere in the late 3.x spec):
Thank you @Tobias_Trabelsi for your insights! I think i definitely need to brush up my knowledge about those init containers (i think i wrote something similar already some time ago, didnāt i ahh, nvm)
Concerning the āengineeringā part: My decision was based on either editing the compose file or editing the env-file when i want to upgrade while retaining the possibility to down-grade if smth fails.
For example: I just realized today that there are already two minor LTS updates for 8.9. i can copy my compose setup, clone the volumes to new names, edit the .env to point to the new volumes and try out the upgrade.
I could, ofc, do this all in-place in the existing compose setup ā¦ but ā¦ i perceive that as a bit more messy.
cheers
Daniel
P.S.: As always, if my reasoning doesnāt sound sound, sound a controverse voice, please
This is a decision that you made on your personal use case.
maybe as a fyi: when you want to test a upgrade or downgrade from a failed migration you only need a database backup. all other data (not including the 3rd party plugins) can be recovered from this.
anyway this is a little out of context for the original question of which i think is now covered enough
Please excuse the following if i am mistaken! (But:)
My interpretation right now is that this thread has been marked āavailable for closureā by you without me marking an answer as a Solution.
If that is the case: please unmark it, this is not something i would want to happen with this thread.
It is your (sonarsources) forum, which implies that you (sonarsource) obviously have every right to moderate as you want. You already found a way to āmass-administrateā many issues into discontinuability by forcing closure after someone marked an answer as a reply and seven days have passed. (i myself am not a friend of this forced mute-ing)
Please do not start to do this on purpose without asking for the thread openers agreement. And to repeat: if that is none of your doing, please do take it kindly (take it kindly too, if it is your doing ) ā¦ but please, if you can, see to undo it.
Hi @ganncamp, this is baiting to create the grounds for setting an example why this thread is in need of closure
I started a different thread already because of this. Might be a bit diluting because there i include a slightly different POV, but most probably our conversation should continue here
To give a short answer here (and repeat my asking for un-setting the solution, which definitely is not the reply i would select): I like to keep my threads open because the forced closing kills community interaction. And sometimes the better answers take longer than this artificial time-frame allows for.