starred/cloudbeaver

Fork 0

mirror of https://github.com/dbeaver/cloudbeaver.git synced 2026-04-25 05:36:14 +03:00

[PR #649] [CLOSED] Feature: fully dockerized multi-stage build with 1 single Dockerfile + Github's CI/CD Pipeline #1848

New issue

Closed

opened 2026-03-07 21:06:00 +03:00 by kerem · 0 comments

kerem commented

2026-03-07 21:06:00 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/dbeaver/cloudbeaver/pull/649
Author: @marcellodesales
Created: 2/11/2022
Status: ❌ Closed

Base: devel ← Head: feature/dockerized-multi-stages-build

📝 Commits (10+)

2540b77 dbeaver/cloudbeaver#663 ♻️ 🐳 Dockerfile: ultimate multi-stage builds
c4d57a6 dbeaver/cloudbeaver#663 🙈 ignoring the workspace dir when running in docker
05b4d36 dbeaver/cloudbeaver#663 ♻️ 🔒 run cloudbeaver with different user
8a6e332 dbeaver/cloudbeaver#663 ⚗️ Add the postgres example to connect to dbeaver
67730dd dbeaver/cloudbeaver#663 🔥 Removing shell scripts that calls deploy/docker
405e117 dbeaver/cloudbeaver#663 👷 🐳 Add the dockerized CI/CD pipeline: docker image
4863d3b dbeaver/cloudbeaver#663 💡 Add comment docker-compose.yaml for prd volumes
1cd104e dbeaver/cloudbeaver#663 💚 🐳 pipeline fix: docker login before push
8c4a06a dbeaver/cloudbeaver#663 🚚 Rename docker-compose.yml to .yaml
048c9f7 dbeaver/cloudbeaver#663 ♻️ 🚚 rename the docker image before build/push

📊 Changes

10 files changed (+391 additions, -27 deletions)

View changed files

➕ .github/workflows/docker-image.yaml (+219 -0)
📝 .gitignore (+2 -0)
➕ Dockerfile (+91 -0)
➖ deploy/docker/.gitignore (+0 -1)
➖ deploy/docker/Dockerfile (+0 -8)
➖ deploy/docker/docker-compose.yml (+0 -10)
➖ deploy/docker/make-docker-container.sh (+0 -3)
➖ deploy/docker/run-docker-container.sh (+0 -5)
➕ docker-compose.yaml (+57 -0)
➕ samples/postgres-docker-compose.yaml (+22 -0)

📄 Description

🐳 Ultimate Multi-staged Dockerized builds + Gihub's Dockerized Pipeline to Docker Hub and Github Container Registry

This commit adds initial implementation of multi-stage builds that replicates the steps from deploy/build.sh. The reason why is very simple: docker multi-stage builds are parallelized and all the stages steps are eligible for cache. This is mostly done because I want to have a very quick developer experience with DBeaver in Kubernetes and the only way to get it right is to properly have a clean and light docker image that I can run in both Kubernetes or during Services Development. That is, I don't want any Engineer in my company to have to install DBeaver as a Desktop, but use this containerized version to quickly test their services... An option that I've been using is docker-compose for development and Kubernetes for Runtime in Production... The story is very simple: a UI engineer would not know how to setup the Database nor care about it, but with a 1-liner setup they have considered to use this setup...

Personal problem with current version: After running the App, we can use DB locally without using the infamous deploy/build.sh that depends on specific versions of required build systems, as well as requiring the users to know all the pieces... (deploy, etc)

Goal: A single Dockerfile that performs 3 builds, uses 4 stages, and coordinates the build in parallel: Get myself using Cloudbeaver in 1 command!

NOTE: I like this too so much that If this PR is merged, I will also contribute the Github CI/CD pipeline for the Docker image + the deployment scripts I use for Kubernetes/Kustomize/ArgoCD deployment for CloudBeaver! :D that uses this docker image

🍷 Cheers!

NOTE2: The runtime docker image size is super small 266.9 MB

$ docker-compose images
            Container                    Repository         Tag       Image Id       Size
-------------------------------------------------------------------------------------------
cloudbeaver                          dbeaver/cloudbeaver   latest   b0a0c4b7bf34   266.9 MB
cloudbeaver_cloudbeaver-backend_1    <none>                <none>   bdde1c816c2c   1.32 GB
cloudbeaver_cloudbeaver-frontend_1   <none>                <none>   e4f86e49ab72   1.028 GB

📐 Architecture and Design

This docker image does NOT

The docker build is split into 3 components:

core: dbeaver: Cloned as the git repo
- Performance improvement: the build will ONLY PULL the HEAD commit instead of
  the entire history. That makes the build way faster!
- This is eligiable for a Docker build Cache: for now only main branch is cloned
- This layer adds nothing but the git repo contents
backend: cloudbeaver: the current project's server
- Added all the steps as the script
- Simplified and used full paths to avoid debugging confusion when reviewing build.sh
- Based on a public JDK docker image
frontend: web: a regular Node.js build with yard and lerna
- Same principal used here
- Based on a node.js Lerna instructions

We are providing a docker-compose as the main driver for users to properly build each individual component.

The Dockerfile was built with caches from a local build. It can be further improved if dbeaver/cloudbeaver adds this to a pipeline (I can send a PR for that as well) and sets up the env vars with the secrets to Dockerhub.

Also, users can use versions from Github's Container Registry if that's an option (for development)
The docker image users Alpine linux, a much more secure image than the original's DBeaver verison
- At least does not present the security threat described at https://github.com/dbeaver/cloudbeaver/issues/349#issuecomment-903716506
- https://avd.aquasec.com/nvd/cve-2021-33910/ does NOT list alpine linux as affected, however, you anyone can add a user as well

🔊 Docker build Logs

You can verify all the steps and where the cache was properly used in my 1s subsequent build

$ docker-compose build  2>&1 | tee output
Building cloudbeaver-backend
#1 [internal] load build definition from Dockerfile
#1 sha256:b094ed2d72447eabf2da9d70707bdce549fed1c5f926441397466b9686e339eb
#1 transferring dockerfile: 37B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:c54f08aba9edbe6b41a53e95daa74a0cf69a083680acb60bbf12130475efc077
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/alpine/git:latest
#3 sha256:53c8397ef7cbc2a2b64b29948591c91cdd7638c705f9e00a31c645ddbf85d4d4
#3 DONE 0.0s

#4 [internal] load metadata for docker.io/library/maven:3.8-eclipse-temurin-11-alpine
#4 sha256:8ec9051ab2a279abdabaf2797ebaa34fbe96426bbf67b810cc35309ea00e4fc3
#4 DONE 0.4s

#6 importing cache manifest from dbeaver/cloudbeaver-backend
#6 sha256:4f595ca88a8968210fab7ea2f6989d9719172393c434ca16acac1144d0902d8b
#6 DONE 0.0s

#8 [core-builder 1/3] FROM docker.io/alpine/git
#8 sha256:51f8b5880c93adc1c41b2c3e0ef44d59de9626d5528dd6aa0d642c23517f1198
#8 DONE 0.0s

#5 [backend-builder  1/10] FROM docker.io/library/maven:3.8-eclipse-temurin-11-alpine@sha256:67ff1b5f33540c4a7cf25138b43e890dc18e4b88baedf7f0275658e4b6c70fac
#5 sha256:edb695fdbe963a99a08d116c38c18b47fb5a4460671109479cf8387cb2582ad3
#5 DONE 0.0s

#13 [internal] load build context
#13 sha256:1fce9ba74c48461a3d723191c5dbb537e6ebed438cfbd5511317d6e0eabf1d56
#13 transferring context: 37.36kB 0.1s done
#13 DONE 0.1s

#7 [backend-builder  2/10] WORKDIR /dbeaver/cloud/backend
#7 sha256:d4e1ddc9189a7df6e98c04acb060ee10b609f62e183746d1842fef7c6e549c25
#7 CACHED

#18 [backend-builder  9/10] COPY samples /dbeaver/cloud/backend/cloudbeaver/samples
#18 sha256:e3d7452e06df82c8d931fc6cfea9a1fd0958eeeea38982fedbce50bba255185b
#18 CACHED

#16 [backend-builder  7/10] COPY samples/sample-databases/db /dbeaver/cloud/backend/cloudbeaver/samples
#16 sha256:e71b4233be517ada6fa19bf17690a81167cd1d3ee93104d3f2e66ee1720f29f6
#16 CACHED

#12 [backend-builder  4/10] RUN echo "Build Cloudbeaver server" &&     mkdir ./cloudbeaver &&     mkdir ./cloudbeaver/conf &&     mkdir ./cloudbeaver/samples &&     mkdir ./cloudbeaver/workspace &&     mkdir ./cloudbeaver/web
#12 sha256:578f46f6a902bfaf39369b70cebf1a437e16bb2f9f9345ab5af97fdcccc7ffdb
#12 CACHED

#14 [backend-builder  5/10] COPY server /dbeaver/cloud/backend/server-source
#14 sha256:5edc9a310cfe22e11702e599301f5f07031192ae98fa02b2a121483960e305bf
#14 CACHED

#15 [backend-builder  6/10] RUN echo "Build CloudBeaver server" &&     cd /dbeaver/cloud/backend/server-source/product/aggregate &&     mvn clean package -Dheadless-platform:
#15 sha256:7b3bd765c25356b013642f7d8b3860e905dddd9f78b20965cdf51e734c8f9fe4
#15 CACHED

#10 [core-builder 3/3] RUN echo "Clone dbeaver platform" &&     git clone --depth=1 https://github.com/dbeaver/dbeaver.git
#10 sha256:586addf8e3e5d2189836b1105c7c4f265de2f7cfafaa1a2e72b5dc1898f5c9ac
#10 CACHED

#11 [backend-builder  3/10] COPY --from=core-builder /dbeaver/core/dbeaver /dbeaver/cloud/dbeaver
#11 sha256:243f6c10c42b5a77f43eca93b11eda46b1f83571f1a65c09c9d36de0c4928acd
#11 CACHED

#17 [backend-builder  8/10] COPY deploy/scripts /dbeaver/cloud/backend/cloudbeaver/scripts
#17 sha256:6d1fd0e05ff0be709ec3d7326b7d8676a69ed0081d30f092cddaa3e9a87c6f60
#17 CACHED

#9 [core-builder 2/3] WORKDIR /dbeaver/core
#9 sha256:81d6845562bcab73a8ae7ad3c8ce5b5f8f86f9065fae9dc5466ad139d31f4bc7
#9 CACHED

#19 [backend-builder 10/10] RUN echo "Copy server packages" &&    mv /dbeaver/cloud/backend/server-source/product/web-server/target/products/io.cloudbeaver.product/all/all/all /dbeaver/cloud/backend/cloudbeaver/server &&    cp /dbeaver/cloud/backend/cloudbeaver/samples/sample-databases/GlobalConfiguration/.dbeaver/data-sources.json /dbeaver/cloud/backend/cloudbeaver/conf/initial-data-sources.conf &&    cp /dbeaver/cloud/backend/cloudbeaver/samples/sample-databases/*.conf /dbeaver/cloud/backend/cloudbeaver/conf &&    mv /dbeaver/cloud/backend/deploy/drivers /dbeaver/cloud/backend/cloudbeaver/drivers
#19 sha256:3f769067327c6bf8c68ad90d192dd8eb9b712610c627e3ca79a628683f5367d1
#19 CACHED

#20 exporting to image
#20 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#20 exporting layers done
#20 writing image sha256:6111c0d5a0c7fc84d073352ca3101e242a29deadba06bb71bc3434c89d49c30f done
#20 naming to docker.io/dbeaver/cloudbeaver-backend done
#20 DONE 0.0s
Building cloudbeaver-frontend
#1 [internal] load build definition from Dockerfile
#1 sha256:59b91e01265e09a3154a18309cf68d6bfa855ffbce550ccf71b7c5ecca0f663d
#1 transferring dockerfile: 37B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:cafc13f768a3ba542bec05f6a98f3be2885cf43ef5db9be4993a633edce572b6
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/node:16-alpine3.15
#3 sha256:874a4c0aa0a7f216ec26c7b83c24ba14fdb01bbc1aa9262ffcf8648918805772
#3 DONE 0.4s

#5 importing cache manifest from dbeaver/cloudbeaver-frontend
#5 sha256:d7cafdc4d13b2a7cd7d44f9f65a6797b6663524b381f35d2acc28f798e45705b
#5 DONE 0.0s

#4 [frontend-builder 1/6] FROM docker.io/library/node:16-alpine3.15@sha256:2c6c59cf4d34d4f937ddfcf33bab9d8bbad8658d1b9de7b97622566a52167f2b
#4 sha256:fb7df7ff6aaf5b77fd015d60b9efd057eae205beca35245388ca834e00b00079
#4 DONE 0.0s

#8 [internal] load build context
#8 sha256:da45e60a892447492457650be14aa09e4db311b7f4eba2b7b15feffcde9066b8
#8 transferring context: 9.11MB 0.4s done
#8 DONE 0.4s

#7 [frontend-builder 3/6] WORKDIR /dbeaver/cloud/frontend
#7 sha256:74c6a6845fe56dbf11c0e508ce0048977dfd816ec59b9f92ef4ec8fdb6a4b38a
#7 CACHED

#9 [frontend-builder 4/6] COPY webapp /dbeaver/cloud/frontend/webapp
#9 sha256:8ef53481136edc85f177353689d9ff91035a73aa995687096269bad791f64718
#9 CACHED

#10 [frontend-builder 5/6] RUN echo "Build static content" &&     cd webapp &&     yarn &&     lerna run bootstrap &&     lerna run build --no-bail --stream --scope=@cloudbeaver/product-default #-- -- --env source-map
#10 sha256:a71238b2312500755e7452aab32f0aa62eef5aa63d2975b787668d62cd4f36ab
#10 CACHED

#6 [frontend-builder 2/6] RUN yarn global add lerna
#6 sha256:31c2ca09d72a73a77ac4893a45660d21390abebcf59259eeb388bf7600f18c60
#6 CACHED

#11 [frontend-builder 6/6] RUN echo "Copy static content" &&     cp -rp /dbeaver/cloud/frontend/webapp/packages/product-default/lib /dbeaver/cloud/frontend/cloudbeaver-web-build
#11 sha256:06d5d76738cacf98a8677b371a32f955c6cd0ec175091c6fe783ddb9b2caf89e
#11 CACHED

#12 exporting to image
#12 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#12 exporting layers done
#12 writing image sha256:730d161d696f08e017270259662d96baad0fc5c8a332fd070a58b43748b45505 done
#12 naming to docker.io/dbeaver/cloudbeaver-frontend done
#12 DONE 0.0s
Building cloudbeaver
#1 [internal] load build definition from Dockerfile
#1 sha256:dba38602d98d8dc1bd037a5a4667b9e502ad32fa009ad062ea72c2cc09552e5c
#1 transferring dockerfile: 37B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:992c4e94f530b9ded43a346dbebd89a7139935fb74b3968145136cb19eb78c0f
#2 transferring context: 2B done
#2 DONE 0.0s

#5 [internal] load metadata for docker.io/alpine/git:latest
#5 sha256:53c8397ef7cbc2a2b64b29948591c91cdd7638c705f9e00a31c645ddbf85d4d4
#5 DONE 0.0s

#6 [internal] load metadata for docker.io/library/node:16-alpine3.15
#6 sha256:874a4c0aa0a7f216ec26c7b83c24ba14fdb01bbc1aa9262ffcf8648918805772
#6 ...

#4 [internal] load metadata for docker.io/library/maven:3.8-eclipse-temurin-11-alpine
#4 sha256:8ec9051ab2a279abdabaf2797ebaa34fbe96426bbf67b810cc35309ea00e4fc3
#4 DONE 0.2s

#6 [internal] load metadata for docker.io/library/node:16-alpine3.15
#6 sha256:874a4c0aa0a7f216ec26c7b83c24ba14fdb01bbc1aa9262ffcf8648918805772
#6 DONE 0.2s

#3 [internal] load metadata for docker.io/adoptopenjdk/openjdk11:x86_64-alpine-jre-11.0.14.1_1
#3 sha256:40d5a019c576dc0091a8ba49d832d7040492c24ec02cc8b64987929e7e51335d
#3 DONE 0.5s

#7 importing cache manifest from dbeaver/cloudbeaver-backend
#7 sha256:4f595ca88a8968210fab7ea2f6989d9719172393c434ca16acac1144d0902d8b
#7 DONE 0.0s

#8 importing cache manifest from dbeaver/cloudbeaver-frontend
#8 sha256:d7cafdc4d13b2a7cd7d44f9f65a6797b6663524b381f35d2acc28f798e45705b
#8 DONE 0.0s

#28 [frontend-builder 1/6] FROM docker.io/library/node:16-alpine3.15@sha256:2c6c59cf4d34d4f937ddfcf33bab9d8bbad8658d1b9de7b97622566a52167f2b
#28 sha256:fb7df7ff6aaf5b77fd015d60b9efd057eae205beca35245388ca834e00b00079
#28 DONE 0.0s

#15 [core-builder 1/3] FROM docker.io/alpine/git
#15 sha256:51f8b5880c93adc1c41b2c3e0ef44d59de9626d5528dd6aa0d642c23517f1198
#15 DONE 0.0s

#9 [stage-3 1/7] FROM docker.io/adoptopenjdk/openjdk11:x86_64-alpine-jre-11.0.14.1_1@sha256:514659fac8007de195a72d8ddaf06bb8b6a05a404b9a933cf641bf7c151136c2
#9 sha256:3a5ac154752125d85bf5e2435dc640aa6ec9206df085e0e756d0a39b0db48bdf
#9 DONE 0.0s

#13 [backend-builder  1/10] FROM docker.io/library/maven:3.8-eclipse-temurin-11-alpine@sha256:67ff1b5f33540c4a7cf25138b43e890dc18e4b88baedf7f0275658e4b6c70fac
#13 sha256:edb695fdbe963a99a08d116c38c18b47fb5a4460671109479cf8387cb2582ad3
#13 DONE 0.0s

#20 [internal] load build context
#20 sha256:8ab99465ae621d998daaf2ec52354bd6d349b29ad4379e0a999b1111e075a4b0
#20 transferring context: 2.27MB 0.2s done
#20 DONE 0.2s

#10 [stage-3 2/7] RUN apk update && apk add bash
#10 sha256:1cb632e5ccccd2eb771bee02adaa0dbefbe9a383540f3356cf8006054e02ae31
#10 CACHED

#11 [stage-3 3/7] WORKDIR /opt/cloudbeaver
#11 sha256:f6636a820889a778c07d42a0e0c97f8b72dccc86c628dd7139cfa5f4c172f63f
#11 CACHED

#22 [backend-builder  6/10] RUN echo "Build CloudBeaver server" &&     cd /dbeaver/cloud/backend/server-source/product/aggregate &&     mvn clean package -Dheadless-platform:
#22 sha256:8d4134cc135dab94d64ff6e97aad54a97f37cfc21bb22e5aca58c0c314dfb5e4
#22 CACHED

#34 [stage-3 6/7] COPY --from=frontend-builder /dbeaver/cloud/frontend/cloudbeaver-web-build /opt/cloudbeaver/web
#34 sha256:ed2470fc584aafc2d374d63421d2252c92d114bfc9c57277c88fd39e03dc3d27
#34 CACHED

#29 [frontend-builder 2/6] RUN yarn global add lerna
#29 sha256:31c2ca09d72a73a77ac4893a45660d21390abebcf59259eeb388bf7600f18c60
#29 CACHED

#16 [core-builder 2/3] WORKDIR /dbeaver/core
#16 sha256:81d6845562bcab73a8ae7ad3c8ce5b5f8f86f9065fae9dc5466ad139d31f4bc7
#16 CACHED

#31 [frontend-builder 4/6] COPY webapp /dbeaver/cloud/frontend/webapp
#31 sha256:a2f3638d6dbdd2d9c92d4d55767407a0bf2f389ee77f79e8c609916f4d937539
#31 CACHED

#17 [core-builder 3/3] RUN echo "Clone dbeaver platform" &&     git clone --depth=1 https://github.com/dbeaver/dbeaver.git
#17 sha256:586addf8e3e5d2189836b1105c7c4f265de2f7cfafaa1a2e72b5dc1898f5c9ac
#17 CACHED

#33 [frontend-builder 6/6] RUN echo "Copy static content" &&     cp -rp /dbeaver/cloud/frontend/webapp/packages/product-default/lib /dbeaver/cloud/frontend/cloudbeaver-web-build
#33 sha256:5464f0528f3eee032da152532d84c2aa53490c81e3953a49728463c1ff938673
#33 CACHED

#27 [stage-3 5/7] COPY --from=backend-builder /dbeaver/cloud/backend/cloudbeaver /opt/cloudbeaver
#27 sha256:a9bc12c69d54760fbe711c8f88e9d2a395afc14a7c00dcd73a208d465384bd31
#27 CACHED

#32 [frontend-builder 5/6] RUN echo "Build static content" &&     cd webapp &&     yarn &&     lerna run bootstrap &&     lerna run build --no-bail --stream --scope=@cloudbeaver/product-default #-- -- --env source-map
#32 sha256:b68503a91019a5d805678fc3ba69d191bd950ebcee7b255fbac8fd16e56212e6
#32 CACHED

#23 [backend-builder  7/10] COPY samples/sample-databases/db /dbeaver/cloud/backend/cloudbeaver/samples
#23 sha256:5185434611d30f58156f6f2b9b09c0e53f359f62027209dd7a6823aa8798229a
#23 CACHED

#26 [backend-builder 10/10] RUN echo "Copy server packages" &&    mv /dbeaver/cloud/backend/server-source/product/web-server/target/products/io.cloudbeaver.product/all/all/all /dbeaver/cloud/backend/cloudbeaver/server &&    cp /dbeaver/cloud/backend/cloudbeaver/samples/sample-databases/GlobalConfiguration/.dbeaver/data-sources.json /dbeaver/cloud/backend/cloudbeaver/conf/initial-data-sources.conf &&    cp /dbeaver/cloud/backend/cloudbeaver/samples/sample-databases/*.conf /dbeaver/cloud/backend/cloudbeaver/conf &&    mv /dbeaver/cloud/backend/deploy/drivers /dbeaver/cloud/backend/cloudbeaver/drivers
#26 sha256:682ba9c0d9e137b97ce535be128e3e7b9eb1239526c02b8a4fd4b8540aa8dae9
#26 CACHED

#12 [stage-3 4/7] RUN mkdir /opt/cloudbeaver/web
#12 sha256:72964d02ca4e5447cf74bf5d5e58222d619892c4b57ab15c7db09dc28d0c43a7
#12 CACHED

#18 [backend-builder  3/10] COPY --from=core-builder /dbeaver/core/dbeaver /dbeaver/cloud/dbeaver
#18 sha256:243f6c10c42b5a77f43eca93b11eda46b1f83571f1a65c09c9d36de0c4928acd
#18 CACHED

#19 [backend-builder  4/10] RUN echo "Build Cloudbeaver server" &&     mkdir ./cloudbeaver &&     mkdir ./cloudbeaver/conf &&     mkdir ./cloudbeaver/samples &&     mkdir ./cloudbeaver/workspace &&     mkdir ./cloudbeaver/web
#19 sha256:578f46f6a902bfaf39369b70cebf1a437e16bb2f9f9345ab5af97fdcccc7ffdb
#19 CACHED

#21 [backend-builder  5/10] COPY server /dbeaver/cloud/backend/server-source
#21 sha256:dd844d0470f7136b110fab2e9ee90d6e1c70bb102c7fe4611747529dc0e2f7f4
#21 CACHED

#24 [backend-builder  8/10] COPY deploy/scripts /dbeaver/cloud/backend/cloudbeaver/scripts
#24 sha256:0d7a63be41055f2f6fe477f8ea9f9215f9f21019f21882e6efa97eb4da6e70a0
#24 CACHED

#30 [frontend-builder 3/6] WORKDIR /dbeaver/cloud/frontend
#30 sha256:74c6a6845fe56dbf11c0e508ce0048977dfd816ec59b9f92ef4ec8fdb6a4b38a
#30 CACHED

#25 [backend-builder  9/10] COPY samples /dbeaver/cloud/backend/cloudbeaver/samples
#25 sha256:0460a10f10dbbaf396f216053cb6c8a23a333b9b605c99c07eecc2e68fd73ea1
#25 CACHED

#14 [backend-builder  2/10] WORKDIR /dbeaver/cloud/backend
#14 sha256:d4e1ddc9189a7df6e98c04acb060ee10b609f62e183746d1842fef7c6e549c25
#14 CACHED

#35 [stage-3 7/7] COPY deploy/scripts/run-server.sh /opt/cloudbeaver/run-server.sh
#35 sha256:d437c1f48045a6a2df89787aa0eb6fa10aa34203d6ceb4f4c7462278ed637987
#35 CACHED

#36 exporting to image
#36 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#36 exporting layers done
#36 writing image sha256:b0a0c4b7bf3465aba160dd0ab88236c6fe0b7e90fb6b9c537727dff9177922f8
#36 writing image sha256:b0a0c4b7bf3465aba160dd0ab88236c6fe0b7e90fb6b9c537727dff9177922f8 0.2s done
#36 naming to docker.io/dbeaver/cloudbeaver done
#36 DONE 0.2s

🔉 Run Logs

Just can see the logs if not putting the container in the background.

$ docker-compose up cloudbeaver
Recreating cloudbeaver ... done
Attaching to cloudbeaver
cloudbeaver             | Starting Cloudbeaver Server
cloudbeaver             | 2022-02-11 01:17:21.296 - Using configuration [conf/cloudbeaver.conf]
cloudbeaver             | 2022-02-11 01:17:21.420 - Load product configuration from '/opt/cloudbeaver/conf/product.conf'
cloudbeaver             | 2022-02-11 01:17:21.430 - CloudBeaver CE Server 21.3.4.202202110034 is starting
cloudbeaver             | 2022-02-11 01:17:21.430 -     OS: Linux 5.10.76-linuxkit (amd64)
cloudbeaver             | 2022-02-11 01:17:21.430 -     Java version: 11.0.14.1 by Eclipse Adoptium (64bit)
cloudbeaver             | 2022-02-11 01:17:21.431 -     Install path: '/opt/cloudbeaver/server'
cloudbeaver             | 2022-02-11 01:17:21.431 -     Global workspace: 'file:/opt/cloudbeaver/workspace/'
cloudbeaver             | 2022-02-11 01:17:21.431 -     Memory available 258Mb/4124Mb
cloudbeaver             | 2022-02-11 01:17:21.444 - Initializing product: CloudBeaver CE Server 21.3.4
cloudbeaver             | 2022-02-11 01:17:21.445 - Host plugin: io.cloudbeaver.product.ce 21.3.4.202202110034
cloudbeaver             | 2022-02-11 01:17:21.445 - Initialize web platform...
cloudbeaver             | 2022-02-11 01:17:21.595 - BounceCastle bundle found. Use JCE provider BC
cloudbeaver             | 2022-02-11 01:17:21.797 - Initialize base platform...
cloudbeaver             | 2022-02-11 01:17:21.973 - Total database drivers: 96 (96)
cloudbeaver             | 2022-02-11 01:17:21.981 - Available drivers: ClickHouse (Legacy),DB2 LUW,DB2 iSeries/AS 400,Firebird,H2 Embedded,Derby Server,Trino,MS SQL Server / SQL Server,MySQL,MariaDB,Oracle,PostgreSQL,SQLite
cloudbeaver             | 2022-02-11 01:17:21.993 - Web platform initialized (548ms)
cloudbeaver             | 2022-02-11 01:17:21.996 -     Content root: /opt/cloudbeaver/web
cloudbeaver             | 2022-02-11 01:17:21.996 -     Drivers storage: /opt/cloudbeaver/drivers
cloudbeaver             | 2022-02-11 01:17:21.996 -     Listen port: 8978 on all interfaces
cloudbeaver             | 2022-02-11 01:17:21.997 -     Base URI: /api/
cloudbeaver             | 2022-02-11 01:17:21.997 -     Production mode
cloudbeaver             | 2022-02-11 01:17:21.997 -     Server is in configuration mode!
cloudbeaver             | 2022-02-11 01:17:22.000 -     Run in Docker container (host.docker.internal/192.168.65.2)?
cloudbeaver             | 2022-02-11 01:17:22.002 -     Local host addresses:
cloudbeaver             | 2022-02-11 01:17:22.046 -             192.168.65.2 (192.168.65.2)
cloudbeaver             | 2022-02-11 01:17:22.047 -             192.168.176.2 (23870a29a7f4)
cloudbeaver             | 2022-02-11 01:17:22.047 -             127.0.0.1 (localhost)
cloudbeaver             | 2022-02-11 01:17:22.101 - Initiate management database
cloudbeaver             | 2022-02-11 01:17:22.137 -     Initiate connection pool with management database (H2 Embedded; jdbc:h2:/opt/cloudbeaver/workspace/.data/cb.h2.dat)
cloudbeaver             | 2022-02-11 01:17:22.485 -     Connected to H2 1.4.199 (2019-03-13)
cloudbeaver             | 2022-02-11 01:17:22.523 - Create new schema CB
cloudbeaver             | 2022-02-11 01:17:22.816 -     Management database connection established
cloudbeaver             | 2022-02-11 01:17:22.817 - Auth provider 'local' registered
cloudbeaver             | 2022-02-11 01:17:22.819 - No auto configuration was found. Server must be configured manually
cloudbeaver             | 2022-02-11 01:17:22.827 - Starting Jetty server (8978 on all interfaces)
cloudbeaver             | 2022-02-11 01:17:23.152 - Schema extensions loaded: WebServiceBindingCore,WebServiceBindingNavigator,WebServiceBindingSQL,WebServiceBindingAdmin,WebServiceBindingAuth,WebServiceBindingDataTransfer,WebServiceBindingMetadata
cloudbeaver             | 2022-02-11 01:17:23.468 - Active servlets:
cloudbeaver             | 2022-02-11 01:17:23.468 -     static: [/*]
cloudbeaver             | 2022-02-11 01:17:23.468 -     status: [/status]
cloudbeaver             | 2022-02-11 01:17:23.468 -     images: [/api/images/*]
cloudbeaver             | 2022-02-11 01:17:23.469 -     graphql: [/api/gql/*]
cloudbeaver             | 2022-02-11 01:17:23.469 -     sqlResultValueViewer: [/api/sql-result-value/*]
cloudbeaver             | 2022-02-11 01:17:23.469 -     adminLogs: [/api/logs/*]
cloudbeaver             | 2022-02-11 01:17:23.469 -     dataTransfer: [/api/data/*]
cloudbeaver             | 2022-02-11 01:17:23.471 - jetty-10.0.6; built: 2021-06-29T15:28:56.259Z; git: 37e7731b4b142a882d73974ff3bec78d621bd674; jvm 11.0.14.1+1
cloudbeaver             | 2022-02-11 01:17:23.523 - Session workerName=
cloudbeaver             | 2022-02-11 01:17:23.559 - Started o.e.j.s.ServletContextHandler@6f85ee02{/,file:///opt/cloudbeaver/web/,AVAILABLE}
cloudbeaver             | 2022-02-11 01:17:23.571 - Started ServerConnector@7d904ff1{HTTP/1.1, (http/1.1)}{0.0.0.0:8978}
cloudbeaver             | 2022-02-11 01:17:23.571 - Started JettyServer@1f84327b{STARTING}[10.0.6,sto=0] @4127ms

✅ Sample Testing

🔧 Setup a Database

I have added a sample using Postgres for testing... It's on the directory samples

NOTE: This must be a database on the same docker network as CloudBeaver!

docker-compose network can declare network cloudbeaver-network

Now run the following commands to start the containers

$ docker-compose up -d cloudbeaver
Creating network "cloudbeaver-network" with the default driver
Recreating cloudbeaver ... done

Then, run the sample with a regular postgres database

NOTE: The sample is also setup to use the same docker network added by the server

$ docker-compose -f samples/postgres-docker-compose.yml up -d
Starting postgres-server ... done

Making sure that CloudBeaver is running, we can verify the containers

$ docker ps
CONTAINER ID   IMAGE                 COMMAND                  CREATED          STATUS          PORTS                     NAMES
f8b2f1d35b86   postgres              "docker-entrypoint.s…"   2 minutes ago    Up 17 seconds   0.0.0.0:10101->5432/tcp   postgres-server
4d7ae73cd44f   dbeaver/cloudbeaver   "/opt/cloudbeaver/ru…"   36 minutes ago   Up 36 minutes   0.0.0.0:8978->8978/tcp    cloudbeaver

From the UI, we can add this database as follows:

db:5432
postgres-server:5432

From external, we can access them using the exposed port

localhost:10101
Verify that they are running as follows

🎨 Load on the declared port on docker-compose

🎨 Setup as usual

NOTE: this can be configured automatically by providing a config as docker volume

🎨 Add the Database

🎨 Test the Dockerized Database

This is the ultimate test in where CloudBeaver correctly verifies the connectivity to the dockerized Postgres
- Docker network creates a dns entry for both db and postgres-server as they are declared in docker-compose

TESTING WITH postgres-server

TESTING WITH db

Everything works as expected

🎛️ Saved Configuration

docker-compose is the driver of the execution and, we can use the local docker volumes for development
The tests above provided the following configuration locally to my MacOS
- This is the local volume and users can adjust to anywhere

$ cat data/workspace/GlobalConfiguration/.dbeaver/data-sources.json

{
	"folders": {},
	"connections": {
		"sqlite_xerial-sample-database": {
			"provider": "sqlite",
			"driver": "sqlite_jdbc",
			"name": "SQLite - Chinook (Sample)",
			"save-password": true,
			"read-only": false,
			"configuration": {
				"database": "${application.path}/../samples/db/Chinook.sqlitedb",
				"type": "dev",
				"auth-model": "native"
			}
		},
		"postgresql-template-1": {
			"provider": "postgresql",
			"driver": "postgres-jdbc",
			"name": "PostgreSQL (Template)",
			"save-password": false,
			"template": true,
			"read-only": true,
			"configuration": {
				"host": "localhost",
				"port": "5432",
				"database": "postgres",
				"url": "jdbc:postgresql://localhost:5432/postgres",
				"type": "dev",
				"provider-properties": {
					"@dbeaver-show-non-default-db@": "false"
				}
			}
		},
		"postgres-jdbc-17ee7375704-82a6a9f58ba5169": {
			"provider": "postgresql",
			"driver": "postgres-jdbc",
			"name": "supercash-database-dev",
			"save-password": false,
			"show-system-objects": true,
			"read-only": false,
			"configuration": {
				"host": "db",
				"port": "5432",
				"database": "supercash",
				"url": "jdbc:postgresql://db:5432/supercash",
				"type": "dev",
				"auth-model": "native"
			}
		}
	},
	"connection-types": {
		"dev": {
			"name": "Development",
			"color": "255,255,255",
			"description": "Regular development database",
			"auto-commit": true,
			"confirm-execute": false,
			"confirm-data-change": false,
			"auto-close-transactions": false
		}
	}
}

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/dbeaver/cloudbeaver/pull/649 **Author:** [@marcellodesales](https://github.com/marcellodesales) **Created:** 2/11/2022 **Status:** ❌ Closed **Base:** `devel` ← **Head:** `feature/dockerized-multi-stages-build` --- ### 📝 Commits (10+) - [`2540b77`](https://github.com/dbeaver/cloudbeaver/commit/2540b77ee66b33fae4018e58ca8f5c6ba33d72af) dbeaver/cloudbeaver#663 :recycle: :whale: Dockerfile: ultimate multi-stage builds - [`c4d57a6`](https://github.com/dbeaver/cloudbeaver/commit/c4d57a6ee8b8d2c2eb59b15b9c21b031ae6e1d86) dbeaver/cloudbeaver#663 :see_no_evil: ignoring the workspace dir when running in docker - [`05b4d36`](https://github.com/dbeaver/cloudbeaver/commit/05b4d3651ef2da35616ca56004f55dbcc2542c9f) dbeaver/cloudbeaver#663 :recycle: :lock: run cloudbeaver with different user - [`8a6e332`](https://github.com/dbeaver/cloudbeaver/commit/8a6e332ccae7dd469b9e17ca8f6aea37094ba508) dbeaver/cloudbeaver#663 :alembic: Add the postgres example to connect to dbeaver - [`67730dd`](https://github.com/dbeaver/cloudbeaver/commit/67730dd51f1a21f329816ca0ce1972c5bf325a91) dbeaver/cloudbeaver#663 :fire: Removing shell scripts that calls deploy/docker - [`405e117`](https://github.com/dbeaver/cloudbeaver/commit/405e117b2c80e66e66f0e9c878fe5f99b068ceea) dbeaver/cloudbeaver#663 :construction_worker: :whale: Add the dockerized CI/CD pipeline: docker image - [`4863d3b`](https://github.com/dbeaver/cloudbeaver/commit/4863d3b24fa001eeec9398e031ef1047d94bd062) dbeaver/cloudbeaver#663 :bulb: Add comment docker-compose.yaml for prd volumes - [`1cd104e`](https://github.com/dbeaver/cloudbeaver/commit/1cd104e5638986addf8405e5d3b785f7fb03969e) dbeaver/cloudbeaver#663 :green_heart: :whale: pipeline fix: docker login before push - [`8c4a06a`](https://github.com/dbeaver/cloudbeaver/commit/8c4a06a399c16416fbddf92cafb0e713a91b2b9e) dbeaver/cloudbeaver#663 :truck: Rename docker-compose.yml to .yaml - [`048c9f7`](https://github.com/dbeaver/cloudbeaver/commit/048c9f7cd675de59c1616f9681333908e40259b1) dbeaver/cloudbeaver#663 :recycle: :truck: rename the docker image before build/push ### 📊 Changes **10 files changed** (+391 additions, -27 deletions) <details> <summary>View changed files</summary> ➕ `.github/workflows/docker-image.yaml` (+219 -0) 📝 `.gitignore` (+2 -0) ➕ `Dockerfile` (+91 -0) ➖ `deploy/docker/.gitignore` (+0 -1) ➖ `deploy/docker/Dockerfile` (+0 -8) ➖ `deploy/docker/docker-compose.yml` (+0 -10) ➖ `deploy/docker/make-docker-container.sh` (+0 -3) ➖ `deploy/docker/run-docker-container.sh` (+0 -5) ➕ `docker-compose.yaml` (+57 -0) ➕ `samples/postgres-docker-compose.yaml` (+22 -0) </details> ### 📄 Description # 🐳 Ultimate Multi-staged Dockerized builds + Gihub's Dockerized Pipeline to Docker Hub and Github Container Registry This commit adds initial implementation of multi-stage builds that replicates the steps from deploy/build.sh. The reason why is very simple: docker multi-stage builds are parallelized and all the stages steps are eligible for cache. This is mostly done because I want to have a very quick developer experience with DBeaver in Kubernetes and the only way to get it right is to properly have a clean and light docker image that I can run in both Kubernetes or during Services Development. That is, I don't want any Engineer in my company to have to install DBeaver as a Desktop, but use this containerized version to quickly test their services... An option that I've been using is `docker-compose` for development and `Kubernetes` for Runtime in Production... The story is very simple: a UI engineer would not know how to setup the Database nor care about it, but with a 1-liner setup they have considered to use this setup... > Personal problem with current version: After running the App, we can use DB locally without using the infamous `deploy/build.sh` that depends on specific versions of required build systems, as well as requiring the users to know all the pieces... (deploy, etc) * `Goal`: A single `Dockerfile` that performs 3 builds, uses 4 stages, and coordinates the build in parallel: Get myself using Cloudbeaver in 1 command! > NOTE: I like this too so much that If this PR is merged, I will also contribute the Github CI/CD pipeline for the Docker image + the deployment scripts I use for `Kubernetes`/`Kustomize`/`ArgoCD` deployment for CloudBeaver! :D that uses this docker image 🍷 Cheers! > NOTE2: The runtime docker image size is super small `266.9 MB` ```console $ docker-compose images Container Repository Tag Image Id Size ------------------------------------------------------------------------------------------- cloudbeaver dbeaver/cloudbeaver latest b0a0c4b7bf34 266.9 MB cloudbeaver_cloudbeaver-backend_1 <none> <none> bdde1c816c2c 1.32 GB cloudbeaver_cloudbeaver-frontend_1 <none> <none> e4f86e49ab72 1.028 GB ``` # 📐 Architecture and Design > This docker image does NOT The docker build is split into 3 components: * **core: dbeaver**: Cloned as the git repo * Performance improvement: the build will ONLY PULL the HEAD commit instead of the entire history. That makes the build way faster! * This is eligiable for a Docker build Cache: for now only main branch is cloned * This layer adds nothing but the git repo contents * **backend: cloudbeaver**: the current project's server * Added all the steps as the script * Simplified and used full paths to avoid debugging confusion when reviewing build.sh * Based on a public JDK docker image * **frontend: web**: a regular Node.js build with yard and lerna * Same principal used here * Based on a node.js Lerna instructions We are providing a docker-compose as the main driver for users to properly build each individual component. The Dockerfile was built with caches from a local build. It can be further improved if `dbeaver/cloudbeaver` adds this to a pipeline (I can send a PR for that as well) and sets up the env vars with the secrets to Dockerhub. * Also, users can use versions from Github's Container Registry if that's an option (for development) * The docker image users Alpine linux, a much more secure image than the original's DBeaver verison * At least does not present the security threat described at https://github.com/dbeaver/cloudbeaver/issues/349#issuecomment-903716506 * https://avd.aquasec.com/nvd/cve-2021-33910/ does NOT list alpine linux as affected, however, you anyone can add a user as well ## 🔊 Docker build Logs * You can verify all the steps and where the cache was properly used in my 1s subsequent build ```console $ docker-compose build 2>&1 | tee output Building cloudbeaver-backend #1 [internal] load build definition from Dockerfile #1 sha256:b094ed2d72447eabf2da9d70707bdce549fed1c5f926441397466b9686e339eb #1 transferring dockerfile: 37B done #1 DONE 0.0s #2 [internal] load .dockerignore #2 sha256:c54f08aba9edbe6b41a53e95daa74a0cf69a083680acb60bbf12130475efc077 #2 transferring context: 2B done #2 DONE 0.0s #3 [internal] load metadata for docker.io/alpine/git:latest #3 sha256:53c8397ef7cbc2a2b64b29948591c91cdd7638c705f9e00a31c645ddbf85d4d4 #3 DONE 0.0s #4 [internal] load metadata for docker.io/library/maven:3.8-eclipse-temurin-11-alpine #4 sha256:8ec9051ab2a279abdabaf2797ebaa34fbe96426bbf67b810cc35309ea00e4fc3 #4 DONE 0.4s #6 importing cache manifest from dbeaver/cloudbeaver-backend #6 sha256:4f595ca88a8968210fab7ea2f6989d9719172393c434ca16acac1144d0902d8b #6 DONE 0.0s #8 [core-builder 1/3] FROM docker.io/alpine/git #8 sha256:51f8b5880c93adc1c41b2c3e0ef44d59de9626d5528dd6aa0d642c23517f1198 #8 DONE 0.0s #5 [backend-builder 1/10] FROM docker.io/library/maven:3.8-eclipse-temurin-11-alpine@sha256:67ff1b5f33540c4a7cf25138b43e890dc18e4b88baedf7f0275658e4b6c70fac #5 sha256:edb695fdbe963a99a08d116c38c18b47fb5a4460671109479cf8387cb2582ad3 #5 DONE 0.0s #13 [internal] load build context #13 sha256:1fce9ba74c48461a3d723191c5dbb537e6ebed438cfbd5511317d6e0eabf1d56 #13 transferring context: 37.36kB 0.1s done #13 DONE 0.1s #7 [backend-builder 2/10] WORKDIR /dbeaver/cloud/backend #7 sha256:d4e1ddc9189a7df6e98c04acb060ee10b609f62e183746d1842fef7c6e549c25 #7 CACHED #18 [backend-builder 9/10] COPY samples /dbeaver/cloud/backend/cloudbeaver/samples #18 sha256:e3d7452e06df82c8d931fc6cfea9a1fd0958eeeea38982fedbce50bba255185b #18 CACHED #16 [backend-builder 7/10] COPY samples/sample-databases/db /dbeaver/cloud/backend/cloudbeaver/samples #16 sha256:e71b4233be517ada6fa19bf17690a81167cd1d3ee93104d3f2e66ee1720f29f6 #16 CACHED #12 [backend-builder 4/10] RUN echo "Build Cloudbeaver server" && mkdir ./cloudbeaver && mkdir ./cloudbeaver/conf && mkdir ./cloudbeaver/samples && mkdir ./cloudbeaver/workspace && mkdir ./cloudbeaver/web #12 sha256:578f46f6a902bfaf39369b70cebf1a437e16bb2f9f9345ab5af97fdcccc7ffdb #12 CACHED #14 [backend-builder 5/10] COPY server /dbeaver/cloud/backend/server-source #14 sha256:5edc9a310cfe22e11702e599301f5f07031192ae98fa02b2a121483960e305bf #14 CACHED #15 [backend-builder 6/10] RUN echo "Build CloudBeaver server" && cd /dbeaver/cloud/backend/server-source/product/aggregate && mvn clean package -Dheadless-platform: #15 sha256:7b3bd765c25356b013642f7d8b3860e905dddd9f78b20965cdf51e734c8f9fe4 #15 CACHED #10 [core-builder 3/3] RUN echo "Clone dbeaver platform" && git clone --depth=1 https://github.com/dbeaver/dbeaver.git #10 sha256:586addf8e3e5d2189836b1105c7c4f265de2f7cfafaa1a2e72b5dc1898f5c9ac #10 CACHED #11 [backend-builder 3/10] COPY --from=core-builder /dbeaver/core/dbeaver /dbeaver/cloud/dbeaver #11 sha256:243f6c10c42b5a77f43eca93b11eda46b1f83571f1a65c09c9d36de0c4928acd #11 CACHED #17 [backend-builder 8/10] COPY deploy/scripts /dbeaver/cloud/backend/cloudbeaver/scripts #17 sha256:6d1fd0e05ff0be709ec3d7326b7d8676a69ed0081d30f092cddaa3e9a87c6f60 #17 CACHED #9 [core-builder 2/3] WORKDIR /dbeaver/core #9 sha256:81d6845562bcab73a8ae7ad3c8ce5b5f8f86f9065fae9dc5466ad139d31f4bc7 #9 CACHED #19 [backend-builder 10/10] RUN echo "Copy server packages" && mv /dbeaver/cloud/backend/server-source/product/web-server/target/products/io.cloudbeaver.product/all/all/all /dbeaver/cloud/backend/cloudbeaver/server && cp /dbeaver/cloud/backend/cloudbeaver/samples/sample-databases/GlobalConfiguration/.dbeaver/data-sources.json /dbeaver/cloud/backend/cloudbeaver/conf/initial-data-sources.conf && cp /dbeaver/cloud/backend/cloudbeaver/samples/sample-databases/*.conf /dbeaver/cloud/backend/cloudbeaver/conf && mv /dbeaver/cloud/backend/deploy/drivers /dbeaver/cloud/backend/cloudbeaver/drivers #19 sha256:3f769067327c6bf8c68ad90d192dd8eb9b712610c627e3ca79a628683f5367d1 #19 CACHED #20 exporting to image #20 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00 #20 exporting layers done #20 writing image sha256:6111c0d5a0c7fc84d073352ca3101e242a29deadba06bb71bc3434c89d49c30f done #20 naming to docker.io/dbeaver/cloudbeaver-backend done #20 DONE 0.0s Building cloudbeaver-frontend #1 [internal] load build definition from Dockerfile #1 sha256:59b91e01265e09a3154a18309cf68d6bfa855ffbce550ccf71b7c5ecca0f663d #1 transferring dockerfile: 37B done #1 DONE 0.0s #2 [internal] load .dockerignore #2 sha256:cafc13f768a3ba542bec05f6a98f3be2885cf43ef5db9be4993a633edce572b6 #2 transferring context: 2B done #2 DONE 0.0s #3 [internal] load metadata for docker.io/library/node:16-alpine3.15 #3 sha256:874a4c0aa0a7f216ec26c7b83c24ba14fdb01bbc1aa9262ffcf8648918805772 #3 DONE 0.4s #5 importing cache manifest from dbeaver/cloudbeaver-frontend #5 sha256:d7cafdc4d13b2a7cd7d44f9f65a6797b6663524b381f35d2acc28f798e45705b #5 DONE 0.0s #4 [frontend-builder 1/6] FROM docker.io/library/node:16-alpine3.15@sha256:2c6c59cf4d34d4f937ddfcf33bab9d8bbad8658d1b9de7b97622566a52167f2b #4 sha256:fb7df7ff6aaf5b77fd015d60b9efd057eae205beca35245388ca834e00b00079 #4 DONE 0.0s #8 [internal] load build context #8 sha256:da45e60a892447492457650be14aa09e4db311b7f4eba2b7b15feffcde9066b8 #8 transferring context: 9.11MB 0.4s done #8 DONE 0.4s #7 [frontend-builder 3/6] WORKDIR /dbeaver/cloud/frontend #7 sha256:74c6a6845fe56dbf11c0e508ce0048977dfd816ec59b9f92ef4ec8fdb6a4b38a #7 CACHED #9 [frontend-builder 4/6] COPY webapp /dbeaver/cloud/frontend/webapp #9 sha256:8ef53481136edc85f177353689d9ff91035a73aa995687096269bad791f64718 #9 CACHED #10 [frontend-builder 5/6] RUN echo "Build static content" && cd webapp && yarn && lerna run bootstrap && lerna run build --no-bail --stream --scope=@cloudbeaver/product-default #-- -- --env source-map #10 sha256:a71238b2312500755e7452aab32f0aa62eef5aa63d2975b787668d62cd4f36ab #10 CACHED #6 [frontend-builder 2/6] RUN yarn global add lerna #6 sha256:31c2ca09d72a73a77ac4893a45660d21390abebcf59259eeb388bf7600f18c60 #6 CACHED #11 [frontend-builder 6/6] RUN echo "Copy static content" && cp -rp /dbeaver/cloud/frontend/webapp/packages/product-default/lib /dbeaver/cloud/frontend/cloudbeaver-web-build #11 sha256:06d5d76738cacf98a8677b371a32f955c6cd0ec175091c6fe783ddb9b2caf89e #11 CACHED #12 exporting to image #12 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00 #12 exporting layers done #12 writing image sha256:730d161d696f08e017270259662d96baad0fc5c8a332fd070a58b43748b45505 done #12 naming to docker.io/dbeaver/cloudbeaver-frontend done #12 DONE 0.0s Building cloudbeaver #1 [internal] load build definition from Dockerfile #1 sha256:dba38602d98d8dc1bd037a5a4667b9e502ad32fa009ad062ea72c2cc09552e5c #1 transferring dockerfile: 37B done #1 DONE 0.0s #2 [internal] load .dockerignore #2 sha256:992c4e94f530b9ded43a346dbebd89a7139935fb74b3968145136cb19eb78c0f #2 transferring context: 2B done #2 DONE 0.0s #5 [internal] load metadata for docker.io/alpine/git:latest #5 sha256:53c8397ef7cbc2a2b64b29948591c91cdd7638c705f9e00a31c645ddbf85d4d4 #5 DONE 0.0s #6 [internal] load metadata for docker.io/library/node:16-alpine3.15 #6 sha256:874a4c0aa0a7f216ec26c7b83c24ba14fdb01bbc1aa9262ffcf8648918805772 #6 ... #4 [internal] load metadata for docker.io/library/maven:3.8-eclipse-temurin-11-alpine #4 sha256:8ec9051ab2a279abdabaf2797ebaa34fbe96426bbf67b810cc35309ea00e4fc3 #4 DONE 0.2s #6 [internal] load metadata for docker.io/library/node:16-alpine3.15 #6 sha256:874a4c0aa0a7f216ec26c7b83c24ba14fdb01bbc1aa9262ffcf8648918805772 #6 DONE 0.2s #3 [internal] load metadata for docker.io/adoptopenjdk/openjdk11:x86_64-alpine-jre-11.0.14.1_1 #3 sha256:40d5a019c576dc0091a8ba49d832d7040492c24ec02cc8b64987929e7e51335d #3 DONE 0.5s #7 importing cache manifest from dbeaver/cloudbeaver-backend #7 sha256:4f595ca88a8968210fab7ea2f6989d9719172393c434ca16acac1144d0902d8b #7 DONE 0.0s #8 importing cache manifest from dbeaver/cloudbeaver-frontend #8 sha256:d7cafdc4d13b2a7cd7d44f9f65a6797b6663524b381f35d2acc28f798e45705b #8 DONE 0.0s #28 [frontend-builder 1/6] FROM docker.io/library/node:16-alpine3.15@sha256:2c6c59cf4d34d4f937ddfcf33bab9d8bbad8658d1b9de7b97622566a52167f2b #28 sha256:fb7df7ff6aaf5b77fd015d60b9efd057eae205beca35245388ca834e00b00079 #28 DONE 0.0s #15 [core-builder 1/3] FROM docker.io/alpine/git #15 sha256:51f8b5880c93adc1c41b2c3e0ef44d59de9626d5528dd6aa0d642c23517f1198 #15 DONE 0.0s #9 [stage-3 1/7] FROM docker.io/adoptopenjdk/openjdk11:x86_64-alpine-jre-11.0.14.1_1@sha256:514659fac8007de195a72d8ddaf06bb8b6a05a404b9a933cf641bf7c151136c2 #9 sha256:3a5ac154752125d85bf5e2435dc640aa6ec9206df085e0e756d0a39b0db48bdf #9 DONE 0.0s #13 [backend-builder 1/10] FROM docker.io/library/maven:3.8-eclipse-temurin-11-alpine@sha256:67ff1b5f33540c4a7cf25138b43e890dc18e4b88baedf7f0275658e4b6c70fac #13 sha256:edb695fdbe963a99a08d116c38c18b47fb5a4460671109479cf8387cb2582ad3 #13 DONE 0.0s #20 [internal] load build context #20 sha256:8ab99465ae621d998daaf2ec52354bd6d349b29ad4379e0a999b1111e075a4b0 #20 transferring context: 2.27MB 0.2s done #20 DONE 0.2s #10 [stage-3 2/7] RUN apk update && apk add bash #10 sha256:1cb632e5ccccd2eb771bee02adaa0dbefbe9a383540f3356cf8006054e02ae31 #10 CACHED #11 [stage-3 3/7] WORKDIR /opt/cloudbeaver #11 sha256:f6636a820889a778c07d42a0e0c97f8b72dccc86c628dd7139cfa5f4c172f63f #11 CACHED #22 [backend-builder 6/10] RUN echo "Build CloudBeaver server" && cd /dbeaver/cloud/backend/server-source/product/aggregate && mvn clean package -Dheadless-platform: #22 sha256:8d4134cc135dab94d64ff6e97aad54a97f37cfc21bb22e5aca58c0c314dfb5e4 #22 CACHED #34 [stage-3 6/7] COPY --from=frontend-builder /dbeaver/cloud/frontend/cloudbeaver-web-build /opt/cloudbeaver/web #34 sha256:ed2470fc584aafc2d374d63421d2252c92d114bfc9c57277c88fd39e03dc3d27 #34 CACHED #29 [frontend-builder 2/6] RUN yarn global add lerna #29 sha256:31c2ca09d72a73a77ac4893a45660d21390abebcf59259eeb388bf7600f18c60 #29 CACHED #16 [core-builder 2/3] WORKDIR /dbeaver/core #16 sha256:81d6845562bcab73a8ae7ad3c8ce5b5f8f86f9065fae9dc5466ad139d31f4bc7 #16 CACHED #31 [frontend-builder 4/6] COPY webapp /dbeaver/cloud/frontend/webapp #31 sha256:a2f3638d6dbdd2d9c92d4d55767407a0bf2f389ee77f79e8c609916f4d937539 #31 CACHED #17 [core-builder 3/3] RUN echo "Clone dbeaver platform" && git clone --depth=1 https://github.com/dbeaver/dbeaver.git #17 sha256:586addf8e3e5d2189836b1105c7c4f265de2f7cfafaa1a2e72b5dc1898f5c9ac #17 CACHED #33 [frontend-builder 6/6] RUN echo "Copy static content" && cp -rp /dbeaver/cloud/frontend/webapp/packages/product-default/lib /dbeaver/cloud/frontend/cloudbeaver-web-build #33 sha256:5464f0528f3eee032da152532d84c2aa53490c81e3953a49728463c1ff938673 #33 CACHED #27 [stage-3 5/7] COPY --from=backend-builder /dbeaver/cloud/backend/cloudbeaver /opt/cloudbeaver #27 sha256:a9bc12c69d54760fbe711c8f88e9d2a395afc14a7c00dcd73a208d465384bd31 #27 CACHED #32 [frontend-builder 5/6] RUN echo "Build static content" && cd webapp && yarn && lerna run bootstrap && lerna run build --no-bail --stream --scope=@cloudbeaver/product-default #-- -- --env source-map #32 sha256:b68503a91019a5d805678fc3ba69d191bd950ebcee7b255fbac8fd16e56212e6 #32 CACHED #23 [backend-builder 7/10] COPY samples/sample-databases/db /dbeaver/cloud/backend/cloudbeaver/samples #23 sha256:5185434611d30f58156f6f2b9b09c0e53f359f62027209dd7a6823aa8798229a #23 CACHED #26 [backend-builder 10/10] RUN echo "Copy server packages" && mv /dbeaver/cloud/backend/server-source/product/web-server/target/products/io.cloudbeaver.product/all/all/all /dbeaver/cloud/backend/cloudbeaver/server && cp /dbeaver/cloud/backend/cloudbeaver/samples/sample-databases/GlobalConfiguration/.dbeaver/data-sources.json /dbeaver/cloud/backend/cloudbeaver/conf/initial-data-sources.conf && cp /dbeaver/cloud/backend/cloudbeaver/samples/sample-databases/*.conf /dbeaver/cloud/backend/cloudbeaver/conf && mv /dbeaver/cloud/backend/deploy/drivers /dbeaver/cloud/backend/cloudbeaver/drivers #26 sha256:682ba9c0d9e137b97ce535be128e3e7b9eb1239526c02b8a4fd4b8540aa8dae9 #26 CACHED #12 [stage-3 4/7] RUN mkdir /opt/cloudbeaver/web #12 sha256:72964d02ca4e5447cf74bf5d5e58222d619892c4b57ab15c7db09dc28d0c43a7 #12 CACHED #18 [backend-builder 3/10] COPY --from=core-builder /dbeaver/core/dbeaver /dbeaver/cloud/dbeaver #18 sha256:243f6c10c42b5a77f43eca93b11eda46b1f83571f1a65c09c9d36de0c4928acd #18 CACHED #19 [backend-builder 4/10] RUN echo "Build Cloudbeaver server" && mkdir ./cloudbeaver && mkdir ./cloudbeaver/conf && mkdir ./cloudbeaver/samples && mkdir ./cloudbeaver/workspace && mkdir ./cloudbeaver/web #19 sha256:578f46f6a902bfaf39369b70cebf1a437e16bb2f9f9345ab5af97fdcccc7ffdb #19 CACHED #21 [backend-builder 5/10] COPY server /dbeaver/cloud/backend/server-source #21 sha256:dd844d0470f7136b110fab2e9ee90d6e1c70bb102c7fe4611747529dc0e2f7f4 #21 CACHED #24 [backend-builder 8/10] COPY deploy/scripts /dbeaver/cloud/backend/cloudbeaver/scripts #24 sha256:0d7a63be41055f2f6fe477f8ea9f9215f9f21019f21882e6efa97eb4da6e70a0 #24 CACHED #30 [frontend-builder 3/6] WORKDIR /dbeaver/cloud/frontend #30 sha256:74c6a6845fe56dbf11c0e508ce0048977dfd816ec59b9f92ef4ec8fdb6a4b38a #30 CACHED #25 [backend-builder 9/10] COPY samples /dbeaver/cloud/backend/cloudbeaver/samples #25 sha256:0460a10f10dbbaf396f216053cb6c8a23a333b9b605c99c07eecc2e68fd73ea1 #25 CACHED #14 [backend-builder 2/10] WORKDIR /dbeaver/cloud/backend #14 sha256:d4e1ddc9189a7df6e98c04acb060ee10b609f62e183746d1842fef7c6e549c25 #14 CACHED #35 [stage-3 7/7] COPY deploy/scripts/run-server.sh /opt/cloudbeaver/run-server.sh #35 sha256:d437c1f48045a6a2df89787aa0eb6fa10aa34203d6ceb4f4c7462278ed637987 #35 CACHED #36 exporting to image #36 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00 #36 exporting layers done #36 writing image sha256:b0a0c4b7bf3465aba160dd0ab88236c6fe0b7e90fb6b9c537727dff9177922f8 #36 writing image sha256:b0a0c4b7bf3465aba160dd0ab88236c6fe0b7e90fb6b9c537727dff9177922f8 0.2s done #36 naming to docker.io/dbeaver/cloudbeaver done #36 DONE 0.2s ``` ## 🔉 Run Logs Just can see the logs if not putting the container in the background. ```console $ docker-compose up cloudbeaver Recreating cloudbeaver ... done Attaching to cloudbeaver cloudbeaver | Starting Cloudbeaver Server cloudbeaver | 2022-02-11 01:17:21.296 - Using configuration [conf/cloudbeaver.conf] cloudbeaver | 2022-02-11 01:17:21.420 - Load product configuration from '/opt/cloudbeaver/conf/product.conf' cloudbeaver | 2022-02-11 01:17:21.430 - CloudBeaver CE Server 21.3.4.202202110034 is starting cloudbeaver | 2022-02-11 01:17:21.430 - OS: Linux 5.10.76-linuxkit (amd64) cloudbeaver | 2022-02-11 01:17:21.430 - Java version: 11.0.14.1 by Eclipse Adoptium (64bit) cloudbeaver | 2022-02-11 01:17:21.431 - Install path: '/opt/cloudbeaver/server' cloudbeaver | 2022-02-11 01:17:21.431 - Global workspace: 'file:/opt/cloudbeaver/workspace/' cloudbeaver | 2022-02-11 01:17:21.431 - Memory available 258Mb/4124Mb cloudbeaver | 2022-02-11 01:17:21.444 - Initializing product: CloudBeaver CE Server 21.3.4 cloudbeaver | 2022-02-11 01:17:21.445 - Host plugin: io.cloudbeaver.product.ce 21.3.4.202202110034 cloudbeaver | 2022-02-11 01:17:21.445 - Initialize web platform... cloudbeaver | 2022-02-11 01:17:21.595 - BounceCastle bundle found. Use JCE provider BC cloudbeaver | 2022-02-11 01:17:21.797 - Initialize base platform... cloudbeaver | 2022-02-11 01:17:21.973 - Total database drivers: 96 (96) cloudbeaver | 2022-02-11 01:17:21.981 - Available drivers: ClickHouse (Legacy),DB2 LUW,DB2 iSeries/AS 400,Firebird,H2 Embedded,Derby Server,Trino,MS SQL Server / SQL Server,MySQL,MariaDB,Oracle,PostgreSQL,SQLite cloudbeaver | 2022-02-11 01:17:21.993 - Web platform initialized (548ms) cloudbeaver | 2022-02-11 01:17:21.996 - Content root: /opt/cloudbeaver/web cloudbeaver | 2022-02-11 01:17:21.996 - Drivers storage: /opt/cloudbeaver/drivers cloudbeaver | 2022-02-11 01:17:21.996 - Listen port: 8978 on all interfaces cloudbeaver | 2022-02-11 01:17:21.997 - Base URI: /api/ cloudbeaver | 2022-02-11 01:17:21.997 - Production mode cloudbeaver | 2022-02-11 01:17:21.997 - Server is in configuration mode! cloudbeaver | 2022-02-11 01:17:22.000 - Run in Docker container (host.docker.internal/192.168.65.2)? cloudbeaver | 2022-02-11 01:17:22.002 - Local host addresses: cloudbeaver | 2022-02-11 01:17:22.046 - 192.168.65.2 (192.168.65.2) cloudbeaver | 2022-02-11 01:17:22.047 - 192.168.176.2 (23870a29a7f4) cloudbeaver | 2022-02-11 01:17:22.047 - 127.0.0.1 (localhost) cloudbeaver | 2022-02-11 01:17:22.101 - Initiate management database cloudbeaver | 2022-02-11 01:17:22.137 - Initiate connection pool with management database (H2 Embedded; jdbc:h2:/opt/cloudbeaver/workspace/.data/cb.h2.dat) cloudbeaver | 2022-02-11 01:17:22.485 - Connected to H2 1.4.199 (2019-03-13) cloudbeaver | 2022-02-11 01:17:22.523 - Create new schema CB cloudbeaver | 2022-02-11 01:17:22.816 - Management database connection established cloudbeaver | 2022-02-11 01:17:22.817 - Auth provider 'local' registered cloudbeaver | 2022-02-11 01:17:22.819 - No auto configuration was found. Server must be configured manually cloudbeaver | 2022-02-11 01:17:22.827 - Starting Jetty server (8978 on all interfaces) cloudbeaver | 2022-02-11 01:17:23.152 - Schema extensions loaded: WebServiceBindingCore,WebServiceBindingNavigator,WebServiceBindingSQL,WebServiceBindingAdmin,WebServiceBindingAuth,WebServiceBindingDataTransfer,WebServiceBindingMetadata cloudbeaver | 2022-02-11 01:17:23.468 - Active servlets: cloudbeaver | 2022-02-11 01:17:23.468 - static: [/*] cloudbeaver | 2022-02-11 01:17:23.468 - status: [/status] cloudbeaver | 2022-02-11 01:17:23.468 - images: [/api/images/*] cloudbeaver | 2022-02-11 01:17:23.469 - graphql: [/api/gql/*] cloudbeaver | 2022-02-11 01:17:23.469 - sqlResultValueViewer: [/api/sql-result-value/*] cloudbeaver | 2022-02-11 01:17:23.469 - adminLogs: [/api/logs/*] cloudbeaver | 2022-02-11 01:17:23.469 - dataTransfer: [/api/data/*] cloudbeaver | 2022-02-11 01:17:23.471 - jetty-10.0.6; built: 2021-06-29T15:28:56.259Z; git: 37e7731b4b142a882d73974ff3bec78d621bd674; jvm 11.0.14.1+1 cloudbeaver | 2022-02-11 01:17:23.523 - Session workerName= cloudbeaver | 2022-02-11 01:17:23.559 - Started o.e.j.s.ServletContextHandler@6f85ee02{/,file:///opt/cloudbeaver/web/,AVAILABLE} cloudbeaver | 2022-02-11 01:17:23.571 - Started ServerConnector@7d904ff1{HTTP/1.1, (http/1.1)}{0.0.0.0:8978} cloudbeaver | 2022-02-11 01:17:23.571 - Started JettyServer@1f84327b{STARTING}[10.0.6,sto=0] @4127ms ``` # ✅ Sample Testing ## 🔧 Setup a Database I have added a sample using Postgres for testing... It's on the directory `samples` > NOTE: This must be a database on the same `docker network` as CloudBeaver! > * docker-compose network can declare network `cloudbeaver-network` * Now run the following commands to start the containers ```console $ docker-compose up -d cloudbeaver Creating network "cloudbeaver-network" with the default driver Recreating cloudbeaver ... done ``` * Then, run the sample with a regular postgres database > NOTE: The sample is also setup to use the same docker network added by the server ```console $ docker-compose -f samples/postgres-docker-compose.yml up -d Starting postgres-server ... done ``` Making sure that CloudBeaver is running, we can verify the containers ```console $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f8b2f1d35b86 postgres "docker-entrypoint.s…" 2 minutes ago Up 17 seconds 0.0.0.0:10101->5432/tcp postgres-server 4d7ae73cd44f dbeaver/cloudbeaver "/opt/cloudbeaver/ru…" 36 minutes ago Up 36 minutes 0.0.0.0:8978->8978/tcp cloudbeaver ``` From the UI, we can add this database as follows: * db:5432 * postgres-server:5432 From external, we can access them using the exposed port * localhost:10101 * Verify that they are running as follows ## 🎨 Load on the declared port on docker-compose <img width="1362" alt="Screen Shot 2022-02-10 at 5 40 46 PM" src="https://user-images.githubusercontent.com/131457/153531794-9d6484f6-c062-444e-80e9-5926ec657799.png"> ## 🎨 Setup as usual > NOTE: this can be configured automatically by providing a config as `docker volume` <img width="1630" alt="Screen Shot 2022-02-10 at 5 41 00 PM" src="https://user-images.githubusercontent.com/131457/153531806-045175a0-e9c6-4a1d-a170-a1f343c9a713.png"> ## 🎨 Add the Database <img width="987" alt="Screen Shot 2022-02-10 at 7 03 16 PM" src="https://user-images.githubusercontent.com/131457/153531814-e8c263d3-0829-4d96-a086-9a6703b1602d.png"> ## 🎨 Test the Dockerized Database * This is the ultimate test in where CloudBeaver correctly verifies the connectivity to the dockerized Postgres * Docker network creates a dns entry for both `db` and `postgres-server` as they are declared in docker-compose > TESTING WITH `postgres-server` <img width="1773" alt="Screen Shot 2022-02-10 at 7 04 19 PM" src="https://user-images.githubusercontent.com/131457/153531825-4d9d24bf-c964-4d49-a555-e317c3bde23c.png"> > TESTING WITH `db` <img width="1785" alt="Screen Shot 2022-02-10 at 7 11 37 PM" src="https://user-images.githubusercontent.com/131457/153532223-c96d07c8-277c-4c14-9819-57b43b8c42f5.png"> ## 🎨 Login and visualizing * Everything works as expected <img width="1449" alt="Screen Shot 2022-02-10 at 9 21 05 PM" src="https://user-images.githubusercontent.com/131457/153541472-39007bbc-13ca-4618-a4b0-cfff3c0789a9.png"> ## 🎛️ Saved Configuration * docker-compose is the driver of the execution and, we can use the local docker volumes for development * The tests above provided the following configuration locally to my MacOS * This is the local volume and users can adjust to anywhere ```console $ cat data/workspace/GlobalConfiguration/.dbeaver/data-sources.json ``` ```json { "folders": {}, "connections": { "sqlite_xerial-sample-database": { "provider": "sqlite", "driver": "sqlite_jdbc", "name": "SQLite - Chinook (Sample)", "save-password": true, "read-only": false, "configuration": { "database": "${application.path}/../samples/db/Chinook.sqlitedb", "type": "dev", "auth-model": "native" } }, "postgresql-template-1": { "provider": "postgresql", "driver": "postgres-jdbc", "name": "PostgreSQL (Template)", "save-password": false, "template": true, "read-only": true, "configuration": { "host": "localhost", "port": "5432", "database": "postgres", "url": "jdbc:postgresql://localhost:5432/postgres", "type": "dev", "provider-properties": { "@dbeaver-show-non-default-db@": "false" } } }, "postgres-jdbc-17ee7375704-82a6a9f58ba5169": { "provider": "postgresql", "driver": "postgres-jdbc", "name": "supercash-database-dev", "save-password": false, "show-system-objects": true, "read-only": false, "configuration": { "host": "db", "port": "5432", "database": "supercash", "url": "jdbc:postgresql://db:5432/supercash", "type": "dev", "auth-model": "native" } } }, "connection-types": { "dev": { "name": "Development", "color": "255,255,255", "description": "Regular development database", "auto-commit": true, "confirm-execute": false, "confirm-data-change": false, "auto-close-transactions": false } } } ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>