들어가기에 앞서
인증서를 사용하지 않거나, 임시로 만들어진 사설 인증서를 이용해서 Container Registry를 만들면 Image를 Pull, Push할 때 Insecure registry를 등록해 줘야 합니다. 등록 시 Docker daemon(container runtime daemon)을 재시작 시켜줘야 하는데, 이 때 이미 만들어진 container들이 모두 재시작 됩니다. 이 문제를 해소하기 위해 Container Registry를 만들 때 사설 인증서에 계획된 CN(Common Names)과 SAN(Subject Alternative Names)을 등록하여 Insecure registry 등록하지 않고 사용하는 법을 알아보도록 하죠.
아래 예시에서는 Sonatype Nexus Repository를 이용하였습니다.
1. Dockerfile 준비
root@yj4-ussuri-focal-deploy:~/dockerfiles# cat Dockerfile
FROM sonatype/nexus3:3.32.0
MAINTAINER Cyyoon <cyyoon@osci.kr>, youngju LEE <yjlee@linux.com>
USER root
RUN chown -R nexus:nexus ${NEXUS_HOME}/etc \
&& sed '/^application-port/s:$:\napplication-port-ssl=8443:' -i ${NEXUS_HOME}/etc/nexus-default.properties \
&& sed '/^nexus-args/s:$:,${jetty.etc}/jetty-https.xml:' -i ${NEXUS_HOME}/etc/nexus-default.properties \
&& rm -rf ${NEXUS_HOME}/etc/ssl && ln -s ${NEXUS_DATA}/etc/ssl ${NEXUS_HOME}/etc/ssl
COPY start.sh /usr/local/bin
USER nexus
EXPOSE 8443
CMD start.sh
- Nexus Repository를 처음 시작 할 때 인증서를 만들도록 script를 짜두었는데 해당 script는 Cyyoon(https://github.com/ycy1766)님이 예전에 만들어 둔 것을 조금 수정하였습니다.
2. start.sh script 준비
root@yj4-ussuri-focal-deploy:~/dockerfiles# cat start.sh
#! /bin/bash
if [ ! -e "$NEXUS_DATA/etc/ssl/keystore.jks" ]; then
mkdir -p "$NEXUS_DATA/etc/ssl"
chmod go-rwx "$NEXUS_DATA/etc/ssl"
keytool -genkeypair -keystore $NEXUS_DATA/etc/ssl/keystore.jks -storepass password -keypass password \
-alias jetty -keyalg RSA -keysize 2048 -validity 5000 \
-dname "CN=*.${HOSTNAME}, OU=Cloud, O=OSC, L=Gangnam, ST=Seoul, C=KR" \
-ext "SAN=DNS:${SAN_DNS}" -ext "BC=ca:true"
fi
sh -c ${SONATYPE_DIR}/start-nexus-repository-manager.sh- OU, 조직, 위치 등은 맞게 바꾸면 됩니다.
- SAN_DNS에 사용 할 SAN을 넣어줍니다.
3. Container image build
root@yj4-ussuri-focal-deploy:~/dockerfiles# chmod +x start.sh
root@yj4-ussuri-focal-deploy:~/dockerfiles# ls -l
total 8
-rw-r--r-- 1 root root 538 Apr 5 16:20 Dockerfile
-rwxr-xr-x 1 root root 514 Apr 5 16:41 start.sh
root@yj4-ussuri-focal-deploy:~/dockerfiles# docker build -t osc-nexus3:3.32.0 .
Sending build context to Docker daemon 4.096kB
Step 1/8 : FROM sonatype/nexus3:3.32.0
3.32.0: Pulling from sonatype/nexus3
5f6bf015319e: Pull complete
45cc8b7f2b43: Pull complete
e114d7ce4c88: Pull complete
1c7c876dc5dd: Pull complete
5b5ac368a09b: Pull complete
Digest: sha256:4b73d33797727349adb7dff50da9c8eb17298706b481a00b330c589b8a893f36
Status: Downloaded newer image for sonatype/nexus3:3.32.0
---> 0ebd6cc0ce56
Step 2/8 : MAINTAINER Cyyoon <cyyoon@osci.kr>, youngju LEE <yjlee@linux.com>
---> Running in e3fe8b3010a9
Removing intermediate container e3fe8b3010a9
---> bfd8438df420
Step 3/8 : USER root
---> Running in 97cf15ee8ed3
Removing intermediate container 97cf15ee8ed3
---> 4d13e3a44b52
Step 4/8 : RUN chown -R nexus:nexus ${NEXUS_HOME}/etc && sed '/^application-port/s:$:\napplication-port-ssl=8443:' -i ${NEXUS_HOME}/etc/nexus-default.properties && sed '/^nexus-args/s:$:,${jetty.etc}/jetty-https.xml:' -i ${NEXUS_HOME}/etc/nexus-default.properties && rm -rf ${NEXUS_HOME}/etc/ssl && ln -s ${NEXUS_DATA}/etc/ssl ${NEXUS_HOME}/etc/ssl
---> Running in ef4b125d0717
Removing intermediate container ef4b125d0717
---> d159dc9ac5de
Step 5/8 : COPY start.sh /usr/local/bin
---> b1960373b363
Step 6/8 : USER nexus
---> Running in 0fba1bfb241a
Removing intermediate container 0fba1bfb241a
---> b0877e44c0b1
Step 7/8 : EXPOSE 8443
---> Running in 304859e57f02
Removing intermediate container 304859e57f02
---> a74e4953e9db
Step 8/8 : CMD start.sh
---> Running in 66b6adc9cf7f
Removing intermediate container 66b6adc9cf7f
---> 2f0b5996c599
Successfully built 2f0b5996c599
Successfully tagged osc-nexus3:3.32.0
root@yj4-ussuri-focal-deploy:~/dockerfiles#
4. Container 시작
컨테이너 이미지 빌드가 완료되었으면 컨테이너를 시작하도록 합니다.
root@yj4-ussuri-focal-deploy:~/dockerfiles# mkdir /data/
root@yj4-ussuri-focal-deploy:~/dockerfiles# chown 200:200 -R /data/
root@yj4-ussuri-focal-deploy:~/dockerfiles# docker run -d --net=host -e SAN_DNS=nexus.local --name nexus -v /data:/nexus-data osc-nexus3:3.32.0
12f26029cd606117f7e79b5bcf5f87608a7688c25f96342e311d568478d59acd
root@yj4-ussuri-focal-deploy:~/dockerfiles# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
12f26029cd60 osc-nexus3:3.32.0 "/bin/sh -c start.sh" 4 seconds ago Up 3 seconds nexus
5. Service 확인
등록된 서비스를 확인하고 정상적으로 웹 접속이 가능한지 확인합니다.
root@yj4-ussuri-focal-deploy:~/dockerfiles# docker logs -f nexus
...
2022-04-05 07:51:42,510+0000 INFO [jetty-main-1] *SYSTEM org.eclipse.jetty.server.Server - Started @53848ms
2022-04-05 07:51:42,511+0000 INFO [jetty-main-1] *SYSTEM org.sonatype.nexus.bootstrap.jetty.JettyServer -
-------------------------------------------------
Started Sonatype Nexus OSS 3.32.0-03
-------------------------------------------------
root@yj4-ussuri-focal-deploy:~/dockerfiles# netstat -ntpl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:8443 0.0.0.0:* LISTEN 65806/java
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/init
tcp 0 0 0.0.0.0:8081 0.0.0.0:* LISTEN 65806/java
tcp 0 0 127.0.0.1:33429 0.0.0.0:* LISTEN 65806/java
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 640/systemd-resolve
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 714/sshd: /usr/sbin
tcp6 0 0 :::111 :::* LISTEN 1/init
tcp6 0 0 :::22 :::* LISTEN 714/sshd: /usr/sbin
- 초기 Dashboard Login admin password는 /nexus-data/admin.password에 있습니다.
root@yj4-ussuri-focal-deploy:~/dockerfiles# docker exec nexus cat /nexus-data/admin.password
44b4ec3c-748a-4864-8f97-6f48c7e0dab1root@yj4-ussuri-focal-deploy:~/dockerfiles#
- admin / 44b4ec3c-748a-4864-8f97-6f48c7e0dab1
6. 인증서 Export
keytool을 활용하여 인증서를 생성합니다.
root@yj4-ussuri-focal-deploy:~/dockerfiles# docker exec nexus keytool -printcert -sslserver 127.0.0.1:8443 -rfc | tee nexus.crt
-----BEGIN CERTIFICATE-----
MIIDpzCCAo+gAwIBAgIECefZAjANBgkqhkiG9w0BAQsFADBxMQswCQYDVQQGEwJL
UjEOMAwGA1UECBMFU2VvdWwxEDAOBgNVBAcTB0dhbmduYW0xDDAKBgNVBAoTA09T
QzEOMAwGA1UECxMFQ2xvdWQxIjAgBgNVBAMMGSoueWo0LXVzc3VyaS1mb2NhbC1k
ZXBsb3kwHhcNMjIwNDA1MDc1MDQ3WhcNMzUxMjEzMDc1MDQ3WjBxMQswCQYDVQQG
EwJLUjEOMAwGA1UECBMFU2VvdWwxEDAOBgNVBAcTB0dhbmduYW0xDDAKBgNVBAoT
A09TQzEOMAwGA1UECxMFQ2xvdWQxIjAgBgNVBAMMGSoueWo0LXVzc3VyaS1mb2Nh
bC1kZXBsb3kwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCRtiHcuO4b
NCpHeWsTolUniY7DXSzbRH6SogR8arwm9HhuV1vSTDn8LBbjhPxs7o3PjJ3loEY7
t9KZ6NpIe8to+kbbC9iIhmVsKep/zDmhWg9EaiaRyh81HYwwSdLcKieOmAl18Rhw
iAY6zZAp/6CT4ls9XDSbZSRpHTkeH+S6RcfT2bwXefZp0BHe2zKfbh3fuGpuaMis
FGTueasrmhJAvivawGYS2piCfz0p6Bh3Ov4ZdnD28RuAk7X3aa8dugfXrSzRt/Oy
mKZHMxL63wfI1pDRKBZaH14+tIeMlYQVKm360ZBHhpsirGxPO4q1Hq1OL9ZMiElD
YgX8oFqD6dA3AgMBAAGjRzBFMAwGA1UdEwQFMAMBAf8wFgYDVR0RBA8wDYILbmV4
dXMubG9jYWwwHQYDVR0OBBYEFFyGloYh5zZAklDHfH/BLeAoJQyaMA0GCSqGSIb3
DQEBCwUAA4IBAQAzDK7HUvXVUYNc8KHcYUus/WnQaVptBLI2FKpk8Wdl9KdxTne3
Yb5acn/wN2WzXcoq7bJNM9RmsvX945riBtAa4TFndZTXFb1lq9xosOmJx93haH1P
yJ0wFvmncyPfpKoP8clgvdFhuaJgrwrqAz2koxVr2xLtIxkW3exKypBSMADgjRa/
3myoZCN82IryBjP0iX5L1Mqkkb2pGyXodnOga+vRVbaLQaqPvClb22rG1J7+uIOd
xsC3GWWyX8dcPf1Hr6evVKATeUylf4WXDAXqbAU8Ez8vjnm5SaILnnwIP0HjaB8f
tLLOW1jWZibfItnWEePdaeer7DNxDoOPONdO
-----END CERTIFICATE-----
- 아래의 명령어를 통해 인증서 확인 작업을 수행합니다.
root@yj4-ussuri-focal-deploy:~/dockerfiles# openssl x509 -in nexus.crt -text |grep -i ' cn\|dns\|before' -2
Serial Number: 166189314 (0x9e7d902)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = KR, ST = Seoul, L = Gangnam, O = OSC, OU = Cloud, CN = *.yj4-ussuri-focal-deploy
Validity
Not Before: Apr 5 07:50:47 2022 GMT
Not After : Dec 13 07:50:47 2035 GMT
Subject: C = KR, ST = Seoul, L = Gangnam, O = OSC, OU = Cloud, CN = *.yj4-ussuri-focal-deploy
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
--
CA:TRUE
X509v3 Subject Alternative Name:
DNS:nexus.local
X509v3 Subject Key Identifier:
5C:86:96:86:21:E7:36:40:92:50:C7:7C:7F:C1:2D:E0:28:25:0C:9A
- Subject Alternative Name이 nexus.local로 되어 있는것 확인할 수 있습니다.
7. 인증서 Import
해당 인증서를 import하기 위해 아래의 작업을 수행합니다.
root@yj4-ussuri-focal-deploy:~/dockerfiles# cp -av nexus.crt /usr/local/share/ca-certificates/nexus.crt
'nexus.crt' -> '/usr/local/share/ca-certificates/nexus.crt'
root@yj4-ussuri-focal-deploy:~/dockerfiles# update-ca-certificates
Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
8. Nexus Container Registry 생성.
아래의 그림처럼 새로운 컨테이너 리포지토리를 생성하고 realm을 활성화 작업을 진행합니다.
Docker Bearer Token Realm 활성화
9. Docker Login
Docker로 로그인을 수행합니다.
root@yj4-ussuri-focal-deploy:~/dockerfiles# grep -i nexus.local /etc/hosts
10.4.0.3 nexus.local
root@yj4-ussuri-focal-deploy:~/dockerfiles# docker login nexus.local:5443 -u admin -p admin123
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
- 만약 아래와 같은 인증서 오류가 발생 한다면 다음과 같이 인증서를 지정해줍니다.
root@yj4-ussuri-focal-deploy:~/dockerfiles# docker login nexus.local:5443 -u admin -p admin123
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https://nexus.local:5443/v2/: x509: certificate signed by unknown authority
root@yj4-ussuri-focal-deploy:~/dockerfiles# mkdir /etc/docker/certs.d/nexus.local\:5443/ -p
root@yj4-ussuri-focal-deploy:~/dockerfiles# cp -av nexus.crt /etc/docker/certs.d/nexus.local\:5443/ca.crt
'nexus.crt' -> '/etc/docker/certs.d/nexus.local:5443/ca.crt'
root@yj4-ussuri-focal-deploy:~/dockerfiles# docker login nexus.local:5443 -u admin -p admin123
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
10. Container Image push test
컨테이너 이미지 push를 테스트하여 서비스가 정상 작동하는 지 확인합니다.
root@yj4-ussuri-focal-deploy:~/dockerfiles# docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
554879bb3004: Pull complete
Digest: sha256:caa382c432891547782ce7140fb3b7304613d3b0438834dce1cad68896ab110a
Status: Downloaded newer image for busybox:latest
docker.io/library/busybox:latest
root@yj4-ussuri-focal-deploy:~/dockerfiles# docker tag busybox:latest nexus.local:5443/busybox:latest
root@yj4-ussuri-focal-deploy:~/dockerfiles# docker push nexus.local:5443/busybox:latest
The push refers to repository [nexus.local:5443/busybox]
797ac4999b67: Pushed
latest: digest: sha256:14d4f50961544fdb669075c442509f194bdc4c0e344bde06e35dbd55af842a38 size: 527위 모든 것이 정상 작동하는 것을 확인하였으면 좋겠습니다. 만약 정상 작동이 되지 않는다면 해당 내용을 코멘트로 남겨주시면 정정할 수 있도록 하겠습니다.
오픈소스컨설팅의 클라우드 전문 엔지니어입니다. 꾸준히 공부하고 힐링을 위해 바이크 타는 것을 좋아합니다. 라이딩 한 번 같이 하실래요?
Recent Post
Featured Videos
