docs: improve deployment docs in kubernetes. (#2973)

* docs: improve deployment docs in kubernetes.

* move docs path.

* format contents.

* update contents.

* build: update deployment env.

* docs: update deploy docs.

* build: add kafka secret and dependencies.

* docs: update deployment docs.

* Update docs contents.

* update docs contents.

deployments/Readme.md

@@ -1,3 +1,188 @@
-# OpenIM Application Containerization Deployment Guide
+# Kubernetes Deployment
 
-view deploy [README](./deploy/README.md)
+## Resource Requests
+
+- CPU: 2 cores
+- Memory: 4 GiB
+- Disk usage: 20 GiB (on Node)
+
+## Preconditions
+
+ensure that you have already deployed the following components:
+
+- Redis
+- MongoDB
+- Kafka
+- MinIO
+
+## Origin Deploy
+
+### Enter the target dir
+
+`cd ./deployments/deploy/`
+
+### Deploy configs and dependencies
+
+Upate your configMap `openim-config.yml`. **You can check the official docs for more details.**
+
+In `openim-config.yml`, you need modify the following configurations:
+
+**discovery.yml**
+
+- `kubernetes.namespace`: default is `default`, you can change it to your namespace.
+
+**mongodb.yml**
+
+- `address`: set to your already mongodb address or mongo Service name and port in your deployed.
+- `database`: set to your mongodb database name.(Need have a created database.)
+- `authSource`: set to your mongodb authSource. (authSource is specify the database name associated with the user's credentials, user need create in this database.)
+
+**kafka.yml**
+
+- `address`: set to your already kafka address or kafka Service name and port in your deployed.
+
+**redis.yml**
+
+- `address`: set to your already redis address or redis Service name and port in your deployed.
+
+**minio.yml**
+
+- `internalAddress`: set to your minio Service name and port in your deployed.
+- `externalAddress`: set to your already expose minio external address.
+
+### Set the secret
+
+A Secret is an object that contains a small amount of sensitive data. Such as password and secret. Secret is similar to ConfigMaps.
+
+#### Redis:
+
+Update the `redis-password` value in `redis-secret.yml` to your Redis password encoded in base64.
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: openim-redis-secret
+type: Opaque
+data:
+  redis-password: b3BlbklNMTIz # update to your redis password encoded in base64, if need empty, you can set to ""
+```
+
+#### Mongo:
+
+Update the `mongo_openim_username`, `mongo_openim_password` value in `mongo-secret.yml` to your Mongo username and password encoded in base64.
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: openim-mongo-secret
+type: Opaque
+data:
+  mongo_openim_username: b3BlbklN # update to your mongo username encoded in base64, if need empty, you can set to "" (this user credentials need in authSource database).
+  mongo_openim_password: b3BlbklNMTIz # update to your mongo password encoded in base64, if need empty, you can set to ""
+```
+
+#### Minio:
+
+Update the `minio-root-user` and `minio-root-password` value in `minio-secret.yml` to your MinIO accessKeyID and secretAccessKey encoded in base64.
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: openim-minio-secret
+type: Opaque
+data:
+  minio-root-user: cm9vdA== # update to your minio accessKeyID encoded in base64, if need empty, you can set to ""
+  minio-root-password: b3BlbklNMTIz # update to your minio secretAccessKey encoded in base64, if need empty, you can set to ""
+```
+
+#### Kafka:
+
+Update the `kafka-password` value in `kafka-secret.yml` to your Kafka password encoded in base64.
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: openim-kafka-secret
+type: Opaque
+data:
+  kafka-password: b3BlbklNMTIz # update to your kafka password encoded in base64, if need empty, you can set to ""
+```
+
+### Apply the secret.
+
+```shell
+kubectl apply -f redis-secret.yml -f minio-secret.yml -f mongo-secret.yml -f kafka-secret.yml
+```
+
+### Apply all config
+
+`kubectl apply -f ./openim-config.yml`
+
+> Attation: If you use `default` namespace, you can excute `clusterRile.yml` to create a cluster role binding for default service account.
+>
+> Namespace is modify to `discovery.yml` in `openim-config.yml`, you can change `kubernetes.namespace` to your namespace.
+
+**Excute `clusterRole.yml`**
+
+`kubectl apply -f ./clusterRole.yml`
+
+### run all deployments and services
+
+> Note: Ensure that infrastructure services like MinIO, Redis, and Kafka are running before deploying the main applications.
+
+```bash
+kubectl apply \
+  -f openim-api-deployment.yml \
+  -f openim-api-service.yml \
+  -f openim-crontask-deployment.yml \
+  -f openim-rpc-user-deployment.yml \
+  -f openim-rpc-user-service.yml \
+  -f openim-msggateway-deployment.yml \
+  -f openim-msggateway-service.yml \
+  -f openim-push-deployment.yml \
+  -f openim-push-service.yml \
+  -f openim-msgtransfer-service.yml \
+  -f openim-msgtransfer-deployment.yml \
+  -f openim-rpc-conversation-deployment.yml \
+  -f openim-rpc-conversation-service.yml \
+  -f openim-rpc-auth-deployment.yml \
+  -f openim-rpc-auth-service.yml \
+  -f openim-rpc-group-deployment.yml \
+  -f openim-rpc-group-service.yml \
+  -f openim-rpc-friend-deployment.yml \
+  -f openim-rpc-friend-service.yml \
+  -f openim-rpc-msg-deployment.yml \
+  -f openim-rpc-msg-service.yml \
+  -f openim-rpc-third-deployment.yml \
+  -f openim-rpc-third-service.yml
+```
+
+### Verification
+
+After deploying the services, verify that everything is running smoothly:
+
+```bash
+# Check the status of all pods
+kubectl get pods
+
+# Check the status of services
+kubectl get svc
+
+# Check the status of deployments
+kubectl get deployments
+
+# View all resources
+kubectl get all
+```
+
+### clean all
+
+`kubectl delete -f ./`
+
+### Notes:
+
+- If you use a specific namespace for your deployment, be sure to append the -n <namespace> flag to your kubectl commands.

deployments/deploy/README.md

@@ -1,85 +0,0 @@
-# Kubernetes Deployment
-
-## Resource Requests
-
-- CPU: 2 cores
-- Memory: 4 GiB
-- Disk usage: 20 GiB (on Node)
-
-## Origin Deploy
-
-1. Enter the target dir
-   `cd ./deployments/deploy/`
-
-2. Deploy configs and dependencies
-   Upate your `openim-config.yml`
-
-Apply all config and dependencies
-`kubectl apply -f ./openim-config.yml`
-
-> Attation: If you use `default` namespace, you can excute `clusterRile.yml` to create a cluster role binding for default service account.
->
-> Namespace is modify to `discovery.yml` in `openim-config.yml`, you can change `kubernetes.namespace` to your namespace.
-
-Excute `clusterRole.yml`
-`kubectl apply -f ./clusterRole.yml`
-
-Run infrasturcture components.
-
-`kubectl apply -f minio-service.yml -f minio-statefulset.yml -f mongo-service.yml -f mongo-statefulset.yml -f redis-service.yml -f redis-statefulset.yml -f kafka-service.yml -f kafka-statefulset.yml`
-
-> Note: Ensure that infrastructure services like MinIO, Redis, and Kafka are running before deploying the main applications.
-
-3. run all deployments and services
-
-```bash
-kubectl apply \
-  -f openim-api-deployment.yml \
-  -f openim-api-service.yml \
-  -f openim-crontask-deployment.yml \
-  -f openim-rpc-user-deployment.yml \
-  -f openim-rpc-user-service.yml \
-  -f openim-msggateway-deployment.yml \
-  -f openim-msggateway-service.yml \
-  -f openim-push-deployment.yml \
-  -f openim-push-service.yml \
-  -f openim-msgtransfer-service.yml \
-  -f openim-msgtransfer-deployment.yml \
-  -f openim-rpc-conversation-deployment.yml \
-  -f openim-rpc-conversation-service.yml \
-  -f openim-rpc-auth-deployment.yml \
-  -f openim-rpc-auth-service.yml \
-  -f openim-rpc-group-deployment.yml \
-  -f openim-rpc-group-service.yml \
-  -f openim-rpc-friend-deployment.yml \
-  -f openim-rpc-friend-service.yml \
-  -f openim-rpc-msg-deployment.yml \
-  -f openim-rpc-msg-service.yml \
-  -f openim-rpc-third-deployment.yml \
-  -f openim-rpc-third-service.yml
-```
-
-4. Verification
-   After deploying the services, verify that everything is running smoothly:
-
-```bash
-# Check the status of all pods
-kubectl get pods
-
-# Check the status of services
-kubectl get svc
-
-# Check the status of deployments
-kubectl get deployments
-
-# View all resources
-kubectl get all
-```
-
-5. clean all
-
-`kubectl delete -f ./`
-
-### Notes:
-
-- If you use a specific namespace for your deployment, be sure to append the -n <namespace> flag to your kubectl commands.

deployments/deploy/kafka-secret.yml

@@ -0,0 +1,7 @@
+apiVersion: v1
+kind: Secret
+metadata:
+  name: openim-kafka-secret
+type: Opaque
+data:
+  kafka-password: ""

deployments/deploy/minio-secret.yml

@@ -0,0 +1,8 @@
+apiVersion: v1
+kind: Secret
+metadata:
+  name: openim-minio-secret
+type: Opaque
+data:
+  minio-root-user: cm9vdA== # Base64 encoded "root"
+  minio-root-password: b3BlbklNMTIz # Base64 encoded "openIM123"

deployments/deploy/minio-statefulset.yml

@@ -31,12 +31,12 @@ spec:
             - name: MINIO_ROOT_USER
               valueFrom:
                 secretKeyRef:
-                  name: minio-secret
+                  name: openim-minio-secret
                   key: minio-root-user
             - name: MINIO_ROOT_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: minio-secret
+                  name: openim-minio-secret
                   key: minio-root-password
           command:
             - "/bin/sh"
@@ -76,12 +76,4 @@ spec:
     requests:
       storage: 2Gi
 
----
+
-apiVersion: v1
-kind: Secret
-metadata:
-  name: minio-secret
-type: Opaque
-data:
-  minio-root-user: cm9vdA== # Base64 encoded "root"
-  minio-root-password: b3BlbklNMTIz # Base64 encoded "openIM123"

deployments/deploy/mongo-secret.yml

@@ -0,0 +1,8 @@
+apiVersion: v1
+kind: Secret
+metadata:
+  name: openim-mongo-secret
+type: Opaque
+data:
+  mongo_openim_username: b3BlbklN # base64 for "openIM", this user credentials need in authSource database.
+  mongo_openim_password: b3BlbklNMTIz # base64 for "openIM123"

deployments/deploy/mongo-statefulset.yml

@@ -47,27 +47,27 @@ spec:
             - name: MONGO_INITDB_ROOT_USERNAME
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-init-secret
                   key: mongo_initdb_root_username
             - name: MONGO_INITDB_ROOT_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-init-secret
                   key: mongo_initdb_root_password
             - name: MONGO_INITDB_DATABASE
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-init-secret
                   key: mongo_initdb_database
             - name: MONGO_OPENIM_USERNAME
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-init-secret
                   key: mongo_openim_username
             - name: MONGO_OPENIM_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-init-secret
                   key: mongo_openim_password
             - name: TZ
               value: "Asia/Shanghai"
@@ -93,3 +93,16 @@ spec:
   resources:
     requests:
       storage: 5Gi
+
+---
+apiVersion: v1
+kind: Secret
+metadata:
+  name: openim-mongo-init-secret
+type: Opaque
+data:
+  mongo_initdb_root_username: cm9vdA== # base64 for "root"
+  mongo_initdb_root_password: b3BlbklNMTIz # base64 for "openIM123"
+  mongo_initdb_database: b3BlbmltX3Yz # base64 for "openim_v3"
+  mongo_openim_username: b3BlbklN # base64 for "openIM"
+  mongo_openim_password: b3BlbklNMTIz # base64 for "openIM123"

deployments/deploy/openim-api-deployment.yml

@@ -21,20 +21,18 @@ spec:
             - name: IMENV_REDIS_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: redis-secret
+                  name: openim-redis-secret
                   key: redis-password
-
-            - name: IMENV_MONGODB_PASSWORD
-              valueFrom:
-                secretKeyRef:
-                  name: mongo-secret
-                  key: mongo_openim_password
-
             - name: IMENV_MONGODB_USERNAME
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-secret
                   key: mongo_openim_username
+            - name: IMENV_MONGODB_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: openim-mongo-secret
+                  key: mongo_openim_password
 
           volumeMounts:
             - name: openim-config

deployments/deploy/openim-config.yml

@@ -4,7 +4,7 @@ metadata:
   name: openim-config
 data:
   discovery.yml: |
-    enable: "kubernetes"
+    enable: "kubernetes" # "kubernetes" or "etcd"
     kubernetes:
       namespace: default
     etcd:
@@ -26,7 +26,6 @@ data:
 
   log.yml: |
     # Log storage path, default is acceptable, change to a full path if modification is needed
-    # storageLocation: ../../../../logs/
     storageLocation: ./logs/
     # Log rotation period (in hours), default is acceptable
     rotationTime: 24
@@ -49,9 +48,9 @@ data:
     # Name of the database
     database: openim_v3
     # Username for database authentication
-    username: openIM
+    username: ''  # openIM
     # Password for database authentication
-    password: openIM123
+    password: '' # openIM123
     # Authentication source for database authentication, if use root user, set it to admin
     authSource: openim_v3
     # Maximum number of connections in the connection pool
@@ -1055,16 +1054,3 @@ data:
           - targets: [ internal_ip:12320 ]
             labels:
               namespace: default
-
----
-apiVersion: v1
-kind: Secret
-metadata:
-  name: mongo-secret
-type: Opaque
-data:
-  mongo_initdb_root_username: cm9vdA== # base64 for "root"
-  mongo_initdb_root_password: b3BlbklNMTIz # base64 for "openIM123"
-  mongo_initdb_database: b3BlbmltX3Yz # base64 for "openim_v3"
-  mongo_openim_username: b3BlbklN # base64 for "openIM"
-  mongo_openim_password: b3BlbklNMTIz # base64 for "openIM123"

deployments/deploy/openim-msggateway-deployment.yml

@@ -21,7 +21,7 @@ spec:
             - name: IMENV_REDIS_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: redis-secret
+                  name: openim-redis-secret
                   key: redis-password
           volumeMounts:
             - name: openim-config

deployments/deploy/openim-msgtransfer-deployment.yml

@@ -21,13 +21,23 @@ spec:
             - name: IMENV_REDIS_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: redis-secret
+                  name: openim-redis-secret
                   key: redis-password
+            - name: IMENV_MONGODB_USERNAME
+              valueFrom:
+                secretKeyRef:
+                  name: openim-mongo-secret
+                  key: mongo_openim_username
             - name: IMENV_MONGODB_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-secret
                   key: mongo_openim_password
+            - name: IMENV_KAFKA_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: openim-kafka-secret
+                  key: kafka-password
           volumeMounts:
             - name: openim-config
               mountPath: "/config"

deployments/deploy/openim-push-deployment.yml

@@ -21,8 +21,13 @@ spec:
             - name: IMENV_REDIS_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: redis-secret
+                  name: openim-redis-secret
                   key: redis-password
+            - name: IMENV_KAFKA_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: openim-kafka-secret
+                  key: kafka-password
           volumeMounts:
             - name: openim-config
               mountPath: "/config"

deployments/deploy/openim-rpc-auth-deployment.yml

@@ -22,7 +22,7 @@ spec:
             - name: IMENV_REDIS_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: redis-secret
+                  name: openim-redis-secret
                   key: redis-password
           volumeMounts:
             - name: openim-config

deployments/deploy/openim-rpc-conversation-deployment.yml

@@ -21,12 +21,17 @@ spec:
             - name: IMENV_REDIS_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: redis-secret
+                  name: openim-redis-secret
                   key: redis-password
+            - name: IMENV_MONGODB_USERNAME
+              valueFrom:
+                secretKeyRef:
+                  name: openim-mongo-secret
+                  key: mongo_openim_username
             - name: IMENV_MONGODB_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-secret
                   key: mongo_openim_password
           volumeMounts:
             - name: openim-config

deployments/deploy/openim-rpc-friend-deployment.yml

@@ -21,12 +21,17 @@ spec:
             - name: IMENV_REDIS_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: redis-secret
+                  name: openim-redis-secret
                   key: redis-password
+            - name: IMENV_MONGODB_USERNAME
+              valueFrom:
+                secretKeyRef:
+                  name: openim-mongo-secret
+                  key: mongo_openim_username
             - name: IMENV_MONGODB_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-secret
                   key: mongo_openim_password
           volumeMounts:
             - name: openim-config

deployments/deploy/openim-rpc-group-deployment.yml

@@ -15,19 +15,23 @@ spec:
       containers:
         - name: group-rpc-server-container
           image: openim/openim-rpc-group:v3.8.3
-          
           env:
             - name: CONFIG_PATH
               value: "/config"
             - name: IMENV_REDIS_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: redis-secret
+                  name: openim-redis-secret
                   key: redis-password
+            - name: IMENV_MONGODB_USERNAME
+              valueFrom:
+                secretKeyRef:
+                  name: openim-mongo-secret
+                  key: mongo_openim_username
             - name: IMENV_MONGODB_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-secret
                   key: mongo_openim_password
           volumeMounts:
             - name: openim-config

deployments/deploy/openim-rpc-msg-deployment.yml

@@ -21,13 +21,23 @@ spec:
             - name: IMENV_REDIS_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: redis-secret
+                  name: openim-redis-secret
                   key: redis-password
+            - name: IMENV_MONGODB_USERNAME
+              valueFrom:
+                secretKeyRef:
+                  name: openim-mongo-secret
+                  key: mongo_openim_username
             - name: IMENV_MONGODB_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-secret
                   key: mongo_openim_password
+            - name: IMENV_KAFKA_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: openim-kafka-secret
+                  key: kafka-password
           volumeMounts:
             - name: openim-config
               mountPath: "/config"

deployments/deploy/openim-rpc-third-deployment.yml

@@ -21,22 +21,27 @@ spec:
             - name: IMENV_MINIO_ACCESSKEYID
               valueFrom:
                 secretKeyRef:
-                  name: minio-secret
+                  name: openim-minio-secret
                   key: minio-root-user
             - name: IMENV_MINIO_SECRETACCESSKEY
               valueFrom:
                 secretKeyRef:
-                  name: minio-secret
+                  name: openim-minio-secret
                   key: minio-root-password
             - name: IMENV_REDIS_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: redis-secret
+                  name: openim-redis-secret
                   key: redis-password
+            - name: IMENV_MONGODB_USERNAME
+              valueFrom:
+                secretKeyRef:
+                  name: openim-mongo-secret
+                  key: mongo_openim_username
             - name: IMENV_MONGODB_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-secret
                   key: mongo_openim_password
           volumeMounts:
             - name: openim-config

deployments/deploy/openim-rpc-user-deployment.yml

@@ -21,13 +21,23 @@ spec:
             - name: IMENV_REDIS_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: redis-secret
+                  name: openim-redis-secret
                   key: redis-password
+            - name: IMENV_MONGODB_USERNAME
+              valueFrom:
+                secretKeyRef:
+                  name: openim-mongo-secret
+                  key: mongo_openim_username
             - name: IMENV_MONGODB_PASSWORD
               valueFrom:
                 secretKeyRef:
-                  name: mongo-secret
+                  name: openim-mongo-secret
                   key: mongo_openim_password
+            - name: IMENV_KAFKA_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: openim-kafka-secret
+                  key: kafka-password
           volumeMounts:
             - name: openim-config
               mountPath: "/config"

deployments/deploy/redis-secret.yml

@@ -0,0 +1,7 @@
+apiVersion: v1
+kind: Secret
+metadata:
+  name: openim-redis-secret
+type: Opaque
+data:
+  redis-password: b3BlbklNMTIz # "openIM123" in base64

deployments/deploy/redis-statefulset.yml

@@ -29,9 +29,6 @@ spec:
           volumeMounts:
             - name: redis-data
               mountPath: /data
-            # - name: redis-config-volume
-            #   mountPath: /usr/local/redis/config/redis.conf
-            #   subPath: redis.conf
           command:
             [
               "/bin/sh",
@@ -56,11 +53,3 @@ spec:
   resources:
     requests:
       storage: 5Gi
----
-apiVersion: v1
-kind: Secret
-metadata:
-  name: redis-secret
-type: Opaque
-data:
-  redis-password: b3BlbklNMTIz # "openIM123" in base64