blackteay/open-im-server
1
0
Fork 0
mirror of https://github.com/openimsdk/open-im-server.git synced 2026-06-23 18:14:08 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #9 from sok-im/feature/virgil_v2

Browse Source 
virgil 加密解密
This commit is contained in:
haoyunlt 2026-04-16 16:15:06 +08:00 committed by GitHub
commit 3d95f70689
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
19 changed files with 2282 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
cmd/openim-rpc/openim-rpc-crypto/main.go Normal file
View File

@ -0,0 +1,12 @@
package main
import (
"github.com/openimsdk/open-im-server/v3/pkg/common/cmd"
"github.com/openimsdk/tools/system/program"
)
func main() {
if err := cmd.NewCryptoRpcCmd().Exec(); err != nil {
program.ExitWithError(err)
}
}

14
config/openim-rpc-crypto.yml Normal file
View File

@ -0,0 +1,14 @@
rpc:
registerIP: ''
listenIP: 0.0.0.0
autoSetPorts: true
ports:
- 10190
prometheus:
enable: true
ports:
- 20190
virgil:
appID: ''
appKey: ''
appKeyID: ''

1
config/share.yml
View File

@ -11,6 +11,7 @@ rpcRegisterName:
third: third
captcha: captcha
rtc: rtc
crypto: crypto
imAdminUserID: [ imAdmin ]

501
docs/virgil-e2ee-single-group-minimal-design.md Normal file
View File

@ -0,0 +1,501 @@
# 基于 Virgil Security E3Kit 的单聊 & 群聊最小落地方案
## 一、核心设计原则
| 原则 | 说明 |
|---|---|
| **服务端零知情** | 服务端只接触密文、元数据与业务控制，永远不触碰明文、私钥 |
| **客户端加解密** | 所有加密/解密/签名/验签均在客户端完成，使用 Virgil E3Kit SDK |
| **Virgil Cloud 托管公钥** | 用户公钥（Virgil Card）存储在 Virgil Cloud，服务端不保存 |
| **JWT 桥接认证** | 服务端用 Virgil App Key 签发 Virgil JWT，客户端持 JWT 与 Virgil Cloud 交互 |
## 二、整体架构图
```mermaid
flowchart TB
subgraph Client["客户端 (iOS/Android/Web)"]
C1["E3Kit SDK"]
C2["IM SDK"]
C3["本地密钥存储"]
end
subgraph Server["OpenIM 服务端"]
S1["API Gateway"]
S2["Auth Service"]
S3["CryptoService\n(新增 gRPC)"]
S4["Msg Service"]
S5["Group Service"]
S6["Push Service"]
S7["Sync / MsgTransfer"]
DB["MongoDB / Redis"]
end
subgraph Virgil["Virgil Cloud"]
V1["Cards Service\n(公钥目录)"]
V2["Keyknox\n(密钥备份)"]
V3["Group Tickets\n(群密钥票据)"]
end
C2 -->|"WebSocket / HTTP\n(密文 envelope)"| S1
S1 --> S2
S1 --> S3
S1 --> S4
S1 --> S5
S4 --> S7
S7 --> DB
S4 --> S6
C1 -->|"Virgil JWT"| V1
C1 -->|"密钥备份/恢复"| V2
C1 -->|"群票据同步"| V3
S3 -->|"签发 JWT\n(Virgil App Key)"| C1
S5 -->|"群事件通知"| S6
```
**图意说明：**
1. 客户端持有 E3Kit SDK（负责加解密）和 IM SDK（负责业务通信），私钥存储在设备本地。
2. 服务端新增 `CryptoService` gRPC 服务，核心职责是签发 Virgil JWT 和管理群密钥版本号。
3. Virgil Cloud 承担公钥目录、密钥备份、群加密票据的托管。
4. 消息流：客户端加密 -> 密文 envelope 经 WebSocket/HTTP 到服务端 -> 服务端存储密文+路由 -> 接收端拉取密文 -> 客户端解密。
5. 服务端全程不接触明文，只处理密文 bytes 和元数据。
## 三、单聊方案
### 3.1 单聊加密模型
单聊使用 E3Kit 的 **Default Encryption**（最小 MVP）或 **Double Ratchet**（增强版，提供前向保密）。
MVP 阶段推荐 Default Encryption：
| 特性 | Default Encryption | Double Ratchet |
|---|---|---|
| 前向保密 | 否 | 是 |
| 实现复杂度 | 低 | 中 |
| 平台支持 | JS/Swift/Kotlin | Swift/Kotlin（JS 暂不支持） |
| 适用场景 | MVP 快速落地 | 安全性要求高的正式版 |
### 3.2 单聊时序图
```mermaid
sequenceDiagram
participant A as Alice (客户端)
participant S as OpenIM Server
participant CS as CryptoService
participant VC as Virgil Cloud
participant B as Bob (客户端)
Note over A,B: === 阶段 1: 初始化 (首次登录) ===
A->>S: POST /auth/login
S-->>A: access_token
A->>CS: RegisterDevice(userID, deviceID)
CS-->>A: DeviceInfo
A->>CS: GetVirgilJWT(userID, deviceID)
CS-->>A: virgil_jwt
A->>VC: E3Kit.init(virgilJWT) -> register()
VC-->>A: 生成密钥对, 发布公钥 Card
Note over A,B: === 阶段 2: Alice 给 Bob 发加密消息 ===
A->>VC: findUsers(["bob"])
VC-->>A: Bob 的公钥 Card
A->>A: E3Kit.authEncrypt("Hello", bobCard)
Note right of A: 用 Bob 公钥加密 + Alice 私钥签名
A->>S: SendMsg(密文 envelope)
Note right of A: MsgData.content = ciphertext bytes<br/>MsgData.contentType = E2EE_TEXT<br/>MsgData.ex = {"cipher_suite":"ed25519/aes256-gcm"}
S->>S: 校验身份/会话/幂等<br/>分配 serverMsgID + seq<br/>存储密文到 MongoDB
S-->>A: SendMsgResp(serverMsgID, seq)
S->>B: WebSocket push: message.new
Note right of S: 推送不含明文
Note over A,B: === 阶段 3: Bob 拉取并解密 ===
B->>S: PullMessageBySeqs(seq)
S-->>B: MsgData(密文 envelope)
B->>VC: findUsers(["alice"])
VC-->>B: Alice 的公钥 Card
B->>B: E3Kit.authDecrypt(ciphertext, aliceCard)
Note right of B: 用 Bob 私钥解密 + Alice 公钥验签
B->>B: 显示明文 "Hello"
```
**关键说明：**
- 边界条件：`findUsers` 结果应在客户端缓存，避免每条消息都查询 Virgil Cloud。
- 异常路径：若 Bob 的 Card 不存在（未注册 E3Kit），消息无法加密，客户端应提示“对方尚未启用加密”。
- 幂等性：消息幂等键 = `sendID + deviceID + clientMsgID`，服务端去重。
- 性能：E3Kit 加密单条消息主要是本地计算，通常瓶颈在网络链路而非加密。
### 3.3 单聊消息 Envelope 结构
消息体复用 OpenIM 现有的 `sdkws.MsgData`，加密信息通过现有字段承载：
```protobuf
message MsgData {
string sendID = 1;
string recvID = 2;
string clientMsgID = 4;
int32 sessionType = 9; // 1=单聊
int32 contentType = 11; // 新增: 2001=E2EE_TEXT, 2002=E2EE_IMAGE, ...
bytes content = 12; // 密文 ciphertext (E3Kit 加密输出)
string ex = 23; // JSON: {"envelope_version":1, "cipher_suite":"ed25519/aes256-gcm"}
// ... 其余字段不变
}
```
不需要修改 proto 定义，只需约定 `contentType` 新值和 `ex` 字段的 JSON schema。
## 四、群聊方案
### 4.1 群聊加密模型
群聊使用 E3Kit 的 **Group Encryption**：
- 群主创建群时通过 `E3Kit.createGroup(groupId, members)` 生成群共享密钥票据。
- 票据存储在 Virgil Cloud，群成员通过 `E3Kit.loadGroup(groupId, ownerCard)` 加载。
- 新成员加入后通过 `group.add(newMemberCard)` 获得访问历史消息的能力。
- 成员移除后通过 `group.remove(memberCard)` 撤销访问权限，群密钥自动轮转。
### 4.2 群聊时序图 — 建群与首条消息
```mermaid
sequenceDiagram
participant Owner as 群主 Alice
participant S as OpenIM Server
participant GS as Group Service
participant CS as CryptoService
participant VC as Virgil Cloud
participant M1 as 成员 Bob
participant M2 as 成员 Carol
Note over Owner,M2: === 阶段 1: 创建群 ===
Owner->>GS: CreateGroup({members:[Bob,Carol]})
GS->>GS: 创建群记录, group_key_version=1
GS-->>Owner: {groupID, group_key_version:1}
Owner->>VC: findUsers(["bob","carol"])
VC-->>Owner: Bob & Carol 的 Cards
Owner->>VC: E3Kit.createGroup(groupID, [bobCard, carolCard])
Note right of Owner: 生成群共享密钥<br/>票据上传 Virgil Cloud
VC-->>Owner: Group 对象
GS->>M1: 推送 group.created 通知
GS->>M2: 推送 group.created 通知
Note over Owner,M2: === 阶段 2: 成员加载群密钥 ===
M1->>VC: findUsers(["alice"])
VC-->>M1: Alice (Owner) 的 Card
M1->>VC: E3Kit.loadGroup(groupID, aliceCard)
VC-->>M1: Group 对象 (本地缓存)
M2->>VC: E3Kit.loadGroup(groupID, aliceCard)
VC-->>M2: Group 对象
Note over Owner,M2: === 阶段 3: Alice 发送群加密消息 ===
Owner->>Owner: group.encrypt("大家好!")
Owner->>S: SendMsg(密文 envelope, groupID, group_key_version=1)
S->>S: 校验 Alice 是群成员<br/>group_key_version 合法<br/>存储密文, 分配 seq
S-->>Owner: SendMsgResp
S->>M1: WebSocket push
S->>M2: WebSocket push
M1->>S: PullMessageBySeqs
S-->>M1: MsgData(密文)
M1->>VC: findUsers(["alice"])
M1->>M1: group.decrypt(ciphertext, aliceCard)
M1->>M1: 显示 "大家好!"
```
### 4.3 群成员变更与密钥轮转时序图
```mermaid
sequenceDiagram
participant Owner as 群主 Alice
participant S as OpenIM Server
participant GS as Group Service
participant CS as CryptoService
participant VC as Virgil Cloud
participant New as 新成员 Dan
participant M1 as 成员 Bob
Note over Owner,M1: === 加人场景 ===
Owner->>GS: InviteUserToGroup(groupID, [Dan])
GS->>GS: 添加 Dan 到群成员
GS->>CS: BumpGroupKeyVersion(groupID, "member_added")
CS-->>GS: {group_key_version: 2}
GS-->>Owner: {group_key_version: 2}
Owner->>VC: findUsers(["dan"])
VC-->>Owner: Dan 的 Card
Owner->>VC: group.add(danCard)
Note right of Owner: Virgil Cloud 更新群票据<br/>Dan 可解密历史消息
GS->>New: 推送 group.member_changed
GS->>M1: 推送 group.member_changed (含新 version)
New->>VC: E3Kit.loadGroup(groupID, ownerCard)
VC-->>New: Group 对象
M1->>VC: group.update()
Note right of M1: 拉取最新群票据
Note over Owner,M1: === 踢人场景 ===
Owner->>GS: KickGroupMember(groupID, [Dan])
GS->>GS: 移除 Dan
GS->>CS: BumpGroupKeyVersion(groupID, "member_removed")
CS-->>GS: {group_key_version: 3}
Owner->>VC: group.remove(danCard)
Note right of Owner: Dan 无法再 loadGroup<br/>后续消息 Dan 无法解密
GS->>M1: 推送 group.member_changed
M1->>VC: group.update()
```
**关键说明：**
- 群密钥版本：每次成员变更，服务端 `group_key_version` +1，客户端据此判断是否需要 `group.update()`。
- 加人：新成员可解密加入前的历史消息（E3Kit Group 设计）。
- 踢人：被踢成员无法解密踢出后的新消息，但仍可解密踢出前已获取的消息（E2EE 的固有限制）。
- 并发：多个管理员同时操作成员时，`BumpGroupKeyVersion` 使用数据库原子递增保证版本一致。
- 性能：`group.update()` 涉及一次 Virgil Cloud 请求，建议客户端在收到 `group.member_changed` 通知后异步执行。
## 五、服务端接口清单
### 5.1 CryptoService（新增 gRPC 服务）
基于已有 `protocol/crypto/crypto.proto` 定义：
| RPC 接口 | 请求 | 响应 | 职责说明 |
|---|---|---|---|
| `RegisterDevice` | `RegisterDeviceReq` | `RegisterDeviceResp` | 注册设备，建立 `userID -> deviceID -> virgilIdentity` 映射 |
| `GetDevices` | `GetDevicesReq` | `GetDevicesResp` | 查询用户所有已注册设备 |
| `RevokeDevice` | `RevokeDeviceReq` | `RevokeDeviceResp` | 吊销设备，标记为 inactive |
| `GetVirgilJWT` | `GetVirgilJWTReq` | `GetVirgilJWTResp` | 为合法设备签发 Virgil JWT（核心接口） |
| `GetGroupKeyVersion` | `GetGroupKeyVersionReq` | `GetGroupKeyVersionResp` | 查询群当前密钥版本号 |
| `BumpGroupKeyVersion` | `BumpGroupKeyVersionReq` | `BumpGroupKeyVersionResp` | 群成员变更时递增密钥版本（Group Service 内部调用） |
| `GetGroupKeyEvents` | `GetGroupKeyEventsReq` | `GetGroupKeyEventsResp` | 查询密钥版本变更历史（客户端增量同步） |
| `SecurityPrecheck` | `SecurityPrecheckReq` | `SecurityPrecheckResp` | 安全前置校验（设备状态/风控） |
| `IntegrityReport` | `IntegrityReportReq` | `IntegrityReportResp` | 设备完整性上报 |
### 5.2 现有服务需要的改动
#### Auth Service
| 改动点 | 说明 |
|---|---|
| 登录响应增加字段 | 在 `ex` 或扩展字段中返回 `e2ee_enabled: true`，提示客户端初始化 E3Kit |
#### Msg Service
| 改动点 | 说明 |
|---|---|
| `SendMsg` 校验逻辑 | 当 `contentType` 位于 E2EE 区间时，跳过明文内容校验，仅校验 ciphertext 长度上限 |
| 消息存储 | `content` 字段直接存储密文 bytes，沿用现有存储路径 |
| 推送通知 | 推送 payload 中不携带 `content`，仅携带 `conversationID`、`senderNickname`、占位提示 |
#### Group Service
| 改动点 | 说明 |
|---|---|
| `CreateGroup` | 创建群时初始化 `group_key_version = 1` |
| `InviteUserToGroup` | 成功后调用 `CryptoService.BumpGroupKeyVersion(eventType="member_added")` |
| `KickGroupMember` | 成功后调用 `CryptoService.BumpGroupKeyVersion(eventType="member_removed")` |
| `QuitGroup` | 成功后调用 `CryptoService.BumpGroupKeyVersion(eventType="member_left")` |
| 通知 payload | `group.member_changed` 通知中携带最新 `group_key_version` |
### 5.3 服务端 HTTP API（Gateway 暴露）
```text
POST /api/v1/crypto/device/register -> CryptoService.RegisterDevice
GET /api/v1/crypto/devices -> CryptoService.GetDevices
POST /api/v1/crypto/device/revoke -> CryptoService.RevokeDevice
POST /api/v1/crypto/virgil-jwt -> CryptoService.GetVirgilJWT
GET /api/v1/crypto/group-key-version -> CryptoService.GetGroupKeyVersion
POST /api/v1/crypto/group-key-version/bump -> CryptoService.BumpGroupKeyVersion
GET /api/v1/crypto/group-key-events -> CryptoService.GetGroupKeyEvents
POST /api/v1/crypto/security-precheck -> CryptoService.SecurityPrecheck
POST /api/v1/crypto/integrity-report -> CryptoService.IntegrityReport
```
## 六、客户端接口清单
### 6.1 E3Kit 封装层接口
| 接口 | 输入 | 输出 | 说明 |
|---|---|---|---|
| `initialize(tokenCallback)` | JWT 获取回调 | void | 初始化 E3Kit，设置 JWT 刷新回调 |
| `register()` | - | void | 首次注册：生成密钥对，发布 Virgil Card |
| `restorePrivateKey(password)` | 备份密码 | void | 从 Virgil Keyknox 恢复私钥（换设备场景） |
| `backupPrivateKey(password)` | 备份密码 | void | 备份私钥到 Virgil Keyknox |
| `findUsers(userIDs)` | 用户 ID 列表 | Map<ID, Card> | 批量查找用户公钥，结果缓存 |
| `cleanup()` | - | void | 登出时清理本地私钥 |
| `rotatePrivateKey()` | - | void | 私钥泄露时轮换密钥对 |
### 6.2 单聊加解密接口
| 接口 | 输入 | 输出 | 说明 |
|---|---|---|---|
| `encryptForUser(plaintext, recipientCard)` | 明文 + 接收者 Card | 密文 string | 用接收者公钥加密 + 发送者私钥签名 |
| `decryptFromUser(ciphertext, senderCard)` | 密文 + 发送者 Card | 明文 string | 用本地私钥解密 + 发送者公钥验签 |
| `encryptFileForUser(inputStream, recipientCard)` | 文件流 + Card | 加密流 | 大文件加密 |
| `decryptFileFromUser(inputStream, senderCard)` | 加密流 + Card | 明文流 | 大文件解密 |
### 6.3 群聊加解密接口
| 接口 | 输入 | 输出 | 说明 |
|---|---|---|---|
| `createGroup(groupID, memberCards)` | 群 ID + 成员 Cards | Group 对象 | 群主创建群加密上下文 |
| `loadGroup(groupID, ownerCard)` | 群 ID + 群主 Card | Group 对象 | 非群主加载群加密上下文 |
| `getGroup(groupID)` | 群 ID | Group / null | 从本地缓存获取群对象 |
| `updateGroup(groupID)` | 群 ID | void | 拉取最新群票据（成员变更后调用） |
| `addGroupMember(groupID, newMemberCard)` | 群 ID + 新成员 Card | void | 群主添加成员到加密上下文 |
| `removeGroupMember(groupID, memberCard)` | 群 ID + 成员 Card | void | 群主从加密上下文移除成员 |
| `deleteGroup(groupID)` | 群 ID | void | 群主删除群加密上下文 |
| `encryptForGroup(plaintext, group)` | 明文 + Group | 密文 string | 群消息加密 |
| `decryptFromGroup(ciphertext, senderCard, group)` | 密文 + 发送者 Card + Group | 明文 string | 群消息解密 + 验签 |
### 6.4 IM SDK 业务层接口
| 接口 | 说明 |
|---|---|
| `requestVirgilJWT()` | 调用服务端 `/crypto/virgil-jwt`，获取并缓存 Virgil JWT |
| `registerDevice()` | 调用服务端 `/crypto/device/register` |
| `sendEncryptedMessage(conversationID, plaintext)` | 加密 -> 构造 `MsgData`(E2EE contentType) -> `SendMsg` |
| `onReceiveEncryptedMessage(msgData)` | 判断 contentType -> 查找 senderCard -> 解密 -> 回调 UI |
| `onGroupMemberChanged(groupID, newVersion)` | 收到通知后调用 `updateGroup()` 刷新群票据 |
| `syncGroupKeyVersion(groupID)` | 调用服务端 `/crypto/group-key-version`，对比本地版本决定是否更新 |
## 七、数据模型（服务端新增表）
```mermaid
erDiagram
DEVICE ||--|| USER : belongs_to
DEVICE {
string device_id PK
string user_id FK
string platform
string device_model
string app_version
string virgil_identity
string status "active / revoked"
int64 last_seen_at
int64 create_time
}
GROUP_KEY_VERSION ||--|| GROUP : tracks
GROUP_KEY_VERSION {
string group_id PK
int64 group_key_version "原子递增"
}
GROUP_KEY_EVENT }o--|| GROUP : belongs_to
GROUP_KEY_EVENT {
string event_id PK
string group_id FK
int64 group_key_version
string event_type "member_added/removed/left"
string operator_user_id
int64 create_time
}
```
## 八、contentType 约定
复用 OpenIM 现有 `contentType` 编码空间，为 E2EE 消息分配新区间：
| contentType | 名称 | 说明 |
|---|---|---|
| 2001 | `E2EE_TEXT` | 端到端加密文本 |
| 2002 | `E2EE_IMAGE` | 端到端加密图片（密文 content + 加密缩略图） |
| 2003 | `E2EE_VIDEO` | 端到端加密视频 |
| 2004 | `E2EE_FILE` | 端到端加密文件 |
| 2005 | `E2EE_AUDIO` | 端到端加密语音 |
| 2006 | `E2EE_LOCATION` | 端到端加密位置 |
| 2099 | `E2EE_CUSTOM` | 端到端加密自定义消息 |
`ex` 字段 JSON schema：
```json
{
"envelope_version": 1,
"cipher_suite": "ed25519/aes256-gcm",
"group_key_version": 2,
"sender_device_id": "ios_a1"
}
```
## 九、实施路线（分两个阶段）
### 阶段 1：单聊 MVP（约 2-3 周）
```text
服务端:
├── 实现 CryptoService gRPC (internal/rpc/crypto/)
│ ├── RegisterDevice / GetDevices / RevokeDevice
│ ├── GetVirgilJWT (核心: 用 Virgil App Key 签发)
│ └── SecurityPrecheck
├── API Gateway 新增 /crypto/* 路由
├── Msg Service: E2EE contentType 跳过明文校验
└── Push Service: E2EE 消息推送不含 content
客户端:
├── 集成 E3Kit SDK
├── 实现 E2EEManager (initialize/register/findUsers)
├── 实现单聊 encryptForUser / decryptFromUser
├── IM SDK 封装 sendEncryptedMessage / onReceiveEncryptedMessage
└── UI: 加密消息标识 (锁图标)
```
### 阶段 2：群聊（约 2-3 周）
```text
服务端:
├── CryptoService 补充: GetGroupKeyVersion / BumpGroupKeyVersion / GetGroupKeyEvents
├── Group Service 联动: 成员变更时 BumpGroupKeyVersion
├── GROUP_KEY_VERSION / GROUP_KEY_EVENT 表
└── 通知 payload 携带 group_key_version
客户端:
├── E2EEManager 补充群聊接口 (createGroup/loadGroup/addMember/removeMember)
├── 实现 encryptForGroup / decryptFromGroup
├── onGroupMemberChanged -> group.update()
└── 群聊 UI: 显示加密状态 / 密钥版本
```
## 十、安全风险与缓解
| 风险 | 影响 | 缓解措施 |
|---|---|---|
| 服务端日志泄露明文 | 破坏 E2EE 边界 | 服务端 `content` 字段日志脱敏，E2EE 类型消息禁止打印 `content` |
| 被吊销设备仍获取 JWT | 安全失控 | `GetVirgilJWT` 必须校验设备 `status=active` |
| 推送携带明文 | 绕过加密 | Push payload 仅含 `conversationID` + 占位提示 |
| 群成员变更后未更新群票据 | 用旧密钥加密 | 客户端发送前 `syncGroupKeyVersion`，版本不一致先 `update` |
| 私钥丢失 | 无法解密历史 | 引导用户 `backupPrivateKey`，换设备时 `restorePrivateKey` |
## 参考资料
- [Virgil Security Documentation](https://developer.virgilsecurity.com/)
- [E3Kit Quickstart](https://developer.virgilsecurity.com/docs/e3kit/get-started/quickstart)
- [Generate Client Tokens](https://developer.virgilsecurity.com/docs/e3kit/get-started/generate-client-tokens)
- [User Authentication](https://developer.virgilsecurity.com/docs/e3kit/user-authentication/)
- [Group Encryption](https://developer.virgilsecurity.com/docs/e3kit/end-to-end-encryption/group-chat)
- [Double Ratchet Encryption](https://developer.virgilsecurity.com/docs/e3kit/end-to-end-encryption/double-ratchet)

3
go.mod
View File

@ -64,6 +64,8 @@ require (
cloud.google.com/go/longrunning v0.5.5 // indirect
cloud.google.com/go/storage v1.40.0 // indirect
github.com/MicahParks/keyfunc v1.9.0 // indirect
github.com/VirgilSecurity/virgil-crypto-go v0.0.0-20180221191626-33caf95f9a5d // indirect
github.com/VirgilSecurity/virgil-sdk-go v5.2.1+incompatible // indirect
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible // indirect
github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
github.com/aws/aws-sdk-go-v2 v1.32.5 // indirect
@ -253,6 +255,7 @@ require (
google.golang.org/genproto/googleapis/api v0.0.0-20260209200024-4cfbd4190f57 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260209200024-4cfbd4190f57 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/virgil.v5 v5.2.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gorm.io/gorm v1.25.8 // indirect
k8s.io/api v0.31.2 // indirect

6
go.sum
View File

@ -35,6 +35,10 @@ github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA
github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 h1:TngWCqHvy9oXAN6lEVMRuU21PR1EtLVZJmdB18Gu3Rw=
github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5/go.mod h1:lmUJ/7eu/Q8D7ML55dXQrVaamCz2vxCfdQBasLZfHKk=
github.com/QcloudApi/qcloud_sign_golang v0.0.0-20141224014652-e4130a326409/go.mod h1:1pk82RBxDY/JZnPQrtqHlUFfCctgdorsd9M06fMynOM=
github.com/VirgilSecurity/virgil-crypto-go v0.0.0-20180221191626-33caf95f9a5d h1:ElVLTQRuo+LvdhsvybRwBTXvDCjMyB0Dv4mhOPnjQUQ=
github.com/VirgilSecurity/virgil-crypto-go v0.0.0-20180221191626-33caf95f9a5d/go.mod h1:zyDDPi7Ihhd5JdTYQCcdmzACnF824PYV6E6UELQiZ1w=
github.com/VirgilSecurity/virgil-sdk-go v5.2.1+incompatible h1:icWPcnsM0eqDs3pNxglM/3FbuF0Y9WUygpRM4PdBbec=
github.com/VirgilSecurity/virgil-sdk-go v5.2.1+incompatible/go.mod h1:8kxwYsqg97YNwiVCrte1fqbP6H9VJ2vjSuyj1p1CP/8=
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible h1:8psS8a+wKfiLt1iVDX79F7Y6wUM49Lcha2FMXt4UM8g=
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible/go.mod h1:T/Aws4fEfogEE9v+HPhhw+CntffsBHJ8nXQCwKr0/g8=
github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ=
@ -823,6 +827,8 @@ gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/virgil.v5 v5.2.1 h1:8NnvRXg66qC6C4uqVhuMEfm8wInUGC+QG2vdbMaCbUI=
gopkg.in/virgil.v5 v5.2.1/go.mod h1:h9WN4Q+gzfXIJDa4kpGXzWo68Wuyte51oQejiVSNzwM=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=

47
internal/api/crypto.go Normal file
View File

@ -0,0 +1,47 @@
package api
import (
"github.com/gin-gonic/gin"
"github.com/openimsdk/protocol/crypto"
"github.com/openimsdk/tools/a2r"
)
type CryptoApi struct {
Client crypto.CryptoServiceClient
}
func NewCryptoApi(client crypto.CryptoServiceClient) CryptoApi {
return CryptoApi{Client: client}
}
func (o *CryptoApi) RegisterDevice(c *gin.Context) {
a2r.Call(c, crypto.CryptoServiceClient.RegisterDevice, o.Client)
}
func (o *CryptoApi) GetDevices(c *gin.Context) {
a2r.Call(c, crypto.CryptoServiceClient.GetDevices, o.Client)
}
func (o *CryptoApi) RevokeDevice(c *gin.Context) {
a2r.Call(c, crypto.CryptoServiceClient.RevokeDevice, o.Client)
}
func (o *CryptoApi) GetVirgilJWT(c *gin.Context) {
a2r.Call(c, crypto.CryptoServiceClient.GetVirgilJWT, o.Client)
}
func (o *CryptoApi) GetGroupKeyVersion(c *gin.Context) {
a2r.Call(c, crypto.CryptoServiceClient.GetGroupKeyVersion, o.Client)
}
func (o *CryptoApi) GetGroupKeyEvents(c *gin.Context) {
a2r.Call(c, crypto.CryptoServiceClient.GetGroupKeyEvents, o.Client)
}
func (o *CryptoApi) SecurityPrecheck(c *gin.Context) {
a2r.Call(c, crypto.CryptoServiceClient.SecurityPrecheck, o.Client)
}
func (o *CryptoApi) IntegrityReport(c *gin.Context) {
a2r.Call(c, crypto.CryptoServiceClient.IntegrityReport, o.Client)
}

19
internal/api/router.go
View File

@ -12,6 +12,7 @@ import (
"github.com/openimsdk/protocol/group"
"github.com/openimsdk/protocol/msg"
"github.com/openimsdk/protocol/relation"
pbcrypto "github.com/openimsdk/protocol/crypto"
"github.com/openimsdk/protocol/rtc"
"github.com/openimsdk/protocol/third"
"github.com/openimsdk/protocol/user"
@ -112,6 +113,10 @@ func newGinRouter(ctx context.Context, client discovery.SvcDiscoveryRegistry, co
if err != nil {
return nil, err
}
cryptoConn, err := client.GetConn(ctx, config.Share.RpcRegisterName.Crypto)
if err != nil {
return nil, err
}
gin.SetMode(gin.ReleaseMode)
r := gin.New()
if v, ok := binding.Validator.Engine().(*validator.Validate); ok {
@ -344,6 +349,20 @@ func newGinRouter(ctx context.Context, client discovery.SvcDiscoveryRegistry, co
rtcGroup.POST("/delete_signal_records", rc.DeleteSignalRecords)
}
// Crypto / E2EE
{
cr := NewCryptoApi(pbcrypto.NewCryptoServiceClient(cryptoConn))
cryptoGroup := r.Group("/crypto")
cryptoGroup.POST("/register_device", cr.RegisterDevice)
cryptoGroup.POST("/get_devices", cr.GetDevices)
cryptoGroup.POST("/revoke_device", cr.RevokeDevice)
cryptoGroup.POST("/get_virgil_jwt", cr.GetVirgilJWT)
cryptoGroup.POST("/get_group_key_version", cr.GetGroupKeyVersion)
cryptoGroup.POST("/get_group_key_events", cr.GetGroupKeyEvents)
cryptoGroup.POST("/security_precheck", cr.SecurityPrecheck)
cryptoGroup.POST("/integrity_report", cr.IntegrityReport)
}
{
statisticsGroup := r.Group("/statistics")
statisticsGroup.POST("/user/register", u.UserRegisterCount)

496
internal/rpc/crypto/crypto.go Normal file
View File

@ -0,0 +1,496 @@
package crypto
import (
"context"
"fmt"
"time"
"github.com/VirgilSecurity/virgil-sdk-go/cryptoimpl"
"github.com/VirgilSecurity/virgil-sdk-go/sdk"
"github.com/openimsdk/open-im-server/v3/pkg/authverify"
"github.com/openimsdk/open-im-server/v3/pkg/common/config"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/controller"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/database/mgo"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/model"
pbcrypto "github.com/openimsdk/protocol/crypto"
"github.com/openimsdk/tools/db/mongoutil"
"github.com/openimsdk/tools/discovery"
"github.com/openimsdk/tools/errs"
"github.com/openimsdk/tools/log"
"github.com/openimsdk/tools/mcontext"
"google.golang.org/grpc"
)
const virgilJWTTTL = 20 * time.Minute
type Config struct {
RpcConfig config.Crypto
MongodbConfig config.Mongo
Share config.Share
Discovery config.Discovery
}
type cryptoServer struct {
pbcrypto.UnimplementedCryptoServiceServer
config *Config
cryptoDB controller.CryptoDatabase
jwtGenerator *sdk.JwtGenerator
}
func Start(ctx context.Context, config *Config, _ discovery.SvcDiscoveryRegistry, server *grpc.Server) error {
mgocli, err := mongoutil.NewMongoDB(ctx, config.MongodbConfig.Build())
if err != nil {
return err
}
db := mgocli.GetDB()
deviceDB, err := mgo.NewCryptoDeviceMongo(db)
if err != nil {
return err
}
keyVersionDB, err := mgo.NewGroupKeyVersionMongo(db)
if err != nil {
return err
}
keyEventDB, err := mgo.NewGroupKeyEventMongo(db)
if err != nil {
return err
}
cryptoDB := controller.NewCryptoDatabase(deviceDB, keyVersionDB, keyEventDB, mgocli.GetTx())
var jwtGen *sdk.JwtGenerator
vc := config.RpcConfig.Virgil
if vc.AppID != "" && vc.AppKey != "" && vc.AppKeyID != "" {
virgilCrypto := cryptoimpl.NewVirgilCrypto()
privateKey, err := virgilCrypto.ImportPrivateKey([]byte(vc.AppKey), "")
if err != nil {
return fmt.Errorf("import virgil app key: %w", err)
}
jwtGen = sdk.NewJwtGenerator(
privateKey,
vc.AppKeyID,
cryptoimpl.NewVirgilAccessTokenSigner(),
vc.AppID,
virgilJWTTTL,
)
}
pbcrypto.RegisterCryptoServiceServer(server, &cryptoServer{
config: config,
cryptoDB: cryptoDB,
jwtGenerator: jwtGen,
})
return nil
}
func (s *cryptoServer) RegisterDevice(ctx context.Context, req *pbcrypto.RegisterDeviceReq) (*pbcrypto.RegisterDeviceResp, error) {
opUserID := mcontext.GetOpUserID(ctx)
log.ZDebug(ctx, "RegisterDevice request",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"platform", req.Platform,
"deviceModel", req.DeviceModel,
"appVersion", req.AppVersion,
)
if req.UserID == "" || req.DeviceID == "" {
log.ZError(ctx, "RegisterDevice invalid args", errs.ErrArgs,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return nil, errs.ErrArgs.WrapMsg("userID and deviceID are required")
}
if err := authverify.CheckAccessV3(ctx, req.UserID, s.config.Share.IMAdminUserID); err != nil {
log.ZError(ctx, "RegisterDevice auth check failed", err,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return nil, err
}
device, err := s.cryptoDB.RegisterDevice(ctx, req.UserID, req.DeviceID, req.Platform, req.DeviceModel, req.AppVersion)
if err != nil {
log.ZError(ctx, "RegisterDevice db failed", err,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"platform", req.Platform,
)
return nil, err
}
log.ZDebug(ctx, "RegisterDevice success",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"virgilIdentity", device.VirgilIdentity,
)
return &pbcrypto.RegisterDeviceResp{
Device: modelDeviceToProto(device),
}, nil
}
func (s *cryptoServer) GetDevices(ctx context.Context, req *pbcrypto.GetDevicesReq) (*pbcrypto.GetDevicesResp, error) {
opUserID := mcontext.GetOpUserID(ctx)
log.ZDebug(ctx, "GetDevices request", "opUserID", opUserID, "targetUserID", req.UserID)
if req.UserID == "" {
log.ZError(ctx, "GetDevices invalid args", errs.ErrArgs, "opUserID", opUserID, "targetUserID", req.UserID)
return nil, errs.ErrArgs.WrapMsg("userID is required")
}
if err := authverify.CheckAccessV3(ctx, req.UserID, s.config.Share.IMAdminUserID); err != nil {
log.ZError(ctx, "GetDevices auth check failed", err, "opUserID", opUserID, "targetUserID", req.UserID)
return nil, err
}
devices, err := s.cryptoDB.GetDevices(ctx, req.UserID)
if err != nil {
log.ZError(ctx, "GetDevices db failed", err, "opUserID", opUserID, "targetUserID", req.UserID)
return nil, err
}
pbDevices := make([]*pbcrypto.DeviceInfo, 0, len(devices))
for _, d := range devices {
pbDevices = append(pbDevices, modelDeviceToProto(d))
}
log.ZDebug(ctx, "GetDevices success",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceCount", len(devices),
)
return &pbcrypto.GetDevicesResp{Devices: pbDevices}, nil
}
func (s *cryptoServer) RevokeDevice(ctx context.Context, req *pbcrypto.RevokeDeviceReq) (*pbcrypto.RevokeDeviceResp, error) {
opUserID := mcontext.GetOpUserID(ctx)
log.ZDebug(ctx, "RevokeDevice request",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
if req.UserID == "" || req.DeviceID == "" {
log.ZError(ctx, "RevokeDevice invalid args", errs.ErrArgs,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return nil, errs.ErrArgs.WrapMsg("userID and deviceID are required")
}
if err := authverify.CheckAccessV3(ctx, req.UserID, s.config.Share.IMAdminUserID); err != nil {
log.ZError(ctx, "RevokeDevice auth check failed", err,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return nil, err
}
if err := s.cryptoDB.RevokeDevice(ctx, req.UserID, req.DeviceID); err != nil {
log.ZError(ctx, "RevokeDevice db failed", err,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return nil, err
}
log.ZDebug(ctx, "RevokeDevice success",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return &pbcrypto.RevokeDeviceResp{}, nil
}
func (s *cryptoServer) GetVirgilJWT(ctx context.Context, req *pbcrypto.GetVirgilJWTReq) (*pbcrypto.GetVirgilJWTResp, error) {
opUserID := mcontext.GetOpUserID(ctx)
log.ZDebug(ctx, "GetVirgilJWT request",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
if req.UserID == "" || req.DeviceID == "" {
log.ZError(ctx, "GetVirgilJWT invalid args", errs.ErrArgs,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return nil, errs.ErrArgs.WrapMsg("userID and deviceID are required")
}
if err := authverify.CheckAccessV3(ctx, req.UserID, s.config.Share.IMAdminUserID); err != nil {
log.ZError(ctx, "GetVirgilJWT auth check failed", err,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return nil, err
}
if s.jwtGenerator == nil {
log.ZError(ctx, "GetVirgilJWT jwt generator not configured", errs.New("virgil is not configured"),
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return nil, errs.New("virgil is not configured").Wrap()
}
device, err := s.cryptoDB.GetDevice(ctx, req.UserID, req.DeviceID)
if err != nil {
if errs.ErrRecordNotFound.Is(err) {
log.ZError(ctx, "GetVirgilJWT device not found", err,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return nil, errs.ErrRecordNotFound.WrapMsg("device not found")
}
log.ZError(ctx, "GetVirgilJWT query device failed", err,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return nil, err
}
if device.Status != "active" {
log.ZError(ctx, "GetVirgilJWT device revoked", errs.ErrNoPermission,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"status", device.Status,
)
return nil, errs.ErrNoPermission.WrapMsg("device is revoked")
}
if err := s.cryptoDB.TouchDevice(ctx, req.UserID, req.DeviceID); err != nil {
log.ZError(ctx, "TouchDevice failed", err,
"opUserID", opUserID,
"userID", req.UserID,
"deviceID", req.DeviceID,
)
}
identity := req.UserID + ":" + req.DeviceID
token, err := s.jwtGenerator.GenerateToken(identity, nil)
if err != nil {
log.ZError(ctx, "GetVirgilJWT generate token failed", err,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"virgilIdentity", identity,
)
return nil, errs.New("generate virgil jwt failed").Wrap()
}
log.ZDebug(ctx, "GetVirgilJWT success",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"virgilIdentity", identity,
"expiresInSec", int64(virgilJWTTTL/time.Second),
)
return &pbcrypto.GetVirgilJWTResp{
VirgilJWT: token.String(),
ExpiresIn: int64(virgilJWTTTL / time.Second),
VirgilIdentity: identity,
}, nil
}
func (s *cryptoServer) GetGroupKeyVersion(ctx context.Context, req *pbcrypto.GetGroupKeyVersionReq) (*pbcrypto.GetGroupKeyVersionResp, error) {
opUserID := mcontext.GetOpUserID(ctx)
log.ZDebug(ctx, "GetGroupKeyVersion request", "opUserID", opUserID, "groupID", req.GroupID)
if req.GroupID == "" {
log.ZError(ctx, "GetGroupKeyVersion invalid args", errs.ErrArgs, "opUserID", opUserID, "groupID", req.GroupID)
return nil, errs.ErrArgs.WrapMsg("groupID is required")
}
version, err := s.cryptoDB.GetGroupKeyVersion(ctx, req.GroupID)
if err != nil {
log.ZError(ctx, "GetGroupKeyVersion db failed", err, "opUserID", opUserID, "groupID", req.GroupID)
return nil, err
}
log.ZDebug(ctx, "GetGroupKeyVersion success",
"opUserID", opUserID,
"groupID", req.GroupID,
"groupKeyVersion", version,
)
return &pbcrypto.GetGroupKeyVersionResp{
GroupID: req.GroupID,
GroupKeyVersion: version,
}, nil
}
// BumpGroupKeyVersion is internal-only (not exposed via HTTP API).
// Called by Group Service after membership changes.
func (s *cryptoServer) BumpGroupKeyVersion(ctx context.Context, req *pbcrypto.BumpGroupKeyVersionReq) (*pbcrypto.BumpGroupKeyVersionResp, error) {
opUserID := mcontext.GetOpUserID(ctx)
log.ZDebug(ctx, "BumpGroupKeyVersion request",
"opUserID", opUserID,
"groupID", req.GroupID,
"operatorUserID", req.OperatorUserID,
"eventType", req.EventType,
)
if req.GroupID == "" {
log.ZError(ctx, "BumpGroupKeyVersion invalid args", errs.ErrArgs,
"opUserID", opUserID,
"groupID", req.GroupID,
"operatorUserID", req.OperatorUserID,
"eventType", req.EventType,
)
return nil, errs.ErrArgs.WrapMsg("groupID is required")
}
newVersion, err := s.cryptoDB.BumpGroupKeyVersion(ctx, req.GroupID, req.OperatorUserID, req.EventType)
if err != nil {
log.ZError(ctx, "BumpGroupKeyVersion db failed", err,
"opUserID", opUserID,
"groupID", req.GroupID,
"operatorUserID", req.OperatorUserID,
"eventType", req.EventType,
)
return nil, err
}
log.ZDebug(ctx, "BumpGroupKeyVersion success",
"opUserID", opUserID,
"groupID", req.GroupID,
"operatorUserID", req.OperatorUserID,
"eventType", req.EventType,
"newGroupKeyVersion", newVersion,
)
return &pbcrypto.BumpGroupKeyVersionResp{
GroupID: req.GroupID,
GroupKeyVersion: newVersion,
}, nil
}
func (s *cryptoServer) GetGroupKeyEvents(ctx context.Context, req *pbcrypto.GetGroupKeyEventsReq) (*pbcrypto.GetGroupKeyEventsResp, error) {
opUserID := mcontext.GetOpUserID(ctx)
log.ZDebug(ctx, "GetGroupKeyEvents request",
"opUserID", opUserID,
"groupID", req.GroupID,
"sinceVersion", req.SinceVersion,
)
if req.GroupID == "" {
log.ZError(ctx, "GetGroupKeyEvents invalid args", errs.ErrArgs,
"opUserID", opUserID,
"groupID", req.GroupID,
"sinceVersion", req.SinceVersion,
)
return nil, errs.ErrArgs.WrapMsg("groupID is required")
}
events, err := s.cryptoDB.GetGroupKeyEvents(ctx, req.GroupID, req.SinceVersion)
if err != nil {
log.ZError(ctx, "GetGroupKeyEvents db failed", err,
"opUserID", opUserID,
"groupID", req.GroupID,
"sinceVersion", req.SinceVersion,
)
return nil, err
}
pbEvents := make([]*pbcrypto.GroupKeyEventInfo, 0, len(events))
for _, e := range events {
pbEvents = append(pbEvents, &pbcrypto.GroupKeyEventInfo{
EventID: e.EventID,
GroupID: e.GroupID,
GroupKeyVersion: e.GroupKeyVersion,
EventType: e.EventType,
OperatorUserID: e.OperatorUserID,
CreateTime: e.CreateTime.UnixMilli(),
})
}
log.ZDebug(ctx, "GetGroupKeyEvents success",
"opUserID", opUserID,
"groupID", req.GroupID,
"sinceVersion", req.SinceVersion,
"eventCount", len(events),
)
return &pbcrypto.GetGroupKeyEventsResp{Events: pbEvents}, nil
}
func (s *cryptoServer) SecurityPrecheck(ctx context.Context, req *pbcrypto.SecurityPrecheckReq) (*pbcrypto.SecurityPrecheckResp, error) {
opUserID := mcontext.GetOpUserID(ctx)
log.ZDebug(ctx, "SecurityPrecheck request",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"action", req.Action,
)
if req.UserID == "" || req.DeviceID == "" {
log.ZError(ctx, "SecurityPrecheck invalid args", errs.ErrArgs,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"action", req.Action,
)
return nil, errs.ErrArgs.WrapMsg("userID and deviceID are required")
}
if err := authverify.CheckAccessV3(ctx, req.UserID, s.config.Share.IMAdminUserID); err != nil {
log.ZError(ctx, "SecurityPrecheck auth check failed", err,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"action", req.Action,
)
return nil, err
}
device, err := s.cryptoDB.GetDevice(ctx, req.UserID, req.DeviceID)
if err != nil {
log.ZDebug(ctx, "SecurityPrecheck denied",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"reason", "device not found",
)
return &pbcrypto.SecurityPrecheckResp{Allowed: false, Reason: "device not found"}, nil
}
if device.Status != "active" {
log.ZDebug(ctx, "SecurityPrecheck denied",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"reason", "device is revoked",
)
return &pbcrypto.SecurityPrecheckResp{Allowed: false, Reason: "device is revoked"}, nil
}
log.ZDebug(ctx, "SecurityPrecheck allowed",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return &pbcrypto.SecurityPrecheckResp{Allowed: true}, nil
}
// IntegrityReport is a placeholder for future device integrity verification.
// Currently accepts all reports; implement validation logic when ready.
func (s *cryptoServer) IntegrityReport(ctx context.Context, req *pbcrypto.IntegrityReportReq) (*pbcrypto.IntegrityReportResp, error) {
opUserID := mcontext.GetOpUserID(ctx)
log.ZDebug(ctx, "IntegrityReport request",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"timestamp", req.Timestamp,
"reportSize", len(req.ReportData),
)
if req.UserID == "" || req.DeviceID == "" {
log.ZError(ctx, "IntegrityReport invalid args", errs.ErrArgs,
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
"timestamp", req.Timestamp,
)
return nil, errs.ErrArgs.WrapMsg("userID and deviceID are required")
}
log.ZDebug(ctx, "IntegrityReport accepted",
"opUserID", opUserID,
"targetUserID", req.UserID,
"deviceID", req.DeviceID,
)
return &pbcrypto.IntegrityReportResp{Accepted: true}, nil
}
func modelDeviceToProto(d *model.CryptoDevice) *pbcrypto.DeviceInfo {
return &pbcrypto.DeviceInfo{
DeviceID: d.DeviceID,
UserID: d.UserID,
Platform: d.Platform,
DeviceModel: d.DeviceModel,
AppVersion: d.AppVersion,
VirgilIdentity: d.VirgilIdentity,
Status: d.Status,
LastSeenAt: d.LastSeenAt.UnixMilli(),
CreateTime: d.CreateTime.UnixMilli(),
}
}

13
internal/rpc/group/group.go
View File

@ -66,6 +66,7 @@ type groupServer struct {
userClient *rpcli.UserClient
msgClient *rpcli.MsgClient
conversationClient *rpcli.ConversationClient
cryptoClient *rpcli.CryptoClient
}
type Config struct {
@ -117,12 +118,17 @@ func Start(ctx context.Context, config *Config, client discovery.SvcDiscoveryReg
if err != nil {
return err
}
cryptoConn, err := client.GetConn(ctx, config.Share.RpcRegisterName.Crypto)
if err != nil {
return err
}
gs := groupServer{
config: config,
webhookClient: webhook.NewWebhookClient(config.WebhooksConfig.URL),
userClient: rpcli.NewUserClient(userConn),
msgClient: rpcli.NewMsgClient(msgConn),
conversationClient: rpcli.NewConversationClient(conversationConn),
cryptoClient: rpcli.NewCryptoClient(cryptoConn),
}
gs.db = controller.NewGroupDatabase(rdb, &config.LocalCacheConfig, groupDB, groupMemberDB, groupRequestDB, mgocli.GetTx(), grouphash.NewGroupHashFromGroupServer(&gs))
gs.notification = NewNotificationSender(gs.db, config, gs.userClient, gs.msgClient, gs.conversationClient)
@ -274,6 +280,7 @@ func (s *groupServer) CreateGroup(ctx context.Context, req *pbgroup.CreateGroupR
if err := s.db.CreateGroup(ctx, []*model.Group{group}, groupMembers); err != nil {
return nil, err
}
s.cryptoClient.InitGroupKeyVersion(ctx, group.GroupID)
resp := &pbgroup.CreateGroupResp{GroupInfo: &sdkws.GroupInfo{}}
resp.GroupInfo = convert.Db2PbGroupInfo(group, req.OwnerUserID, uint32(len(userIDs)))
@ -544,6 +551,7 @@ func (s *groupServer) InviteUserToGroup(ctx context.Context, req *pbgroup.Invite
return nil, err
}
}
s.cryptoClient.BumpGroupKeyVersion(ctx, req.GroupID, opUserID, "member_added")
return &pbgroup.InviteUserToGroupResp{}, nil
}
@ -709,6 +717,7 @@ func (s *groupServer) KickGroupMember(ctx context.Context, req *pbgroup.KickGrou
return nil, err
}
s.webhookAfterKickGroupMember(ctx, &s.config.WebhooksConfig.AfterKickGroupMember, req)
s.cryptoClient.BumpGroupKeyVersion(ctx, req.GroupID, opUserID, "member_removed")
return &pbgroup.KickGroupMemberResp{}, nil
}
@ -964,6 +973,7 @@ func (s *groupServer) GroupApplicationResponse(ctx context.Context, req *pbgroup
if err := s.setMemberJoinSeq(ctx, req.GroupID, []string{req.FromUserID}); err != nil {
return nil, err
}
s.cryptoClient.BumpGroupKeyVersion(ctx, req.GroupID, mcontext.GetOpUserID(ctx), "member_added")
}
case constant.GroupResponseRefuse:
s.notification.GroupApplicationRejectedNotification(ctx, req)
@ -1029,6 +1039,7 @@ func (s *groupServer) JoinGroup(ctx context.Context, req *pbgroup.JoinGroupReq)
if err := s.setMemberJoinSeq(ctx, req.GroupID, []string{req.InviterUserID}); err != nil {
return nil, err
}
s.cryptoClient.BumpGroupKeyVersion(ctx, req.GroupID, req.InviterUserID, "member_added")
s.webhookAfterJoinGroup(ctx, &s.config.WebhooksConfig.AfterJoinGroup, req)
return &pbgroup.JoinGroupResp{}, nil
@ -1077,6 +1088,7 @@ func (s *groupServer) QuitGroup(ctx context.Context, req *pbgroup.QuitGroupReq)
return nil, err
}
s.webhookAfterQuitGroup(ctx, &s.config.WebhooksConfig.AfterQuitGroup, req)
s.cryptoClient.BumpGroupKeyVersion(ctx, req.GroupID, req.UserID, "member_left")
return &pbgroup.QuitGroupResp{}, nil
}
@ -1563,6 +1575,7 @@ func (s *groupServer) DismissGroup(ctx context.Context, req *pbgroup.DismissGrou
}
s.webhookAfterDismissGroup(ctx, &s.config.WebhooksConfig.AfterDismissGroup, cbReq)
s.cryptoClient.BumpGroupKeyVersion(ctx, req.GroupID, mcontext.GetOpUserID(ctx), "group_dismissed")
return &pbgroup.DismissGroupResp{}, nil
}

4
pkg/common/cmd/constant.go
View File

@ -44,6 +44,7 @@ var (
OpenIMRPCThirdCfgFileName string
OpenIMRPCUserCfgFileName string
OpenIMRPCRtcCfgFileName string
OpenIMRPCCryptoCfgFileName string
DiscoveryConfigFilename string
)
@ -75,6 +76,7 @@ func init() {
OpenIMRPCThirdCfgFileName = "openim-rpc-third.yml"
OpenIMRPCUserCfgFileName = "openim-rpc-user.yml"
OpenIMRPCRtcCfgFileName = "openim-rpc-rtc.yml"
OpenIMRPCCryptoCfgFileName = "openim-rpc-crypto.yml"
DiscoveryConfigFilename = "discovery.yml"
ConfigEnvPrefixMap = make(map[string]string)
@ -85,7 +87,7 @@ func init() {
OpenIMAPICfgFileName, OpenIMCronTaskCfgFileName, OpenIMMsgGatewayCfgFileName,
OpenIMMsgTransferCfgFileName, OpenIMPushCfgFileName, OpenIMCaptchaCfgFileName, OpenIMRPCAuthCfgFileName, OpenIMRPCCaptchaCfgFileName,
OpenIMRPCConversationCfgFileName, OpenIMRPCFriendCfgFileName, OpenIMRPCGroupCfgFileName,
OpenIMRPCMsgCfgFileName, OpenIMRPCThirdCfgFileName, OpenIMRPCUserCfgFileName, OpenIMRPCRtcCfgFileName, DiscoveryConfigFilename,
OpenIMRPCMsgCfgFileName, OpenIMRPCThirdCfgFileName, OpenIMRPCUserCfgFileName, OpenIMRPCRtcCfgFileName, OpenIMRPCCryptoCfgFileName, DiscoveryConfigFilename,
}
for _, fileName := range fileNames {

47
pkg/common/cmd/rpc_crypto.go Normal file
View File

@ -0,0 +1,47 @@
package cmd
import (
"context"
"github.com/openimsdk/open-im-server/v3/internal/rpc/crypto"
"github.com/openimsdk/open-im-server/v3/pkg/common/startrpc"
"github.com/openimsdk/open-im-server/v3/version"
"github.com/openimsdk/tools/system/program"
"github.com/spf13/cobra"
)
type CryptoRpcCmd struct {
*RootCmd
ctx context.Context
configMap map[string]any
cryptoConfig *crypto.Config
}
func NewCryptoRpcCmd() *CryptoRpcCmd {
var cryptoConfig crypto.Config
ret := &CryptoRpcCmd{cryptoConfig: &cryptoConfig}
ret.configMap = map[string]any{
OpenIMRPCCryptoCfgFileName: &cryptoConfig.RpcConfig,
MongodbConfigFileName: &cryptoConfig.MongodbConfig,
ShareFileName: &cryptoConfig.Share,
DiscoveryConfigFilename: &cryptoConfig.Discovery,
}
ret.RootCmd = NewRootCmd(program.GetProcessName(), WithConfigMap(ret.configMap))
ret.ctx = context.WithValue(context.Background(), "version", version.Version)
ret.Command.RunE = func(cmd *cobra.Command, args []string) error {
return ret.runE()
}
return ret
}
func (c *CryptoRpcCmd) Exec() error {
return c.Execute()
}
func (c *CryptoRpcCmd) runE() error {
return startrpc.Start(c.ctx, &c.cryptoConfig.Discovery, &c.cryptoConfig.RpcConfig.Prometheus, c.cryptoConfig.RpcConfig.RPC.ListenIP,
c.cryptoConfig.RpcConfig.RPC.RegisterIP, c.cryptoConfig.RpcConfig.RPC.AutoSetPorts, c.cryptoConfig.RpcConfig.RPC.Ports,
c.Index(), c.cryptoConfig.Share.RpcRegisterName.Crypto, &c.cryptoConfig.Share, c.cryptoConfig,
nil,
crypto.Start)
}

24
pkg/common/config/config.go
View File

@ -426,6 +426,7 @@ type RpcRegisterName struct {
Third string `mapstructure:"third"`
Captcha string `mapstructure:"captcha"`
Rtc string `mapstructure:"rtc"`
Crypto string `mapstructure:"crypto"`
}
func (r *RpcRegisterName) GetServiceNames() []string {
@ -441,6 +442,7 @@ func (r *RpcRegisterName) GetServiceNames() []string {
r.Third,
r.Captcha,
r.Rtc,
r.Crypto,
}
}
@ -463,6 +465,23 @@ type Rtc struct {
LiveKit LiveKit `mapstructure:"liveKit"`
}
type Crypto struct {
RPC struct {
RegisterIP string `mapstructure:"registerIP"`
ListenIP string `mapstructure:"listenIP"`
AutoSetPorts bool `mapstructure:"autoSetPorts"`
Ports []int `mapstructure:"ports"`
} `mapstructure:"rpc"`
Prometheus Prometheus `mapstructure:"prometheus"`
Virgil VirgilConfig `mapstructure:"virgil"`
}
type VirgilConfig struct {
AppID string `mapstructure:"appID"`
AppKey string `mapstructure:"appKey"`
AppKeyID string `mapstructure:"appKeyID"`
}
// FullConfig stores all configurations for before and after events
type Webhooks struct {
@ -674,6 +693,7 @@ var (
OpenIMRPCThirdCfgFileName = "openim-rpc-third.yml"
OpenIMRPCUserCfgFileName = "openim-rpc-user.yml"
OpenIMRPCRtcCfgFileName = "openim-rpc-rtc.yml"
OpenIMRPCCryptoCfgFileName = "openim-rpc-crypto.yml"
RedisConfigFileName = "redis.yml"
ShareFileName = "share.yml"
WebhooksConfigFileName = "webhooks.yml"
@ -763,6 +783,10 @@ func (r *Rtc) GetConfigFileName() string {
return OpenIMRPCRtcCfgFileName
}
func (c *Crypto) GetConfigFileName() string {
return OpenIMRPCCryptoCfgFileName
}
func (r *Redis) GetConfigFileName() string {
return RedisConfigFileName
}

133
pkg/common/storage/controller/crypto.go Normal file
View File

@ -0,0 +1,133 @@
package controller
import (
"context"
"time"
"github.com/google/uuid"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/database"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/model"
"github.com/openimsdk/tools/db/tx"
"github.com/openimsdk/tools/log"
)
type CryptoDatabase interface {
RegisterDevice(ctx context.Context, userID, deviceID, platform, deviceModel, appVersion string) (*model.CryptoDevice, error)
GetDevices(ctx context.Context, userID string) ([]*model.CryptoDevice, error)
GetDevice(ctx context.Context, userID, deviceID string) (*model.CryptoDevice, error)
RevokeDevice(ctx context.Context, userID, deviceID string) error
TouchDevice(ctx context.Context, userID, deviceID string) error
GetGroupKeyVersion(ctx context.Context, groupID string) (int64, error)
BumpGroupKeyVersion(ctx context.Context, groupID, operatorUserID, eventType string) (int64, error)
GetGroupKeyEvents(ctx context.Context, groupID string, sinceVersion int64) ([]*model.GroupKeyEvent, error)
}
type cryptoDatabase struct {
deviceDB database.CryptoDevice
keyVersionDB database.GroupKeyVersion
keyEventDB database.GroupKeyEvent
tx tx.Tx
}
func NewCryptoDatabase(
deviceDB database.CryptoDevice,
keyVersionDB database.GroupKeyVersion,
keyEventDB database.GroupKeyEvent,
tx tx.Tx,
) CryptoDatabase {
return &cryptoDatabase{
deviceDB: deviceDB,
keyVersionDB: keyVersionDB,
keyEventDB: keyEventDB,
tx: tx,
}
}
func (c *cryptoDatabase) RegisterDevice(ctx context.Context, userID, deviceID, platform, deviceModel, appVersion string) (*model.CryptoDevice, error) {
virgilIdentity := userID + ":" + deviceID
now := time.Now()
device := &model.CryptoDevice{
DeviceID: deviceID,
UserID: userID,
Platform: platform,
DeviceModel: deviceModel,
AppVersion: appVersion,
VirgilIdentity: virgilIdentity,
Status: "active",
LastSeenAt: now,
CreateTime: now,
}
if err := c.deviceDB.Create(ctx, device); err != nil {
return nil, err
}
return device, nil
}
func (c *cryptoDatabase) GetDevices(ctx context.Context, userID string) ([]*model.CryptoDevice, error) {
return c.deviceDB.FindByUserID(ctx, userID)
}
func (c *cryptoDatabase) GetDevice(ctx context.Context, userID, deviceID string) (*model.CryptoDevice, error) {
return c.deviceDB.FindByUserIDAndDeviceID(ctx, userID, deviceID)
}
func (c *cryptoDatabase) RevokeDevice(ctx context.Context, userID, deviceID string) error {
return c.deviceDB.UpdateStatus(ctx, userID, deviceID, "revoked")
}
func (c *cryptoDatabase) TouchDevice(ctx context.Context, userID, deviceID string) error {
return c.deviceDB.UpdateLastSeen(ctx, userID, deviceID)
}
func (c *cryptoDatabase) GetGroupKeyVersion(ctx context.Context, groupID string) (int64, error) {
v, err := c.keyVersionDB.Find(ctx, groupID)
if err != nil {
return 0, err
}
return v.GroupKeyVersion, nil
}
func (c *cryptoDatabase) BumpGroupKeyVersion(ctx context.Context, groupID, operatorUserID, eventType string) (int64, error) {
log.ZDebug(ctx, "cryptoDatabase BumpGroupKeyVersion begin",
"groupID", groupID,
"operatorUserID", operatorUserID,
"eventType", eventType,
)
var newVersion int64
err := c.tx.Transaction(ctx, func(ctx context.Context) error {
var err error
newVersion, err = c.keyVersionDB.IncrVersion(ctx, groupID)
if err != nil {
return err
}
event := &model.GroupKeyEvent{
EventID: uuid.New().String(),
GroupID: groupID,
GroupKeyVersion: newVersion,
EventType: eventType,
OperatorUserID: operatorUserID,
CreateTime: time.Now(),
}
return c.keyEventDB.Create(ctx, event)
})
if err != nil {
log.ZError(ctx, "cryptoDatabase BumpGroupKeyVersion failed", err,
"groupID", groupID,
"operatorUserID", operatorUserID,
"eventType", eventType,
)
return 0, err
}
log.ZDebug(ctx, "cryptoDatabase BumpGroupKeyVersion success",
"groupID", groupID,
"operatorUserID", operatorUserID,
"eventType", eventType,
"newGroupKeyVersion", newVersion,
)
return newVersion, nil
}
func (c *cryptoDatabase) GetGroupKeyEvents(ctx context.Context, groupID string, sinceVersion int64) ([]*model.GroupKeyEvent, error) {
return c.keyEventDB.FindSinceVersion(ctx, groupID, sinceVersion)
}

25
pkg/common/storage/database/crypto.go Normal file
View File

@ -0,0 +1,25 @@
package database
import (
"context"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/model"
)
type CryptoDevice interface {
Create(ctx context.Context, device *model.CryptoDevice) error
FindByUserID(ctx context.Context, userID string) ([]*model.CryptoDevice, error)
FindByUserIDAndDeviceID(ctx context.Context, userID, deviceID string) (*model.CryptoDevice, error)
UpdateStatus(ctx context.Context, userID, deviceID, status string) error
UpdateLastSeen(ctx context.Context, userID, deviceID string) error
}
type GroupKeyVersion interface {
Find(ctx context.Context, groupID string) (*model.GroupKeyVersion, error)
IncrVersion(ctx context.Context, groupID string) (int64, error)
}
type GroupKeyEvent interface {
Create(ctx context.Context, event *model.GroupKeyEvent) error
FindSinceVersion(ctx context.Context, groupID string, sinceVersion int64) ([]*model.GroupKeyEvent, error)
}

183
pkg/common/storage/database/mgo/crypto.go Normal file
View File

@ -0,0 +1,183 @@
package mgo
import (
"context"
"time"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/database"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/model"
"github.com/openimsdk/tools/errs"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
// ---- CryptoDevice ----
type CryptoDeviceMgo struct {
coll *mongo.Collection
}
func NewCryptoDeviceMongo(db *mongo.Database) (database.CryptoDevice, error) {
coll := db.Collection("crypto_device")
_, err := coll.Indexes().CreateMany(context.Background(), []mongo.IndexModel{
{
Keys: bson.D{{Key: "user_id", Value: 1}, {Key: "device_id", Value: 1}},
Options: options.Index().SetUnique(true),
},
{
Keys: bson.D{{Key: "user_id", Value: 1}},
},
})
if err != nil {
return nil, err
}
return &CryptoDeviceMgo{coll: coll}, nil
}
func (m *CryptoDeviceMgo) Create(ctx context.Context, device *model.CryptoDevice) error {
_, err := m.coll.InsertOne(ctx, device)
return err
}
func (m *CryptoDeviceMgo) FindByUserID(ctx context.Context, userID string) ([]*model.CryptoDevice, error) {
cursor, err := m.coll.Find(ctx, bson.M{"user_id": userID})
if err != nil {
return nil, err
}
var devices []*model.CryptoDevice
if err := cursor.All(ctx, &devices); err != nil {
return nil, err
}
return devices, nil
}
func (m *CryptoDeviceMgo) FindByUserIDAndDeviceID(ctx context.Context, userID, deviceID string) (*model.CryptoDevice, error) {
var device model.CryptoDevice
err := m.coll.FindOne(ctx, bson.M{"user_id": userID, "device_id": deviceID}).Decode(&device)
if err != nil {
if err == mongo.ErrNoDocuments {
return nil, errs.ErrRecordNotFound.WrapMsg("crypto device not found", "userID", userID, "deviceID", deviceID)
}
return nil, err
}
return &device, nil
}
func (m *CryptoDeviceMgo) UpdateStatus(ctx context.Context, userID, deviceID, status string) error {
result, err := m.coll.UpdateOne(ctx,
bson.M{"user_id": userID, "device_id": deviceID},
bson.M{"$set": bson.M{"status": status}},
)
if err != nil {
return err
}
if result.MatchedCount == 0 {
return errs.ErrRecordNotFound.WrapMsg("crypto device not found", "userID", userID, "deviceID", deviceID)
}
return nil
}
func (m *CryptoDeviceMgo) UpdateLastSeen(ctx context.Context, userID, deviceID string) error {
result, err := m.coll.UpdateOne(ctx,
bson.M{"user_id": userID, "device_id": deviceID},
bson.M{"$set": bson.M{"last_seen_at": time.Now()}},
)
if err != nil {
return err
}
if result.MatchedCount == 0 {
return errs.ErrRecordNotFound.WrapMsg("crypto device not found", "userID", userID, "deviceID", deviceID)
}
return nil
}
// ---- GroupKeyVersion ----
type GroupKeyVersionMgo struct {
coll *mongo.Collection
}
func NewGroupKeyVersionMongo(db *mongo.Database) (database.GroupKeyVersion, error) {
coll := db.Collection("group_key_version")
_, err := coll.Indexes().CreateOne(context.Background(), mongo.IndexModel{
Keys: bson.D{{Key: "group_id", Value: 1}},
Options: options.Index().SetUnique(true),
})
if err != nil {
return nil, err
}
return &GroupKeyVersionMgo{coll: coll}, nil
}
func (m *GroupKeyVersionMgo) Find(ctx context.Context, groupID string) (*model.GroupKeyVersion, error) {
var v model.GroupKeyVersion
err := m.coll.FindOne(ctx, bson.M{"group_id": groupID}).Decode(&v)
if err != nil {
if err == mongo.ErrNoDocuments {
return nil, errs.ErrRecordNotFound.WrapMsg("group key version not found", "groupID", groupID)
}
return nil, err
}
return &v, nil
}
func (m *GroupKeyVersionMgo) IncrVersion(ctx context.Context, groupID string) (int64, error) {
var result model.GroupKeyVersion
err := m.coll.FindOneAndUpdate(ctx,
bson.M{"group_id": groupID},
bson.M{"$inc": bson.M{"group_key_version": int64(1)}},
options.FindOneAndUpdate().SetUpsert(true).SetReturnDocument(options.After),
).Decode(&result)
if err != nil {
return 0, err
}
return result.GroupKeyVersion, nil
}
// ---- GroupKeyEvent ----
type GroupKeyEventMgo struct {
coll *mongo.Collection
}
const maxGroupKeyEventsPerQuery = 500
func NewGroupKeyEventMongo(db *mongo.Database) (database.GroupKeyEvent, error) {
coll := db.Collection("group_key_event")
_, err := coll.Indexes().CreateMany(context.Background(), []mongo.IndexModel{
{
Keys: bson.D{{Key: "group_id", Value: 1}, {Key: "group_key_version", Value: 1}},
},
{
Keys: bson.D{{Key: "event_id", Value: 1}},
Options: options.Index().SetUnique(true),
},
})
if err != nil {
return nil, err
}
return &GroupKeyEventMgo{coll: coll}, nil
}
func (m *GroupKeyEventMgo) Create(ctx context.Context, event *model.GroupKeyEvent) error {
_, err := m.coll.InsertOne(ctx, event)
return err
}
func (m *GroupKeyEventMgo) FindSinceVersion(ctx context.Context, groupID string, sinceVersion int64) ([]*model.GroupKeyEvent, error) {
cursor, err := m.coll.Find(ctx, bson.M{
"group_id": groupID,
"group_key_version": bson.M{"$gt": sinceVersion},
}, options.Find().
SetSort(bson.D{{Key: "group_key_version", Value: 1}}).
SetLimit(maxGroupKeyEventsPerQuery))
if err != nil {
return nil, err
}
var events []*model.GroupKeyEvent
if err := cursor.All(ctx, &events); err != nil {
return nil, err
}
return events, nil
}

29
pkg/common/storage/model/crypto.go Normal file
View File

@ -0,0 +1,29 @@
package model
import "time"
type CryptoDevice struct {
DeviceID string `bson:"device_id"`
UserID string `bson:"user_id"`
Platform string `bson:"platform"`
DeviceModel string `bson:"device_model"`
AppVersion string `bson:"app_version"`
VirgilIdentity string `bson:"virgil_identity"`
Status string `bson:"status"`
LastSeenAt time.Time `bson:"last_seen_at"`
CreateTime time.Time `bson:"create_time"`
}
type GroupKeyVersion struct {
GroupID string `bson:"group_id"`
GroupKeyVersion int64 `bson:"group_key_version"`
}
type GroupKeyEvent struct {
EventID string `bson:"event_id"`
GroupID string `bson:"group_id"`
GroupKeyVersion int64 `bson:"group_key_version"`
EventType string `bson:"event_type"`
OperatorUserID string `bson:"operator_user_id"`
CreateTime time.Time `bson:"create_time"`
}

51
pkg/rpcli/crypto.go Normal file
View File

@ -0,0 +1,51 @@
package rpcli
import (
"context"
pbcrypto "github.com/openimsdk/protocol/crypto"
"github.com/openimsdk/tools/log"
"google.golang.org/grpc"
)
func NewCryptoClient(cc grpc.ClientConnInterface) *CryptoClient {
return &CryptoClient{pbcrypto.NewCryptoServiceClient(cc)}
}
type CryptoClient struct {
pbcrypto.CryptoServiceClient
}
func (x *CryptoClient) BumpGroupKeyVersion(ctx context.Context, groupID, operatorUserID, eventType string) {
log.ZDebug(ctx, "BumpGroupKeyVersion start", "groupID", groupID, "operatorUserID", operatorUserID, "eventType", eventType)
resp, err := x.CryptoServiceClient.BumpGroupKeyVersion(ctx, &pbcrypto.BumpGroupKeyVersionReq{
GroupID: groupID,
OperatorUserID: operatorUserID,
EventType: eventType,
})
if err != nil {
log.ZError(ctx, "BumpGroupKeyVersion failed", err,
"groupID", groupID,
"operatorUserID", operatorUserID,
"eventType", eventType,
)
return
}
log.ZDebug(ctx, "BumpGroupKeyVersion success", "groupID", groupID, "newVersion", resp.GroupKeyVersion)
}
func (x *CryptoClient) InitGroupKeyVersion(ctx context.Context, groupID string) {
log.ZDebug(ctx, "InitGroupKeyVersion start", "groupID", groupID, "eventType", "group_created")
_, err := x.CryptoServiceClient.BumpGroupKeyVersion(ctx, &pbcrypto.BumpGroupKeyVersionReq{
GroupID: groupID,
EventType: "group_created",
})
if err != nil {
log.ZError(ctx, "InitGroupKeyVersion failed", err,
"groupID", groupID,
"eventType", "group_created",
)
return
}
log.ZDebug(ctx, "InitGroupKeyVersion success", "groupID", groupID)
}

675
virgil_chat_server_design.md Normal file
View File

@ -0,0 +1,675 @@
# Virgil Security 接入到聊天系统设计方案（服务器）
## 1. 服务器设计目标
服务端的目标不是参与解密，而是提供 **业务承载层**：
- 业务身份认证
- 设备管理
- Virgil JWT 签发
- 单聊/群聊会话管理
- 密文消息接入与路由
- 离线同步
- 文件上传/下载票据
- 群成员关系与群密钥版本索引
- 风控与设备吊销
---
## 2. 服务器总体架构
```mermaid
flowchart LR
subgraph Backend["业务服务端"]
B1["API Gateway"]
B2["Auth Service"]
B3["Virgil JWT Service"]
B4["Conversation Service"]
B5["Message Router / Sync"]
B6["Group Service"]
B7["Push Service"]
B8["File Ticket Service"]
B9["Risk / Device Service"]
B10["Metadata DB"]
end
subgraph Virgil["Virgil Cloud"]
V1["Cards / Public Key Directory"]
V2["Restore Related Services"]
end
subgraph OSS["Object Storage / CDN"]
O1["Encrypted File Objects"]
end
B1 --> B2
B1 --> B3
B1 --> B4
B1 --> B5
B1 --> B6
B1 --> B8
B1 --> B9
B2 --> B10
B3 --> B10
B4 --> B10
B5 --> B10
B6 --> B10
B8 --> O1
B9 --> B10
B3 --> V1
B3 --> V2
```
---
## 3. 服务器职责边界
## 3.1 服务端应该负责什么
- 用户登录与 token
- 设备注册与吊销
- Virgil JWT 签发
- 会话与群组元数据
- 密文消息存储
- 消息序列号与游标
- 离线消息投递
- 已读/已送达状态
- 文件票据
- 群成员变更事件
- 风控控制
## 3.2 服务端不应该负责什么
- 私钥保存
- 文本明文解析
- 文件明文解析
- 群密钥管理的明文操作
- 替客户端做消息加解密
---
## 4. 推荐服务拆分
```text
api-gateway
auth-service
virgil-jwt-service
conversation-service
message-service
group-service
sync-service
push-service
file-ticket-service
risk-device-service
```
### 各服务说明
#### api-gateway
- 统一入口
- 鉴权
- 请求限流
- trace 注入
#### auth-service
- 登录
- refresh
- logout
#### virgil-jwt-service
- 生成 Virgil JWT
- user/device 到 virgil identity 的映射
- 风控前置校验
#### conversation-service
- 单聊会话
- 会话设置
- 会话列表
#### message-service
- 接收密文 envelope
- 分配 message_id / seq
- 存储与 fanout
#### group-service
- 创建群
- 加人/踢人/退群
- 群资料
- 维护 group_key_version
#### sync-service
- bootstrap
- 增量消息
- 群事件同步
#### push-service
- message.new
- message.recall
- group.member_changed
- device.revoked
#### file-ticket-service
- 上传票据
- 下载票据
- 文件元信息
#### risk-device-service
- 设备状态
- 完整性校验
- 风险拦截
---
## 5. 服务端数据模型
## 5.1 用户与设备
```mermaid
erDiagram
USER ||--o{ DEVICE : owns
USER {
string user_id
string status
int64 created_at
}
DEVICE {
string device_id
string user_id
string platform
string device_model
string app_version
string virgil_identity
string status
int64 last_seen_at
}
```
## 5.2 会话与消息
```mermaid
erDiagram
CONVERSATION ||--o{ MESSAGE : contains
CONVERSATION {
string conversation_id
string type
string peer_key
string group_id
int64 created_at
}
MESSAGE {
string message_id
string conversation_id
string chat_type
string sender_user_id
string sender_device_id
string receiver_user_id
string group_id
int64 group_key_version
string content_type
int envelope_version
string cipher_suite
bytes ciphertext
bytes signature
int64 seq
int64 server_timestamp
}
```
## 5.3 群与群事件
```mermaid
erDiagram
GROUP ||--o{ GROUP_MEMBER : has
GROUP ||--o{ GROUP_KEY_EVENT : emits
GROUP {
string group_id
string conversation_id
string owner_user_id
string name
int64 group_key_version
string status
}
GROUP_MEMBER {
string group_id
string user_id
string role
string status
int64 join_version
int64 leave_version
}
GROUP_KEY_EVENT {
string event_id
string group_id
int64 group_key_version
string event_type
string operator_user_id
int64 created_at
}
```
---
## 6. 服务端接口设计
## 6.1 认证与设备
### 登录
```http
POST /api/v1/auth/login
```
### 刷新 token
```http
POST /api/v1/auth/refresh
```
### 登出
```http
POST /api/v1/auth/logout
```
### 注册设备
```http
POST /api/v1/devices/register
```
### 查询设备列表
```http
GET /api/v1/devices
```
### 吊销设备
```http
POST /api/v1/devices/{device_id}:revoke
```
---
## 6.2 Virgil JWT
### 获取 Virgil JWT
```http
POST /api/v1/security/virgil-jwt
```
请求：
```json
{
"device_id": "ios_a1"
}
```
响应：
```json
{
"virgil_jwt": "virgil_jwt_xxx",
"expires_in": 3600,
"virgil_identity": "u_1001:ios_a1"
}
```
---
## 6.3 会话
### 创建/获取单聊
```http
POST /api/v1/conversations/single
```
### 会话列表
```http
GET /api/v1/conversations
```
### 会话详情
```http
GET /api/v1/conversations/{conversation_id}
```
### 会话设置
```http
POST /api/v1/conversations/{conversation_id}/settings
```
---
## 6.4 消息
### 发送消息
```http
POST /api/v1/messages
```
### 批量发送
```http
POST /api/v1/messages:batchSend
```
### 拉取指定消息
```http
GET /api/v1/messages/{message_id}
```
### 撤回消息
```http
POST /api/v1/messages/{message_id}:recall
```
### 仅自己删除
```http
POST /api/v1/messages/{message_id}:deleteForMe
```
---
## 6.5 群组
### 创建群
```http
POST /api/v1/groups
```
### 群详情
```http
GET /api/v1/groups/{group_id}
```
### 群上下文
```http
GET /api/v1/groups/{group_id}/context
```
### 群成员列表
```http
GET /api/v1/groups/{group_id}/members
```
### 加人
```http
POST /api/v1/groups/{group_id}/members:add
```
### 移除成员
```http
POST /api/v1/groups/{group_id}/members:remove
```
### 退群
```http
POST /api/v1/groups/{group_id}:leave
```
### 解散群
```http
POST /api/v1/groups/{group_id}:dismiss
```
### 修改群资料
```http
POST /api/v1/groups/{group_id}/profile
```
---
## 6.6 文件
### 申请上传票据
```http
POST /api/v1/files/upload-ticket
```
### 上传完成
```http
POST /api/v1/files/{file_id}:complete
```
### 获取下载票据
```http
POST /api/v1/files/{file_id}/download-ticket
```
### 获取文件元信息
```http
GET /api/v1/files/{file_id}
```
---
## 6.7 同步
### 冷启动 bootstrap
```http
GET /api/v1/sync/bootstrap
```
### 增量消息同步
```http
GET /api/v1/sync/messages
```
### 群事件同步
```http
GET /api/v1/sync/group-events
```
---
## 6.8 回执与状态
### 已送达/已读
```http
POST /api/v1/messages/ack
```
### 批量已读
```http
POST /api/v1/conversations/{conversation_id}:markRead
```
### 输入中
```http
POST /api/v1/conversations/{conversation_id}/typing
```
---
## 6.9 风控与安全
### 预检查
```http
POST /api/v1/security/precheck
```
### 设备完整性上报
```http
POST /api/v1/security/integrity-report
```
---
## 7. 服务端核心流程
## 7.1 签发 Virgil JWT
```mermaid
sequenceDiagram
participant C as 客户端
participant J as Virgil JWT Service
participant A as Auth Service
participant R as Risk / Device Service
C->>J: /security/virgil-jwt
J->>A: 校验 access_token
A-->>J: 用户有效
J->>R: 校验 device_id / 风险状态
R-->>J: 允许或拒绝
J-->>C: 返回 virgil_jwt
```
## 7.2 消息接收与路由
```mermaid
sequenceDiagram
participant C as 客户端
participant M as Message Service
participant S as Sync Service
participant P as Push Service
participant DB as Metadata DB
C->>M: POST /messages
M->>DB: 存储 envelope
M->>DB: 分配 message_id / seq
M->>S: 写离线同步流
M->>P: 触发新消息推送
M-->>C: 返回 accepted
```
## 7.3 群成员变更
```mermaid
sequenceDiagram
participant A as 管理员客户端
participant G as Group Service
participant DB as Metadata DB
participant S as Sync Service
participant P as Push Service
A->>G: /groups/{id}/members:add
G->>DB: 更新群成员
G->>DB: group_key_version + 1
G->>S: 写入 group event
G->>P: 推送 group.member_changed
G-->>A: 返回最新 group_key_version
```
---
## 8. 服务端幂等与校验
## 8.1 幂等设计
### 发消息幂等键
推荐：
```text
sender_user_id + sender_device_id + client_message_id
```
### 创建群幂等
推荐使用：
```http
Idempotency-Key: <uuid>
```
## 8.2 消息接口校验项
- 当前用户是会话参与者
- `sender_device_id` 属于当前用户
- 单聊的 `receiver_user_id` 与会话匹配
- 群聊的 `group_id` 与会话匹配
- 当前用户是群成员
- `group_key_version` 合法
- `ciphertext` 长度不超过限制
## 8.3 Virgil JWT 接口校验项
- 业务 token 有效
- `device_id` 属于当前用户
- 设备状态为 active
- 未被吊销
- 风控未阻断
---
## 9. WebSocket / 推送事件建议
### WebSocket 事件
- `message.new`
- `message.recall`
- `message.ack`
- `group.member_changed`
- `group.profile_changed`
- `device.revoked`
- `security.force_logout`
### 推送原则
推送中不要放消息明文，建议只放：
- 会话 ID
- 发送方昵称
- 占位提示
---
## 10. 服务端风险点
- 若服务端误记录明文日志，会破坏 E2EE 边界
- 若文件下载链接长期有效，文件密文泄露面会扩大
- 若群成员变更后未提升 `group_key_version`，客户端可能使用旧上下文
- 若被吊销设备仍可获取 Virgil JWT，会造成安全失控
- 若推送带明文，会绕过加密链路
---
## 11. 推荐实施路径
## 阶段 1：单聊 MVP
实现：
- 登录
- Virgil JWT
- 单聊消息发送
- 增量同步
- 已读回执
## 阶段 2：群聊
实现：
- 创建群
- 群成员管理
- group_key_version
- 群事件同步
## 阶段 3：文件与恢复
实现：
- 文件票据
- 加密文件上传下载
- 新设备恢复
- 风控增强
---
## 12. 服务器总结
服务器在 Virgil Security 接入中的核心定位是：
- 不参与消息解密
- 不保存私钥
- 只处理密文、元数据和业务控制逻辑
具体来说，服务器负责：
- 身份认证
- 设备管理
- Virgil JWT 签发
- 密文消息接入、存储、路由
- 会话与群成员关系
- 文件票据
- 同步与推送
- 风控和设备吊销
这样才能既保留现有聊天系统的业务能力，又维持端到端加密的安全边界。