# galog

galog是一个无三方依赖、支持单CPU最高5wQPS写入的ga-sdk组件，可直接用于GA事实日志和资产日志打点。

> 说明：默认是异步写入模式，异步队列长度=100，较大QPS时可自行调整，详见参数：
> - enableAsync
> - defaultAsyncMsgLen

## Usage

```golang
// ga日志初始化
projectID := "20433"
clientID := "Android_5.0_tyGuest,weixinPay,tyAccount.alipay.0-hall20433.tuyoo.sdktest"
glogger, err := galog.NewGALogger("logs", projectID, clientID, galog.LogTypeTrack)
if err != nil {
    panic(err)
}

// ga日志打点调用
props := map[string]interface{}{
    "ip_address":  "127.0.0.1",
    "proj_app_id": "10010",
    "uuid":        "4951d472-2c46-4fe5-9c4f-c35b6fb53f67",
    "ts":          tt.UnixNano(),
}
glogger.
    GetEntry("sdk_s_login_succ").
    SetDeviceID("device001").
    SetUserID("10086").
    SetProperties(props).
    Flush()
```

```golang
// asset日志初始化
projectID := "28"
clientID := "Android_4.827_tyGuest,nearme.nearme.0-hall28.oppo.bydzz"
glogger, err := galog.NewGALogger("logs", projectID, clientID, galog.LogTypeAsset)
if err != nil {
    panic(err)
}

// asset日志打点调用
asset := galog.AssetProperties{}
asset.
    SetAssetID("13101").
    SetAssetName("\u9501\u5b9a").
    SetAssetType("6").
    SetAssetFinal("2").
    SetAssetAssociated("3").
    SetAssetStartTime("0").
    SetAssetTimeLimit("0").
    SetAssetSource("").
    SetKV("uuid", "uuid-v4")

glogger.
    GetEntry("asset_increase").
    SetDeviceID("").
    SetUserID("10086").
    SetProperties(asset).
    Flush()
```

## suger usage

```golang
package chat_log

import (
	"log"
	"strconv"

	"tygit.tuyoo.com/gocomponents/galog"
)

var (
	slogger  *galog.Logger
	galogger *galog.GALogger

	lerr error
)

// MustInitLoggerOnce 服务启动后初始化一次
func MustInitLoggerOnce() {
	slogger, lerr = galog.NewServerLogger(&galog.ServerLogOptions{
		LogDir:         "run/logs",
		EnableAsync:    true,
		AsyncQueueSize: 1000,
	})
	if lerr != nil {
		log.Printf("MustInitLoggerOnce err: %s\n", lerr.Error())
		panic("MustInitLoggerOnce err")
	}
}

// MustInitGALoggerOnce 服务启动后初始化一次
func MustInitGALoggerOnce() {
	galogger, lerr = galog.NewServerGALogger(&galog.ServerGALogOptions{
		LogDir:         "run/bi",
		EnableAsync:    true,
		AsyncQueueSize: 1000,
		ProjectID:      ProjectId,
		ClientID:       ClientId,
		LogType:        galog.LogTypeTrack,
	})
	if lerr != nil {
		log.Printf("MustInitGALoggerOnce err: %s\n", lerr.Error())
		panic("MustInitGALoggerOnce err")
	}
}

func LogToGANew(eventName, deviceId string, userId int64, prop map[string]interface{}, version string) {
	lib := map[string]string{
        "lib_type":    "golang",
        "lib_version": version,
    }

    galogger.
        GetEntry(eventName).
        SetDeviceID(deviceId).
        SetUserID(strconv.FormatInt(userId, 10)).
        SetLib(lib).
        SetProperties(prop).
        Flush()
}
```

## 日志目录结构

```
{logDir}/ga_log/{日志类型}_{hostname}_{年月日}_{小时}.log
{logDir}/asset_log/{日志类型}_{hostname}_{年月日}_{小时}.log
```

## Benchmark

- 测试机：Mac-M1 8c 16G

- 同步写入模式
```
goos: darwin
goarch: arm64
pkg: tygit.tuyoo.com/gocomponents/galog
BenchmarkGaLog-8           87565             12831 ns/op            7590 B/op         90 allocs/op
BenchmarkAssetLog-8       234127              5832 ns/op            3028 B/op         44 allocs/op
PASS
ok      tygit.tuyoo.com/gocomponents/galog      3.190s

# 100w行日志文件，写入性能未见明显衰减
BenchmarkGaLog-8           89175             12481 ns/op            7590 B/op         90 allocs/op
BenchmarkAssetLog-8       195075              5933 ns/op            3028 B/op         44 allocs/op
```

- 异步写入模式（默认）
```
goos: darwin
goarch: arm64
pkg: tygit.tuyoo.com/gocomponents/galog
BenchmarkGaLog-8           95530             11331 ns/op            7599 B/op         90 allocs/op
BenchmarkAssetLog-8       337963              3512 ns/op            3032 B/op         44 allocs/op
PASS
ok      tygit.tuyoo.com/gocomponents/galog      3.040s

# 120w行日志文件，写入性能未见明显衰减
BenchmarkGaLog-8          116559             10450 ns/op            7598 B/op         90 allocs/op
BenchmarkAssetLog-8       335670              3512 ns/op            3032 B/op         44 allocs/op
```

## Load Testing

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o galog examples/hey.go

- 测试机：sa101-ecs-bj4-pt57-test-wxy
- linux服务器配置：2c4G
- 测试时间：2024-02-26 17:45 ~ 18:45

```
./galog -n 1000000 -c 2 -q 100
```

| 压测说明   | CPU平台使用率 | 内存使用率 | IO平均使用率 | 进程打开的文件句柄数 |
|----- | ----- | ----- | ----- | ----- |
| 无任务 | 1% | 52%  | 0.08% | 0 |
| 并发=2，qps=200 | 1% | 52%  | 0.08% | 7 |
| 并发=5，qps=5000 | 4% | 52%  | 1% | 7 |
| 并发=5，qps=50000(队列1000，有阻塞) realqps=5000 | 40% | 52%  | 30% | 7 |
| 并发=5，qps=50000(队列10000，有阻塞) realqps=6500 | 6% | 52%  | 1% | 7 |
| 并发=20，qps=50000(队列10000，有阻塞) realqps=18000 | 10% | 52%  | 3% | 7 |
| 并发=50，qps=50000(队列10000) realqps=50000 | 35% | 52%  | 10% | 7 |
| 并发=100，qps=100000(队列10000) realqps=90000 | 75% | 52%  | 16% | 7 |

- 压测结果：写入队列=10000，单核支持最高写入QPS约5w，CPU=75%。

- 监控结果如下：
![img2.png](img/WX20240226-183250@2x.png)
![img2.png](img/WX20240226-184752@2x.png)

## ga日志格式范式

```json
{
	"project_id": "20437",
	"type": "track",
	"event": "sdk_sword_holder_succ",
	"event_time": 1708568434744,
    "device_id": "",
	"user_id": "908825698",
	"client_id": "Android_5.1_tyGuest,weixinPay,tyAccount.alipay.0-hall20437.tuyoo.sdkonline",
	"properties": {
		"sdk_track_id": "3:6060:636978360:1708568434",
		"sdk_sub_channel": "fish3d",
		"country": "中国",
		"proj_game_id": "",
		"sdk_error_code": "",
		"sdk_error_msg": "",
		"city": "宁德市",
		"sdk_sword_track_id": "807370ff-ceac-4ee3-b68f-50642ca4953c",
		"sub_platform_id": "2",
		"sdk_sword_holder_id": "320",
		"sdk_sword_version": "v1.1.7",
		"uuid": "56fdf0f9-9daa-4bf6-be69-88e15f05c36c",
		"province": "福建省",
		"sdk_login_channel_type": "",
		"app_id": "20437",
		"sdk_s_login_channel_type": "",
		"sdk_main_channel": "official",
		"game_id": "20437",
		"sdk_s_route": "/api/sworder-server/rule/v1/getUserRisk",
		"sdk_sword_holder_result": "999999",
		"proj_cloud_id": "3",
		"proj_app_id": "10010",
		"ip_address": "59.58.58.18",
		"sdk_sword_holder_type": "login",
		"sdk_sword_holder_name": "非信任设备禁止登录",
		"proj_client_id": "Android_5.505_tyGuest,tyAccount,yidunlogin.weixinPay,alipay,yinlian,jingdong,weixinShare.0-hall28.official.fish3d",
		"sdk_sword_holder_process": "{\"330\":[\"[330-0]未命中\"]}",
		"sdk_yidun_device_id": "UuCJztYTtxRETUVUAEaFoCQaw8H2qkWE",
		"platform_id": "1",
		"sub_channel_id": "sdkonline",
		"proj_package_name": "",
		"channel_id": "tuyoo"
	},
    "lib": {
        "lib_version": "v1.0.0",
		"lib_type": "go"
	},
}
```

## asset日志格式范例

```json
{
	"project_id": "28",
	"type": "asset",
	"event": "asset_increase",
	"event_time": 1706631881042,
	"device_id": "",
	"user_id": "928426614",
	"client_id": "Android_4.827_tyGuest,nearme.nearme.0-hall28.oppo.bydzz",
	"properties": {
		"proj_ga_eventId": "STARTUP_QUEST_REWARD_COIN",
		"proj_asset_value": "2",
		"proj_asset_final": "2",
		"proj_chip_type": "6",
		"proj_asset_id": "13101",
		"proj_asset_name": "\u9501\u5b9a",
		"proj_asset_type": "free",
		"proj_asset_time_limit": "0",
		"proj_asset_start_time": "0",
		"proj_asset_source": "",
		"proj_tuyoo_order_id": "",
		"game_id": "28",
		"app_id": "10063",
		"proj_project_id": "28",
		"proj_client_id": "Android_4.827_tyGuest,nearme.nearme.0-hall28.oppo.bydzz",
		"uuid": "77b907688c624076a84ca7b4b29668a5"
	},
    "lib": {
        "lib_version": "v1.0.0",
		"lib_type": "go"
	}
}
```

## Changelog

| 版本   | 修订说明 | 提交人 | 发布时间 |
|----- | ----- | ----- | ----- |
| v1.0.0 | galog 0依赖、支持异步写入、小时切割日志、ga和asset日志类型 | 田文  | 2024.02.22 |
| v1.1.0 | add suger | 田文  | 2025.06.30 |