[TOC]
awesome-go
【GO】golang 降级|熔断|限流实战
Go实现各类限流 推荐看看
https://github.com/alibaba/sentinel-golang 2.5k
https://github.com/uber-go/ratelimit 3.7k
把请求数平均到耗时, 比如QPS=10/s 那么两个请求间隔100ms; 缺点是只能阻塞获取
https://github.com/juju/ratelimit 2.8k
功能更丰富
golang.org/x/time/rate
该限流器是基于Token Bucket(令牌桶)实现的。
golang 提供了拓展库(golang.org/x/time/rate)提供了限流器组件
https://pkg.go.dev/golang.org/x/time/rate
功能更丰富
https://github.com/sony/gobreaker 2.5k
https://github.com/afex/hystrix-go 4k
Simple middleware to rate-limit HTTP requests.
https://github.com/didip/tollbooth 2.4k
缓存击穿
将一组相同的请求合并成一个请求,实际上只会去请求一次,然后对所有的请求返回相同的结果。
"golang.org/x/sync/singleflight"
"github.com/zeromicro/go-zero/core/syncx"的 syncx.SingleFlight
如果第一个请求在执行了, 后面的请求阻塞等待获取结果(会指定key, 相同的key才会)
https://github.com/buraksezer/olric
https://github.com/dgraph-io/badger
go.etcd.io/bbolt
github.com/syndtr/goleveldb
goroutine 作为 golang 并发实现的核心组成部分,非常容易上手使用,但却很难驾驭得好。我们经常会遭遇各种形式的 goroutine 泄漏,这些泄漏的 goroutine 会一直存活直到进程终结。它们的占用的栈内存一直无法释放、关联的堆内存也不能被 GC 清理,系统的可用内存会随泄漏 goroutine 的增多越来越少,直至崩溃!
https://github.com/uber-go/goleak
io.Reader and io.Writer
bufio.Reader或bufio.Writer
io.Reader and io.Writer的替代品, 支持统计字节数
https://github.com/samber/go-metered-io
https://visualgdb.com/gdbreference/commands/x
https://github.com/jondot/goweight
$ go get github.com/jondot/goweight
$ cd current-project
$ goweight
只支持linux的二进制, 可以看到每个包的大小(与goweight类似, 但是以图的形式展示)
$ go install github.com/nikolaydubina/go-binsize-treemap@latest
$ go tool nm -size <binary finename> | go-binsize-treemap > binsize.svg
https://github.com/mailgun/godebug 已过时
https://github.com/go-delve/delve
https://github.com/containous/yaegi
可以提供交互环境
https://github.com/topxeq/gotx
https://github.com/cosmtrek/air
https://github.com/air-verse/air
https://github.com/topics/live-reload?l=go
https://github.com/nikolaydubina/go-recipes#monitoring
go get -u github.com/google/gops
go install github.com/google/gops@latest
C:\Users\35084>gops tree
...
├── 16712
│ └── 5988 (gops.exe) {go1.14.1}
├── 4728
│ ├── 16028 (com.docker.backend.exe) {go1.12.16}
│ └── 3708 (com.docker.proxy.exe) {go1.12.16}
└── 5172
└── 12080 (gopls.exe) {go1.14.1}
C:\Users\35084>gops
18256 16712 gops.exe go1.14.1 D:\go\bin\gops.exe
16028 4728 com.docker.backend.exe go1.12.16 C:\Program Files\Docker\Docker\resources\com.docker.backend.exe
12080 5172 gopls.exe go1.14.1 D:\go\bin\gopls.exe
3708 4728 com.docker.proxy.exe go1.12.16 C:\Program Files\Docker\Docker\resources\com.docker.proxy.exe
> 依次PID,PPID,进程名称,编译版本,进程路径
C:\Users\35084>gops 3708
parent PID: 4728
threads: 12
memory usage: 0.058%
cpu usage: 0.001%
username: DESKTOP-PK520IC\35084
cmd+args: "com.docker.proxy.exe" -dockerExe "C:\Program Files\Docker\Docker\resources\bin\docker.exe" -host-names host.docker.internal,docker.for.win.host.internal,docker.for.win.localhost -gateway-names gateway.docker.internal,docker.for.win.gateway.internal,docker.for.win.http.internal -vm-names vm.docker.internal,docker-for-desktop,docker-desktop,kubernetes.docker.internal -host-ip 192.168.65.2 -gateway-ip 192.168.65.1 -vm-ip 192.168.65.3 -pki "C:\ProgramData\DockerDesktop\pki" -inject-hosts=True
elapsed time: 02:45:26
local/remote: 127.0.0.1:33499 <-> 0.0.0.0:0 (LISTEN)
local/remote: 127.0.0.1:53974 <-> :0 ()
gops还有大量其它的功能
SSA
Go 1.7开始,Go将原来的IR(Intermediate Representation,中间代码)转换成SSA(Static Single Assignment,静态单赋值)形式的IR
https://golang.design/gossa https://github.com/golang-design/ssaplayground
https://silverrainz.me/go-ssaviz/ https://github.com/SilverRainZ/go-ssaviz
汇编
https://godbolt.org/ https://github.com/compiler-explorer/compiler-explorer
https://github.com/loov/lensm
https://github.com/mmcloughlin/avo
AST
官方go/ast
package main
import (
"go/ast"
"go/parser"
"go/token"
)
func main() {
fs := token.NewFileSet()
tr, _ := parser.ParseExpr("(3-1) * 5")
ast.Print(fs, tr)
}
https://github.com/reflog/go2ast
https://github.com/xiazemin/ast_graph
go install github.com/goreleaser/goreleaser@latest
添加git 支持同时添加tag
git tag markdown/v1.0.5
git push -u origin main
git push -u origin markdown/v1.0.5
or
git tag -a markdown/v1.0.5 -m "markdown/v1.0.5"
git push origin markdown/v1.0.5
添加goreleaser 支持
goreleaser init
获取 Github Token
访问 Settings / Developer Settings / Personal access tokens,点击 Generate new token 按钮,生成一个新的 Token,将 Token 保存到 ~/.config/goreleaser/github_token
文件中 或者 export GITHUB_TOKEN="YOUR_GH_TOKEN"
。
goreleaser release
goreleaser 的功能还是很强大的,同时支持github 的release,同时我们也可以配置docker
网络唤醒
https://github.com/seriousm4x/UpSnap
https://github.com/google/gopacket
https://github.com/gopacket/gopacket
https://github.com/urfave/negroni
https://github.com/urfave/negroni#third-party-middleware
HTTP caching proxy, implementing RFC 7234
https://github.com/pquerna/cachecontrol
https://github.com/darkweak/souin
https://github.com/darkweak/souin/blob/master/pkg/rfc
https://github.com/apache/trafficcontrol/tree/master/grove
https://github.com/gofiber/fiber
基于fasthttp
https://github.com/cloudwego/hertz
tus(Terminated Uploads)
https://github.com/tus/tusd/tree/main
https://github.com/rainycape/vfs
https://github.com/avfs/avfs
https://github.com/C2FO/vfs
https://github.com/spf13/afero
https://github.com/peterbourgon/diskv
https://github.com/buger/goreplay/
https://kgithub.com/didi/sharingan
分析网站使用的技术
https://github.com/rverton/webanalyze/
https://github.com/wappalyzer/wappalyzer
可视化 Go 程序的调用图
https://github.com/TrueFurby/go-callvis
github.com/liamylian/json-hashids
序列化自动加密
https://github.com/twpayne/go-jsonstruct
facebook开源的新的go语言orm模块,An entity framework for Go
Simple, yet powerful ORM for modeling and querying data.
https://github.com/facebookincubator/ent
https://github.com/google/wire
Wire 可以生成 Go 源码并在编译期完成依赖注入。它不需要反射机制或 Service Locators
好处:
Compile-time Dependency Injection With Go Cloud's Wire
运行时依赖注入
https://github.com/uber-go/dig
运行时依赖注入
https://github.com/facebookarchive/inject
https://github.com/iancoleman/strcase
https://github.com/fyne-io/fyne 24.9k
Cross platform GUI in Go based on Material Design https://fyne.io/
跨平台支持手机端: go run -tags mobile main.go
fyne package -os android -appID my.domain.appname
fyne install -os android
fyne package -os ios -appID my.domain.appname
fyne package -os iossimulator -appID my.domain.appname
支持web
If you’re using an older version of Go (<1.16), you should install fyne using
go get fyne.io/fyne/v2/cmd/fyne
go install fyne.io/fyne/v2/cmd/fyne@latest
fyne serve # 在线运行
fyne package -os web
go install fyne.io/fyne/v2/cmd/fyne@latest
fyne package -os darwin -icon myapp.png
fyne package -os linux -icon myapp.png
fyne package -os windows -icon myapp.png
# 在当前系统安装应用
fyne install -icon myapp.png
请注意,默认情况下,Windows应用程序是从命令提示符加载的,这意味着,如果单击图标,则可能会看到命令窗口。 要解决此问题,请在运行或构建命令中添加参数-ldflags="-H windowsgui"
。
避免出现控制台:
go run -ldflags="-H windowsgui"
Prerequisites
https://fyne.io/develop/
Windows
环境安装
go get fyne.io/fyne/v2@latest
(or, if using Go before 1.16, then go get fyne.io/fyne/v2
)https://github.com/wailsapp/wails/ 25k
go install github.com/wailsapp/wails/v2/cmd/wails@latest
C/S模式,一个后端服务,一个前端页面作为UI。前端可以使用 Vue / React / Angular,可以说很适合偏前端的选手。
环境安装: https://wails.io/docs/gettingstarted/installation/
可以运行
wails doctor
检查环境
创建项目: https://wails.io/docs/gettingstarted/firstproject
vue+ts: wails init -n myproject -t vue-ts
编译运行: https://wails.io/docs/guides/manual-builds/
wails build
or wails dev
参数:
https://wails.io/docs/reference/cli/#build
https://github.com/gioui/gio 1.7k
https://git.sr.ht/~eliasnaur/gio
https://gioui.org/
支持移动端和web端
go install gioui.org/cmd/gogio@latest
默认使用appicon.png做icon
开发:
go get gioui.org
https://github.com/webview/webview 12.6k
Tiny cross-platform webview library for C/C++. Uses WebKit (GTK/Cocoa) and Edge WebView2 (Windows).
https://github.com/electron/electron C++ 86.5k -> 114k
https://github.com/tauri-apps/tauri rust 7k -> 83.5k
https://github.com/create-go-app/cli
https://github.com/go-nunu/nunu
github.com/Knetic/govaluate
https://github.com/douyu/jupiter
《Go语法树入门》(开源免费图书/Go语言进阶/掌握抽象语法树/Go语言AST/LLVM/LLIR)
https://github.com/chai2010/go-ast-book
变量调试神器
github.com/davecgh/go-spew
wal(Write Ahead Log)
https://github.com/rosedblabs/wal
https://github.com/tidwall/wal
log接口
https://github.com/rs/zerolog
https://github.com/uber-go/zap
日志分割/日志滚动
https://github.com/natefinch/lumberjack
monibuca 丰富的内置插件提供了流媒体服务器的常见功能,例如rtmp server、http-flv、视频录制、QoS等
https://github.com/Monibuca 100star
Darwin Streaming Server 没有维护了?
https://github.com/macosforge/dss
高性能开源RTSP流媒体服务器,基于go语言研发,维护和优化:RTSP推模式转发、RTSP拉模式转发、录像、检索、回放、关键帧缓存、秒开画面、RESTful接口、WEB后台管理、分布式负载均衡
https://github.com/EasyDarwin/EasyDarwin 4k
https://github.com/pion/ion
https://github.com/pion/webrtc
IOT 平台?
https://github.com/nats-io/nats-streaming-server
https://github.com/chromedp/chromedp
https://www.toutiao.com/i6843024797073408515
https://github.com/mgechev/revive
它是一个 Golang 命令行工具,它根据目标源文件的功能和方法签名生成表驱动测试。
https://github.com/cweill/gotests
https://lrita.github.io/2019/06/06/golang-graceful-upgrade/#so_reuseport-%E5%A4%9A%E8%BF%9B%E7%A8%8B
他是一个 gRPC 的 Web 页面调试工具,提供交互式的调试界面。
https://github.com/fullstorydev/grpcui
grpcui -plaintext 127.0.0.1:9901
127.0.0.1:9901 是grpc server的地址
GUI Client for GRPC Services
https://github.com/uw-labs/bloomrpc
GraphQL、REST和gRPC的开源、跨平台API客户端。
https://github.com/Kong/insomnia
package dingtalk
import (
"bytes"
"encoding/json"
"errors"
"io/ioutil"
"net/http"
"strconv"
"time"
)
// SendMessage 发送钉钉机器人消息
func SendMessage(url, message string, ats ...string) (respContent string, err error) {
c := &http.Client{
Timeout: time.Second * 30,
}
data := map[string]interface{}{
"msgtype": "text",
"text": map[string]string{"content": message},
}
if len(ats) != 0 {
isAtAll := false
atMobiles := []string{}
for i := range ats {
if ats[i] == "all" {
isAtAll = true
} else {
atMobiles = append(atMobiles, ats[i])
}
}
data["at"] = map[string]interface{}{
"isAtAll": isAtAll,
"atMobiles": atMobiles,
}
}
b, _ := json.Marshal(data)
resp, err := c.Post(url, "application/json", bytes.NewReader(b))
if err != nil {
return "post请求失败", err
}
defer resp.Body.Close()
body, _ := ioutil.ReadAll(resp.Body)
if resp.StatusCode == 200 {
return string(body), nil
}
return string(body), errors.New(strconv.Itoa(resp.StatusCode))
}
package main
import (
"fmt"
"dingtalk"
)
func main() {
result, err := dingtalk.SendMessage("https://oapi.dingtalk.com/robot/send?access_token=XXXXXX", "测试消息通知", "all")
if err != nil {
fmt.Println("发送失败", result)
return
}
fmt.Println("发送成功", result)
}
从Go到JavaScript的编译器,用于在浏览器中运行Go代码
https://github.com/gopherjs/gopherjs
https://github.com/hexops/vecty
https://github.com/evanw/esbuild
这是一个 JavaScript 打包和压缩程序。它用于打包 JavaScript 和 TypeScript 代码以在网络上分发。
我目前有两个基准测试用于衡量 esbuild 的性能。对于这些基准测试,esbuild 比我测试的其他 JavaScript 打包程序 快至少 100 倍 :https://docs.breword.com/evanw-esbuild/
类似工具:
使用者:
Vite,snowpack使用了esbuild
esbuild 不可能替代 webpack、parcel 等构建工具
https://github.com/augmentable-dev/askgit
也可以跑docker
docker run -v F:/github/openjw/openself:/repo:ro augmentable/askgit "SELECT * FROM commits"
将 markdown 中的 go 代码块进行格式化。
https://github.com/po3rin/gofmtmd
https://github.com/JohannesKaufmann/html-to-markdown
golang将静态资源文件打包进二进制文件
golang1.16内嵌静态资源指南
https://github.com/golang/proposal/blob/master/design/draft-embed.md#background
2020 年 10 月 30 日,Russ Cox 提交了最终的实现:[cmd/go: add //go:embed support](cmd/go: add //go:embed support),意味着你在 tip 版本可以试用该功能了。Go1.16 版本会包含该功能。
Embed 设计提案:https://github.com/golang/proposal/blob/master/design/draft-embed.md
示例参考:https://github.com/mattn/go-embed-example
tip 相关文档:https://tip.golang.org
https://github.com/go-bindata/go-bindata
https://github.com/iawia002/annie
https://github.com/iawia002/lux
抖音 https://www.douyin.com ✓
哔哩哔哩 https://www.bilibili.com ✓ ✓ ✓
半次元 https://bcy.net ✓
pixivision https://www.pixivision.net ✓
优酷 https://www.youku.com ✓ ✓
YouTube https://www.youtube.com ✓ ✓
爱奇艺 https://www.iqiyi.com ✓
芒果TV https://www.mgtv.com ✓
糖豆广场舞 http://www.tangdou.com ✓ ✓
Tumblr https://www.tumblr.com ✓ ✓
Vimeo https://vimeo.com ✓
Facebook https://facebook.com ✓
斗鱼视频 https://v.douyu.com ✓
秒拍 https://www.miaopai.com ✓
微博 https://weibo.com ✓
Instagram https://www.instagram.com ✓ ✓
Twitter https://twitter.com ✓
腾讯视频 https://v.qq.com ✓
网易云音乐 https://music.163.com ✓
音悦台 https://yinyuetai.com ✓
极客时间 https://time.geekbang.org ✓
Pornhub https://pornhub.com ✓
XVIDEOS https://xvideos.com ✓
聯合新聞網 https://udn.com ✓
TikTok https://www.tiktok.com ✓
好看视频 https://haokan.baidu.com ✓
https://api.godoc.org/search?q=etcd
https://github.com/clearcodecn/gosearch
https://github.com/owenthereal/goup
goc 是专为 Go 语言打造的一个综合覆盖率收集系统,尤其适合复杂的测试场景,比如系统测试时的代码覆盖率收集以及精准测试。
https://github.com/qiniu/goc
go tool cover -mode=count -var=CoverageVariableName xxxx.go
> 相信大家一定见过表示go覆盖率结果的coverprofile数据,类似下面: github.com/qiniu/goc/goc.go:21.13,23.2 1 1
其基本语义为 "文件:起始行.起始列,结束行.结束列 该基本块中的语句数量 该基本块被执行到的次数"
[聊聊 Go 代码覆盖率技术与最佳实践](https://xie.infoq.cn/article/ca1cc8ba293eddf793b3b0613)
go原生plugin
github.com/hashicorp/go-plugin
https://github.com/nikolaydubina/go-recipes?tab=readme-ov-file#dependencies
golang.org/x/exp/cmd/modgraphviz
https://github.com/nikolaydubina/go-recipes?tab=readme-ov-file#-make-graph-of-upstream-modules-with-modgraphviz
github.com/PaulXu-cn/go-mod-graph-chart/gmchart
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/go/README.md
https://github.com/blushft/go-diagrams
可以生成graphviz DOT file
go-diagrams实现了diagrams的部分接口
https://github.com/cshum/imagor
Imagor 是一个用 Go 编写的快速、支持 Docker 的图像处理服务器。
Imagor 使用最高效的图像处理库 libvips 之一(使用govips)。它通常比使用最快的 ImageMagick 和 GraphicsMagick 设置快4-8倍。
Imagor 是一个易于扩展的 Go 库,可以在任何 Unix 环境中安装和使用,并且可以使用 Docker 进行容器化。
Imagor 采用Thumbor URL 语法,涵盖了大多数 Web 图像处理用例。如果这些符合您的要求,Imagor 将是一种轻便、高性能的替代品。
https://github.com/vdobler/chart 695
https://github.com/Arafatk/glot 354
https://github.com/wcharczuk/go-chart 3.2k
https://github.com/gonum/plot 2k
https://github.com/ScottPlot/ScottPlot 1.1k
https://github.com/oxyplot/oxyplot 2.4k
https://github.com/Live-Charts/Live-Charts 5k
https://github.com/beto-rodriguez/LiveCharts2 771
https://github.com/microcharts-dotnet/Microcharts 1.6k
交互式绘图工具
http://www.gnuplot.info/
你可以在c#程序中编写数据文件,从c#调用gnuplot可执行文件,并在c#图片框中显示生成的图像。
https://github.com/topics/websocket?l=go
https://github.com/centrifugal/centrifugo
(WebSocket, HTTP-streaming, SSE/EventSource, GRPC, SockJS, WebTransport)
https://github.com/olahol/melody
https://github.com/shirou/gopsutil
psutil是一个开源且跨平台的库,其提供了便利的函数用来获取操作系统的信息,如cpu、内存、磁盘、网络等信息。此外,psutil还可以用来进行进程管理,包括判断进程是否存在、获取进程列表、获取进程的详细信息等。psutil广泛应用于系统监控、进程管理、资源限制等场景。此外,psutil还提供了许多命令行工具提供的功能,包括ps, top, lsof, netstat, ifconfig, who, df, kill, free, nice, ionice, iostat, iotop, uptime, pidof, tty, taskset, pmap。有python、go、rust等版本。
https://github.com/petersunbag/coven 27 (有用于生产经验)
这个也是很NB
https://github.com/jinzhu/copier 5.5k (有用于生产经验)
https://github.com/imdario/mergo 2.9k
https://github.com/mitchellh/mapstructure 7.9k
https://github.com/zyedidia/generic
array2d: a 2-dimensional array.
avl: an AVL tree.
bimap: a bi-directional map; a map that allows lookups on both keys and values.
btree: a B-tree.
cache: a wrapper around map[K]V that uses a maximum size and evicts elements using LRU when full.
hashmap: a hashmap with linear probing. The main feature is that the hashmap can be efficiently copied, using copy-on-write under the hood.
hashset: a hashset that uses the hashmap as the underlying storage.
heap: a binary heap.
interval: an interval tree, implemented as an augmented AVL tree.
list: a doubly-linked list.
mapset: a set that uses Go's built-in map as the underlying storage.
multimap: an associative container that permits multiple entries with the same key.
queue: a First In First Out (FIFO) queue.
rope: a generic rope, which is similar to an array but supports efficient insertion and deletion from anywhere in the array. Ropes are typically used for arrays of bytes, but this rope is generic.
prope: a persistent version of the rope, which allows for keeping different versions of the rope with only a little extra time or memory.
stack: a LIFO stack.
trie: a ternary search trie.
ulist: an un-rolled doubly-linked list.
https://github.com/gotenberg/gotenberg
https://github.com/smallnest/exp/
https://github.com/duke-git/lancet
github.com/dchest/siphash
github.com/cespare/xxhash/v2
github.com/spf13/cast
github.com/com/google/go-cmp/cmp
github.com/com/google/go-cmp
github.com/kr/pretty
https://github.com/sirkon/go-format 功能最强大
type t struct {
A string
Field int
}
var s = t{
A: "str",
Field: 12,
}
var d struct {
F t
Entry float64
}
d.F = s
d.Entry = 0.5
res := format.Formatg("${F.A} ${F.Field} $Entry", d)
// res = "str 12 0.500000"
https://github.com/chonla/format
var params = map[string]interface{}{
"sister": "Susan",
"brother": "Louis",
}
format.Printf("%<brother> loves %<sister>.", params)
https://github.com/go-ffmt/ffmt/blob/master/format.go
Format("hello {name}", "ffmt") to "hello ffmt"
https://github.com/Wissance/stringFormatter
FormatComplex("Hello {user} what are you doing here {app} ?", map[string]interface{}{"user":"vpupkin", "app":"mn_console"})
https://github.com/delicb/gstring
// outpits "Bracket {, }, key value, key value, key value"
gstring.Printm("Bracket {{, }}, {key}, {key}, {key}", map[string]interaface{}{"key", "key value"})
https://github.com/mgenware/go-string-format
strf.Format("🐆{0} {1} {0}", "leopard", -3)
// returns "🐆leopard -3 leopard"
https://github.com/taloric/strfmt
format_string := "Today is a {what} {desc}"
args := make(map[string]string)
args["what"] = "wonderful"
args["desc"] = "day"
res,err := strfmt.FormatMap(format_string, &args)
fmt.Println(res)
https://github.com/yangchenxing/go-string-mapformatter
fmt.Println(mapformatter.MustFormat("Hello %(name|s), you owe me %(money|.2f) dollar.",
map[string]interface{}{
"name": "anyone",
"money": 10.3,
}))
// Output: Hello anyone, you owe me 10.30 dollar.
https://github.com/christianhujer/tfmt/blob/master/main.go
https://golangdocs.com/string-formatting-in-golang
https://zetcode.com/golang/string-format/
https://github.com/dustin/go-humanize
fmt.Printf("That file is %s.", humanize.Bytes(82854982)) // That file is 83 MB.
fmt.Printf("This was touched %s.", humanize.Time(someTimeInstance)) // This was touched 7 hours ago.
支持多种格式
https://github.com/araddon/dateparse
https://github.com/WnP/go-sfmt
https://github.com/jetpack-io/devbox
https://github.com/golang-migrate/migrate
https://github.com/prabhatsharma/zinc
https://github.com/augmentable-dev/gitqlite
https://github.com/inbucket/inbucket
massh:通过 SSH 方式运行 Linux 分布式 Shell 命令。
sh: 一个支持 Bash 的 Shell 解析器、格式化器。