Keen的博客

记录所思、所想、所遇

欢迎来到我的个人站~


Go学习笔记

1. CentOS安装go

参考:https://www.jianshu.com/p/c314f2edf19e

2. 标准库

Go语言标准库包名 功 能
bufio 带缓冲的 I/O 操作
bytes 实现字节操作
container 封装堆、列表和环形列表等容器
crypto 加密算法
database 数据库驱动和接口
debug 各种调试文件格式访问及调试功能
encoding 常见算法如 JSON、XML、Base64 等
flag 命令行解析
fmt 格式化操作
go Go 语言的词法、语法树、类型等。可通过这个包进行代码信息提取和修改
html HTML 转义及模板系统
image 常见图形格式的访问及生成
io 实现 I/O 原始访问接口及访问封装
math 数学库
net 网络库,支持 Socket、HTTP、邮件、RPC、SMTP 等
os 操作系统平台不依赖平台操作封装
path 兼容各操作系统的路径操作实用函数
plugin Go 1.7 加入的插件系统。支持将代码编译为插件,按需加载
reflect 语言反射支持。可以动态获得代码中的类型信息,获取和修改变量的值
regexp 正则表达式封装
runtime 运行时接口
sort 排序接口
strings 字符串转换、解析及实用函数
time 时间接口
text 文本模板及 Token 词法器

3. 转义符

转移符 含 义
\r 回车符(返回行首)
\n 换行符(直接跳到下一行的同列位置)
\t 制表符
' 单引号
" 双引号
\ 反斜杠

4. 格式化

动 词 功 能
%v 按值的本来值输出
%+v 在 %v 基础上,对结构体字段名和值进行展开
%#v 输出 Go 语言语法格式的值
%T 输出 Go 语言语法格式的类型和值
%% 输出 % 本体
%b 整型以二进制方式显示
%o 整型以八进制方式显示
%d 整型以十进制方式显示
%x 整型以十六进制方式显示
%X 整型以十六进制、字母大写方式显示
%U Unicode 字符
%f 浮点数
%p 指针,十六进制方式显示

5. Go struct的Tag的作用

(1)可以用来在序列化和反序列化中,用某种名称

type User struct {
    UserId   int    `json:"user_id" bson:"user_id"`		// json表示用于配置文件json类型中,bson表示mongdb中
    UserName string `json:"user_name" bson:"user_name"`
}

u := &User{UserId: 1, UserName: "tony"}
j, _ := json.Marshal(u)
fmt.Println(string(j))
// 输出内容:{"user_id":1,"user_name":"tony"}

// 如果不用tag,则输出内容:{"UserId":1,"UserName":"tony"}

(2)可以用来类型转换

type Info struct {
    Name string
    Age  int   `json:"age,string"`	// 这样生成的json对象中,age就为字符串
    Sex  string
}

(3)只对配置项中部分进行解析

type stConfig struct {
	Server            	stConfigServer 	`toml:"server"`
	Data				stConfigQuery	`toml:"config_query"`
}

var conf stConfig
GetConfigStructEx("service.conf", conf)

// service.conf
[queue]
beanstalk_ipport = "127.0.0.1:xxxxx"
beanstalk_reserve_timeout = 5

[server]
conn_sc_ipport_server = "0.0.0.0:xxxxx"
conn_sc_ipport_client = "127.0.0.1:xxxxx"
conn_sc_min_gzip_length = xxx

[config_query]
PushQps =300
StrategyRetrySnd = 600
TTRSnd = 1800

6. defer, panic, recover的使用

这里类似c++的try catch异常捕获,一般使用方法如下:

package main
 
import "fmt"
 
func main(){
    defer func(){ // 必须要先声明defer,否则不能捕获到panic异常
        fmt.Println("c")
        if err:=recover();err!=nil{
            fmt.Println(err) // 这里的err其实就是panic传入的内容,55
        }
        fmt.Println("d")
    }()
    f()
}
 
func f(){
    fmt.Println("a")
    panic(55)
    fmt.Println("b")
    fmt.Println("f")
}
输出结果
a
c
55
d
exit code 0, process exited normally.

7. zip解压

import (
    "io"
    "os"
    "archive/zip"
    "path/filepath"
)

func UnZip(archive, target_dir string) error {
	reader, err := zip.OpenReader(archive)
	if err != nil {
		return err
	}

	if err := os.MkdirAll(target_dir, 755); err != nil {
		return err
	}

	for _, file := range reader.File {
		path := filepath.Join(target_dir, file.Name)
		if file.FileInfo().IsDir() {
			os.MkdirAll(path, file.Mode())
			continue
		}

		fileReader, err := file.Open()
		if err != nil {
			return err
		}
		defer fileReader.Close()

		targetFile, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, file.Mode())
		if err != nil {
			return err
		}
		defer targetFile.Close()

		if _, err := io.Copy(targetFile, fileReader); err != nil {
			return err
		}
	}

	return nil
}

8. http下载

import (
	"io"
	"net/http"
	"os"
)

// Download 针对小文件的下载(建议10M以内),根据链接下载文件,下载到dstPath目录。
func Download(url, dstPath string) error {
	resp, err := http.Get(url)
	if err != nil {
		return err
	}

	if IsPathExist(dstPath) {
		os.RemoveAll(dstPath)
	}

	f, err := os.Create(dstPath)
	if err != nil {
		return err
	}
	defer f.Close()

	io.Copy(f, resp.Body)

	return nil
}

// DownloadBigFile 下载大文件,根据链接下载文件,下载到dstPath。
func DownloadBigFile(url, dstPath string, onProgress func(err error, totalLen, downloadLen int64)) error {
	client := new(http.Client)
	resp, err := client.Get(url)
	if err != nil {
		return err
	}

	// 读取服务器返回的文件大小
	fsize, err := strconv.ParseInt(resp.Header.Get("Content-Length"), 10, 32)
	if err != nil {
		return err
	}

	if fsize == 0 {
		return errors.New("文件大小为0")
	}

	// 创建文件
	file, err := os.Create(dstPath)
	if err != nil {
		return err
	}
	defer file.Close()

	var downloadLen int64

	buf := make([]byte, 8192)

	for {
		readSize, err := resp.Body.Read(buf)
		if readSize == 0 {
			break
		}

		file.Write(buf[:readSize])

		downloadLen += int64(readSize)

		if onProgress != nil {
			onProgress(nil, fsize, downloadLen)
		}

		if err != nil {
			break
		}
	}

	if downloadLen < fsize {
		err = errors.New("下载未完成")
		onProgress(err, fsize, downloadLen)
	}

	return err
}

9. go管道与shell程序通信

package main

import (
	"bufio"
	"fmt"
	"io"
	"os/exec"
	"strings"
)

func main() {
	xxx_cmd := exec.Command("./xxxxx", "/xxx")
	xxx_In, _ := xxx_cmd.StdinPipe()
	xxx_Out, _ := xxx_cmd.StdoutPipe()

	xxx_cmd.Start()

	scanner := bufio.NewScanner(xxx_Out)

	io.WriteString(xxx_In, "word/00bc595ba1ca523347d86c952972e740\n")
	for scanner.Scan() {
		value := scanner.Text()
		fmt.Println(value)
		if strings.TrimSpace(value) == "====================" {
			break
		}

	}

	io.WriteString(xxx_In, "word/0041811cb9868b31107bfe5a7fdbc57a\n")
	for scanner.Scan() {
		value := scanner.Text()
		fmt.Println(value)
		if strings.TrimSpace(value) == "====================" {
			break
		}

	}
}

10. 多个指令同时运行

简单示例:

cmd1 := exec.Command("ps", "-ef")
cmd2 := exec.Command("grep", "engine_mirror_server")
cmd3 := exec.Command("grep", "-v", "grep")

cmd3.Stdin, _ = cmd2.StdoutPipe()
cmd2.Stdin, _ = cmd1.StdoutPipe()

cmd3.Stdout = os.Stdout
cmd3.Stderr = os.Stderr

cmd3.Start()
cmd2.Start()
cmd1.Run()
cmd2.Wait()

统一函数

func runCombineCommand(args ...string) ([]byte, error) {
	var err error

	var cmdArray []*exec.Cmd
	var arg []string
	var stdout, stderr bytes.Buffer

	for _, v := range args {
		if v != "|" {
			arg = append(arg, v)
		} else {
			cmd := exec.Command(arg[0], arg[1:]...)
			cmdArray = append(cmdArray, cmd)
			arg = arg[:0]
		}
	}
	if len(arg) > 0 {
		cmd := exec.Command(arg[0], arg[1:]...)
		cmdArray = append(cmdArray, cmd)
		arg = arg[:0]
	}

	for i, cmd := range cmdArray {
		if i < len(cmdArray)-1 {
			cmdArray[i+1].Stdin, err = cmd.StdoutPipe()
			if err != nil {
				return nil, err
			}
		} else {
			cmd.Stdout = &stdout
			cmd.Stderr = &stderr
			break
		}
	}

	for i, cmd := range cmdArray {
		if i == 0 {
			continue
		} else if i == len(cmdArray)-1 {
			err = cmd.Start()
			if err != nil {
				return nil, err
			}
			err = cmdArray[0].Run()
			if err != nil {
				return nil, err
			}
			err = cmd.Wait()
			if err != nil {
				return nil, err
			}
		} else {
			err = cmd.Start()
			if err != nil {
				return nil, err
			}
		}
	}

	return stdout.Bytes(), nil
}

b, _ := runCombineCommand("ps", "-ef", "|", "grep", "engine_mirror_server", "|", "grep", "-v", "grep", "|", "awk", "{printf $2}")
fmt.Printf("result: %s", string(b))

11. 清空数组方式

letters := []string{"a", "b", "c", "d"}

// 直接在原 slice 上操作,故无 GC 行为
// 清空后 cap 值和之前相同,len 值清零
letters = letters[:0]

// 类似 C 语言中赋值空指针,原内容会被 GC 处理
// 清空后 cap 值清零,len 值清零
letters = nil

12. 大文件MD5

// CaclBigFileMD5 计算大文件MD5
func CaclBigFileMD5(filePath string) []byte {
	hash := md5.New()

	file, err := os.Open(filePath)
	if err != nil {
		return nil
	}

	for {
		buf := make([]byte, 8192)

		readSize, err := file.Read(buf)
		if err != nil || readSize == 0 {
			break
		}

		hash.Write(buf[:readSize])
	}

	return hash.Sum(nil)
}

13. 平台编译问题

有些特定库是在特定平台才能使用例如syscall.Dup2等只在linux下生效因此需要使用不同平台不同编译可以用go文件的头部tag来判定用哪个如下

log_panic_win.go

// +build !linux

package test

// InitPanicFile def
func InitPanicFile(panicFile string) error {

	return nil
}

log_panic_linux.go

// +build linux

package test

// InitPanicFile def
func InitPanicFile(panicFile string) error {
	xxxxxxxxxx
	return nil
}

14. AES加解密

// AESEncrypt AES加密算法
func AESEncrypt(secret, data []byte) ([]byte, error) {
	output := make([]byte, aes.BlockSize+len(data))
	iv := output[:aes.BlockSize]
	encrypted := output[aes.BlockSize:]

	if _, err := io.ReadFull(rand.Reader, iv); err != nil {
		return nil, err
	}

	tmp := sha256.Sum256(secret)
	key := tmp[0:]
	keyblock, err := aes.NewCipher(key)
	if err != nil {
		return nil, err
	}
	stream := cipher.NewCFBEncrypter(keyblock, iv)
	stream.XORKeyStream(encrypted, data)

	return output, nil
}

// AESDecrypt AES解密算法
func AESDecrypt(secret, data []byte) ([]byte, error) {
	if len(data) < aes.BlockSize {
		return nil, errors.New("加密数据太短")
	}
	output := make([]byte, len(data))
	copy(output, data)
	iv := output[:aes.BlockSize]
	encrypted := output[aes.BlockSize:]

	tmp := sha256.Sum256(secret)
	key := tmp[0:]
	keyblock, err := aes.NewCipher(key)
	if err != nil {
		return nil, err
	}

	stream := cipher.NewCFBDecrypter(keyblock, iv)
	stream.XORKeyStream(encrypted, encrypted)
	return encrypted, nil
}

15. 拷贝大文件

// CopyFile 拷贝任意大小文件。内部按照块拷贝,所以不在乎文件有多大
func CopyFile(src, dst string) error {
	if !IsPathExist(src) {
		return errors.New("src不存在")
	}

	srcFile, err := os.Open(src)
	if err != nil {
		return err
	}
	defer srcFile.Close()

	if IsPathExist(dst) {
		os.Readlink(dst)
	}
	dstFile, err := os.Create(dst)
	if err != nil {
		return err
	}
	defer dstFile.Close()

	buf := make([]byte, 8192)

	for {
		readSize, err := srcFile.Read(buf)
		if readSize == 0 {
			break
		}

		dstFile.Write(buf[:readSize])

		if err == io.EOF {
			return nil
		}

		if err != nil {
			return err
		}
	}

	return err
}

16. 一些文件的操作

// GetFileSize 获取文件大小
func GetFileSize(path string) int64 {
	st, err := os.Stat(path)
	if err != nil || st.IsDir() == true {
		return -1
	}

	return st.Size()
}

// IsPathExist 判断文件或者目录是否存在
func IsPathExist(path string) bool {
	_, err := os.Stat(path)
	return err == nil || os.IsExist(err)
}

// IsFileCanExec 判断文件是否拥有可执行权限
func IsFileCanExec(path string) bool {
	fileInfo, err := os.Stat(path)
	if err != nil {
		return false
	}

	return uint32(73) == uint32(fileInfo.Mode().Perm()&os.FileMode(0111))
}

17. boltdb基本操作

package main

import (
	"errors"
	"fsslog"
	"helper"
	"path/filepath"
	"time"

	"github.com/boltdb/bolt"
)

// BoltDBReader 存储所有注册的pid和uid,为查找校验做准备
type BoltDBReader struct {
	db     *bolt.DB
	secret string
}

// Load 加载db。如果db不存在,则创建db,数据通过密码secret加密。取出的时候,也需要通过密码取出来
func (b *BoltDBReader) Load(secret string) error {
	db, err := bolt.Open(filepath.Join(".", "sync_cfg.db"), 0600, &bolt.Options{Timeout: 5 * time.Second})
	if err != nil {
		fsslog.LogError.Printf("读取或者创建db失败, err: %s", err)
		return err
	}
	b.secret = secret
	b.db = db
	return nil
}

// UnLoad 卸载
func (b *BoltDBReader) UnLoad() {
	b.db.Close()
}

// QueryUID 通过pid,查询uid
func (b *BoltDBReader) QueryUID(pid string) string {
	uid := ""
	b.db.View(func(tx *bolt.Tx) error {
		bk := tx.Bucket([]byte("pid_uid"))
		if bk == nil {
			return errors.New("bucket not exist")
		}
		v := bk.Get([]byte(pid))
		out, err := helper.AESDecrypt([]byte(b.secret), v)
		if err != nil {
			return err
		}
		uid = string(out)
		return nil
	})
	return uid
}

// WriteUID 设置PID UID
func (b *BoltDBReader) WriteUID(pid, uid string) error {
	var err error
	b.db.Update(func(tx *bolt.Tx) error {
		bk, err1 := tx.CreateBucketIfNotExists([]byte("pid_uid"))
		if err1 != nil {
			err = err1
			return err1
		}
		if bk == nil {
			err = errors.New("无效bucket")
			return err
		}
		data, err1 := helper.AESEncrypt([]byte(b.secret), []byte(uid))
		if err1 != nil {
			err = err1
		}
		err = bk.Put([]byte(pid), data)
		return err
	})
	return err
}

18. httprouter搭建的httpserver

// 注册文件服务器、Get方法、Post方法
router = httprouter.New()

// 使用http请求 + download + 特定的文件(例如:http://127.0.0.1:21200/download/test.txt),可以实现具体文件的下载,文件映射到/data/test.txt
router.ServeFiles("/download/*filepath", http.Dir("/data/"))

// 使用http请求 + info + 特定信息,获取数据
router.GET("/info/*engType", info)
router.POST("/upload", upload)

addr := "0.0.0.0:" + 21200
http.ListenAndServe(addr, router)

func  info(w http.ResponseWriter, r *http.Request, p httprouter.Params) {
	fmt.Printf("remoteaddr: %s, requesturi: %s", r.RemoteAddr, r.RequestURI)
	local := false
	if r.RemoteAddr != "" {
		addr := strings.Split(r.RemoteAddr, ":")
		if len(addr) > 0 && addr[0] == "127.0.0.1" {
			local = true
		}
	}

	w.Header().Add("Content-Type", "application/json; charset=utf-8")
	engType := p.ByName("engType")	// 通过请求的具体名字,来干对应的事情
	fmt.Println(engType)

	out := `{}`

	......

	w.Write([]byte(out))
}

func  upload(w http.ResponseWriter, r *http.Request, p httprouter.Params) {
	fmt.Printf("upload now!")
	w.Header().Add("Content-Type", "application/json; charset=utf-8")
	file, head, err := r.FormFile("UploadFile")	// form表格形式提交的文件,采用这种方法
	fmt.Printf("收到上传文件,文件名:%s,大小:%d B", head.Filename, head.Size)
	defer file.Close()

	if err == nil {
		// 从文件读取,然后存储
		var downloadLen int64
		buf := make([]byte, 8192)

		newPath := filepath.Join("/tmp", head.Filename)
		if helper.IsPathExist(newPath) {
			os.Remove(newPath)
		}
		newFile, err := os.Create(newPath)
		defer os.Remove(newPath)

		defer newFile.Close()

		if err == nil {
			for {
				readSize, err := file.Read(buf)
				if readSize == 0 {
					break
				}

				newFile.Write(buf[:readSize])
				downloadLen += int64(readSize)

				if err != nil {
					break
				}
			}
		}
	}

	out, _ := json.Marshal(resp)
	fmt.Printf("%s", string(out))
	w.Write(out)
}

19. http请求post方法。有带签名校验的

import (
	...
	
	"github.com/tidwall/sjson"
	"github.com/tidwall/gjson"
)

// caclSign 计算签名
func caclSign(version, randn int, pid, uid string) string {
	signBuf := fmt.Sprintf("version%dpid%srandn%duid%s", version, pid, randn, uid)
	sign := fmt.Sprintf("%x", md5.Sum([]byte(signBuf)))
	return sign
}

// 请求
func request() (string, error) {
	body := `{
		"version": 0,
		"pid": "",
		"randn": 0,
		"sign": ""
	}`
	randn := rand.New(rand.NewSource(time.Now().UnixNano())).Intn(100000)
	sign := caclSign(version, randn, pid, uid)
	body, _ = sjson.Set(body, "version", version)
	body, _ = sjson.Set(body, "pid", pid)
	body, _ = sjson.Set(body, "randn", randn)
	body, _ = sjson.Set(body, "sign", sign)
	fmt.Printf("request: %s", body)
	req, err := http.NewRequest("POST", url, bytes.NewBufferString(body))
	if err != nil {
		return "", err
	}
	req.Header.Set("Content-Type", "application/json")

	client := &http.Client{}
	resp, err := client.Do(req)
	if err != nil {
		return "", err
	}
	defer resp.Body.Close()

	buf, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		return "", err
	}
	respdata := string(buf)
	retcode := gjson.Get(respdata, "code").Int()
	retmsg := gjson.Get(respdata, "msg").String()
	fmt.Printf("response: %s", respdata)
}

20. http请求post表单

import (
	"net/http"
	"net/url"
)


// 发送post请求
var dataValue string
if u.autoUpdate {
	dataValue = "1"
} else {
	dataValue = "0"
}
http.PostForm("http://xxx.xxx.xxx:xxxx/xxxx", url.Values{"DataType": {"AutoUpdateSwitch"}, "DataValue": {dataValue}})

21. https请求,免SSL证书

import (
	...
	"crypto/tls"
	...
)

...

req, err := http.NewRequest("POST", "https://xxxxxxxxx", bytes.NewBufferString(body))
if err != nil {
	fmt.Printf("蓝鲸邮件发送失败, err: %s", err)
	return err
}
req.Header.Set("Content-Type", "application/json")

client := &http.Client{Transport: &http.Transport{
	TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}}
resp, err := client.Do(req)
if err != nil {
	fmt.Printf("蓝鲸邮件发送失败, err: %s", err)
	return err
}
defer resp.Body.Close()

...

22. smtp邮件服务


maillist := make([]string, 1)
maillist[0] = "xxx@xxx"
msg := "xxxx"

UserEmail = "xxxxxx@qq.com"
Passwd = "xxxxxxx"
SmtpHost = "smtp.qq.com"
SmtpPort = "587"
NickName = "消息告警"

auth := smtp.PlainAuth("", UserEmail, Passwd, SmtpHost)
contentType := "xxxx"

err := smtp.SendMail(SmtpHost+":"+SmtpPort, auth, UserEmail, maillist, msg)
return err

23. elastic操作

// 裸http操作
func (e *ESHandler) QueryAssets(assetType string) ([]*AssetInfo, error) {
	txlog.LogInfo.Printf("query es assets. assetType:%s", assetType)
	param := `{
		"query" : {
			"match": {
				"AssetType": ""
			}
		},  
		"_source": []
	}`
	param, _ = sjson.Set(param, "query.match.AssetType", assetType)
	_source := []string{"AssetType","AssetVersion","Region","Uid"}
	for _, item := range datadef.CspmCfg.AssetInstIdName {
		_source = append(_source, item.Name)
	}
	param, _ = sjson.Set(param, "_source", _source)
	_url := fmt.Sprintf("http://%s:%d/assets/_search", datadef.CspmCfg.Elasticsearch.Server, datadef.CspmCfg.Elasticsearch.Port)
	res, err := netopt.CurlPostJson(_url, param)
	if err != nil {
		return nil, err
	}
	total := gjson.Get(res, "hits.total").Int()
	if total == 0 {
		return nil, nil
	}
	hits := gjson.Get(res, "hits.hits").Array()
	out := []*AssetInfo{}
	for _, item := range hits {
		assetInfo := &AssetInfo{
			Uid: item.Get("_source.Uid").String(),
			AssetType: item.Get("_source.AssetType").String(),
			AssetVersion: item.Get("_source.AssetVersion").String(),
			Region: item.Get("_source.Region").String(),
		}
		for _, n := range datadef.CspmCfg.AssetInstIdName {
			if n.Type != "" && n.Type != assetType {
				continue
			}
			assetInfo.InstanceId = item.Get(n.Name).String()
		}

		if assetInfo.InstanceId == "" {
			return nil, errors.New("no instanceid name")
		}

		out = append(out, assetInfo)
	}
	return out, nil
}
// elastic库操作
package main

import (
	"github.com/olivere/elastic"
	"fmt"
	"context"
	"io"
)

func main() {
	// 创建一个连接
	client, err := elastic.NewClient(elastic.SetURL("http://127.0.0.1:9200"))
	if err != nil {
		fmt.Println("err: ", err)
		return
	}

	info, code, err := client.Ping("http://127.0.0.1:9200").Do(context.Background())
	if err != nil {
		fmt.Println("err: ", err)
		return
	}
	fmt.Printf("ES returned with code %d and info %v\n", code, info)

	esVersion, err := client.ElasticsearchVersion("http://127.0.0.1:9200")
	if err != nil {
		fmt.Print("err: ", err)
		return
	}
	fmt.Printf("Es version: %s\n", esVersion)

	// 创建index
	put, err := client.Index().Index("keen_index").Type("keen_type").Id("keen_id1").BodyJson(`{"keen_key1": "keen_value1", "keen_key2": {"key1": "value1"}}`).Do(context.Background())
	if err != nil {
		fmt.Println("err: ", err)
		return
	}
	fmt.Printf("create: %v\n", put)

	client.Index().Index("keen_index").Type("keen_type").Id("keen_id2").BodyJson(`{"keen_key1": "keen_value1", "keen_key2": {"key1": "value1"}}`).Do(context.Background())
	client.Index().Index("keen_index").Type("keen_type").Id("keen_id3").BodyJson(`{"keen_key1": "keen_value1", "keen_key2": {"key1": "value1"}}`).Do(context.Background())
	client.Index().Index("keen_index").Type("keen_type").Id("keen_id4").BodyJson(`{"keen_key1": "keen_value1", "keen_key2": {"key1": "value1"}}`).Do(context.Background())

	// 增加和修改,如果直接改body,这里可以直接修改。不太适合直接用于body的子项目的修改,只能从根结点开始修改
	update, err := client.Update().Index("keen_index").Type("keen_type").Id("keen_id1").Doc(map[string]interface{}{"key1": "1111"}).Do(context.Background())
	if err != nil {
		fmt.Println("err: ", err)
		return
	}
	fmt.Printf("update: %v\n", update)

	// 查找,必须严格到Id级别的获取
	get, err := client.Get().Index("keen_index").Type("keen_type").Id("keen_id1").Do(context.Background())
	if err != nil {
		fmt.Println("err: ", err)
		return
	}
	fmt.Printf("get: %s\n", get.Source)

	// 搜索
	// 取所有
	query, err := client.Search("keen_index").Type("keen_type").Do(context.Background())
	if err != nil {
		fmt.Println("err: ", err)
		return
	}
	if query.Hits.TotalHits.Value > 0 {
		for _, hit := range query.Hits.Hits {
			fmt.Printf("query, id: %s, type: %s, source: %s\n", hit.Id, hit.Type, hit.Source)
		}
	}
	// 无需严格的条件查询
	query1, err := client.Search("keen_index").Do(context.Background())
	if err != nil {
		fmt.Println("err: ", err)
		return
	}
	if query1.Hits.TotalHits.Value > 0 {
		for _, hit := range query1.Hits.Hits {
			fmt.Printf("query, id: %s, type: %s, source: %s\n", hit.Id, hit.Type, hit.Source)
		}
	}
	// 字段相等,注意,这里只能判定body中一级匹配的情景
	query2 := elastic.NewQueryStringQuery("key1:value1")
	query2_1, err := client.Search("keen_index").Type("keen_type").Query(query2).Do(context.Background())
	if err != nil {
		fmt.Println("err: ", err)
		return
	}
	if query2_1.Hits.TotalHits.Value > 0 {
		for _, hit := range query2_1.Hits.Hits {
			fmt.Printf("query2_1, id: %s, type: %s, source: %s\n", hit.Id, hit.Type, hit.Source)
		}
	}
	// 多条件大size查询
	query3 := elastic.NewBoolQuery()
	query3.Should(elastic.NewMatchQuery("AssetUid", "1234"))
	query3.Should(elastic.NewMatchQuery("AssetUid", "1234"))
	query3_1, err := client.Search("assets").Type("soc").Query(query3).Do(context.Background())
	client.Search("assets").Type("soc").Size(10000).Query(query3).DocvalueFields("key1", "key2").Do(context.Background())
	if err != nil {
		fmt.Println("err: ", err)
		return
	}
	if query3_1.Hits.TotalHits.Value > 0 {
		for _, hit := range query3_1.Hits.Hits {
			fmt.Printf("query3_1, id: %s, type: %s, source: %s\n", hit.Id, hit.Type, hit.Source)
		}
	}
	// 滚动查询
	query4 := client.Scroll("assets").Type("soc").Size(100)
	for {
		query4_1, err := query4.Do(context.Background())
		if err == io.EOF {
			fmt.Println("all query finish.")
			return
		}
		if err != nil {
			fmt.Println("err: ", err)
			return
		}
		if query4_1.Hits.TotalHits.Value > 0 {
			for _, hit := range query4_1.Hits.Hits {
				fmt.Printf("query4_1, id: %s, type: %s, source: %s\n", hit.Id, hit.Type, hit.Source)
			}
		}
	}
}

24. sync.Once只执行一次

var once sync.Once

func initOnce() ObjStruct {
	once.Do(func() {
		// 只执行一次的函数
		g_obj = ObjStruct{}
	})
	return g_obj
}

25. 比较优雅的定时器实现

// 实现1:
go func() {
	timeElapse1 := time.Second * time.Duration(10)
	timer1 := time.NewTimer(timeElapse1)

	timeElapse2 := time.Second * time.Duration(60)
	timer2 := time.NewTimer(timeElapse2)

	for {
		select {
		case <-timer1.C:
			// ToDo

			timer1.Reset(timeElapse2)
		case <-timer2.C:
			// ToDo
			
			timer2.Reset(timeElapse2)
		}
	}
}

// 实现2:
go func() {
	for {
		// ToDo

		next_time := time.After(time.Minute)

		select {
		case <-next_time:
			break
		}
	}
}

// 实现3:
go func() {
	for {
		// ToDo
		
		<-time.After(time.Minute)
	}
}

26. 优雅的使用超时机制

func A() int {
	var retCode int
	
	// ToDo

	ctx, cancelFunc := context.WithTimeout(context.Background(), time.Duration(10) * time.Second)
	defer cancelFunc()

	select {
	case retCode = <-g_resChan:
		// ToDo
	case <-ctx.Done():
		retCode = XXXX_TIMEOUT
	}

	return retCode
}

func B() int {
	g_resChan <- xxx
}

27. 协程池的使用,避免过多的Goroutine导致的资源浪费

因为每一个cpu核只能执行一个goroutine,如果太多goroutine,会导致剩余的goroutine处于等待状态,其实是浪费资源的,因为算力跟不上,开那么多goroutine也没用。

func main() {
	jobsCount := 10
	group := sync.WaitGroup{}
	var jobsChan = make(chan int, 3)
	// 生成指定数目的goroutine,每个goroutine消费jobsChan中的任务
	poolCount := 3
	for i := 0; i < poolCount; i++ {
		go func() {
			for j := range jobsChan {
				// ToDo

				group.Done()
			}
		}
	}

	// 把job依次推送到jobsChan,供goroutine消费
	for i := 0; i < jobsCount; i++ {
		jobsChan <- i
		group.Add(1)
	}

	group.Wait()
	close(jobsChan)
}

打赏一个呗

取消

感谢您的支持,我会继续努力的!

扫码支持
扫码支持
扫码打赏,你说多少就多少