mac 上学习k8s系列（45）openresty操作mysql

golangLeetcode

发布于 2022-08-02 19:43:38

6780

发布于 2022-08-02 19:43:38

Nginx Ingress 由资源对象 Ingress、Ingress 控制器、Nginx 三部分组成，Ingress 控制器用以将 Ingress 资源实例组装成 Nginx 配置文件（nginx.conf），并重新加载 Nginx 使变更的配置生效。当它监听到 Service 中 Pod 变化时通过动态变更的方式实现 Nginx 上游服务器组配置的变更，无须重新加载 Nginx 进程。

修改nginx.conf有三种方式，1，直接修改nginx.conf，2，configMap；3，Ingress；其中configMap可以作为基础模板，它能改动的地方更多，分为下面4个部分

main-snippet string "" 在 main 指令域添加 Nginx 配置指令
http-snippet string "" 在 http 指令域添加 Nginx 配置指令
server-snippet string "" 在 server 指令域添加 Nginx 配置指令
location-snippet string "" 在 location 指令域添加 Nginx 配置指令

而Ingress只能修改server和location域的指令，具体是通过annotations：

nginx.ingress.kubernetes.io/server-snippet string 在 server 指令域添加 Nginx 配置指令
nginx.ingress.kubernetes.io/configuration-snippet string 在 location 指令域添加 Nginx 配置指令

首选我们搭建openresty环境，通过直接修改nginx.conf的方式来用lua连接mysql

helm install ingress-nginx bitnami/nginx-ingress-controller
NAME: ingress-nginx
LAST DEPLOYED: Fri Jun  3 22:29:03 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1

把镜像更新一下

% kubectl -n nginx-ingress edit deploy nginx-ingress-nginx-ingress-controller
 image: docker.io/bitnami/nginx-ingress-controller:latest
 
 % kubectl -n nginx-ingress edit deployment.apps/nginx-ingress-nginx-ingress-controller-default-backend
 image： docker.io/bitnami/openresty:latest

修改nginx连接方式为nodePort

 % kubectl -n nginx-ingress edit svc nginx-ingress-nginx-ingress-controller-default-backend
service/nginx-ingress-nginx-ingress-controller-default-backend edited
 type: NodePort

测试下

 % curl http://127.0.0.1:31307/

openresty已经起来了，但是由于配置文件在pod里面，因此我们需要挂载一个volume，把配置文件映射到主机上

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-volume
spec:
  storageClassName: hostpath
  capacity:
    storage: 200Mi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/Users/xiazemin/go/src/github.com/xiazemin/k8s_learn/ingress/lua/"

% kubectl apply -f ingress/lua/pv.yaml
persistentvolume/pv-volume created

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-volume
spec:
  storageClassName: hostpath
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Mi

 % kubectl apply -f ingress/lua/pvc.yaml
persistentvolumeclaim/pvc-volume created

在部署中映射主机目录

       volumeMounts:
        - name: pv-volume
          mountPath: /opt/bitnami/openresty/nginx/conf/server_blocks/
      volumes:
      - name: pv-volume
        persistentVolumeClaim:
          claimName: pvc-volume

% kubectl -n nginx-ingress edit deployment.apps/nginx-ingress-nginx-ingress-controller-default-backend

注意容器内的路径使用

/opt/bitnami/openresty/nginx/conf/server_blocks/

这个bitnami的helm chart上抽象好的nginx配置文件加载路径，否则需要更改docker镜像，设置环境变量，巨麻烦。

修改nginx.conf文件

# Based on https://www.nginx.com/resources/wiki/start/topics/examples/full/#nginx-conf

# you do not need the following line if you are using
    # the ngx_openresty bundle:
server {
    # Port to listen on, can also be set in IP:PORT format
    listen  8080;
    location / {
        stub_status on;
        access_log   on;
        content_by_lua_block {
            ngx.say("hello lua");
        }
    }
}

重启pod，测试下

 % curl http://localhost:31307/
hello lua

说明基础环境已经成功了，下面我们连接mysql，首先部署mysql

% helm install mysql bitnami/mysql -n nginx-ingress
NAME: mysql
LAST DEPLOYED: Sun Jun  5 13:31:57 2022
NAMESPACE: nginx-ingress
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
** Please be patient while the chart is being deployed **

Tip:

修改nginx.conf支持加载lua文件

# Based on https://www.nginx.com/resources/wiki/start/topics/examples/full/#nginx-conf

# you do not need the following line if you are using
    # the ngx_openresty bundle:
lua_package_path "/opt/bitnami/openresty/nginx/conf/server_blocks/rocks-5.1/lua-resty-mysql/lib/?.lua;;";

server {
    lua_code_cache off;
    # Port to listen on, can also be set in IP:PORT format
    listen  8080;
    location /lua {
        resolver kube-dns.kube-system.svc.cluster.local;
        access_log on;
        content_by_lua_file /opt/bitnami/openresty/nginx/conf/server_blocks/hello.lua;
        header_filter_by_lua_file /opt/bitnami/openresty/nginx/conf/server_blocks/header.lua;
    }
    location / {
        resolver kube-dns.kube-system.svc.cluster.local;
        stub_status on;
        access_log   on;
        content_by_lua_block {
            ngx.say("hello lua");
        }
    }
}

关闭lua的代码缓冲，可以方便热更新，不用重启nginx

 lua_code_cache off;

指定lua包的路径

lua_package_path "/opt/bitnami/openresty/nginx/conf/server_blocks/rocks-5.1/lua-resty-mysql/lib/?.lua;;";

lua-resty-mysql 是通过包管理工具luarocks下载的

 % luarocks --lua-dir=/usr/local/opt/lua@5.1 install lua-resty-mysql 0.15-0

安装成功后把路径链接到我们映射的目录里面，我们在代码里面就可以直接引用了。

注意nginx需要在location里指定域名解析器，否则没法访问我们内部的service

 resolver kube-dns.kube-system.svc.cluster.local;

下面是我们修改内容和header的两段代码

ngx.say("hello lua file test");
ngx.say(ngx.config.ngx_lua_version);


local mysql = require "resty.mysql"
local db, err = mysql:new()
if not db then
    ngx.say("failed to instantiate mysql: ", err)
    return
end

db:set_timeout(1000) -- 1 sec

-- or connect to a unix domain socket file listened
-- by a mysql server:
--     local ok, err, errcode, sqlstate =
--           db:connect{
--              path = "/path/to/mysql.sock",
--              database = "ngx_test",
--              user = "ngx_test",
--              password = "ngx_test" }

local ok, err, errcode, sqlstate = db:connect{
    -- host = "mysql.nginx-ingress.svc.cluster.local",
    host = "mysql.nginx-ingress.svc.cluster.local",
    port = 3306,
    database = "my_database",
    user = "root",
    password = "2ZjMDPcySI",
    charset = "utf8",
    max_packet_size = 1024 * 1024,
}

if not ok then
    ngx.say("failed to connect: ", err, ": ", errcode, " ", sqlstate)
    return
end

ngx.say("connected to mysql.")

local res, err, errcode, sqlstate =
    db:query("drop table if exists cats")
if not res then
    ngx.say("bad result: ", err, ": ", errcode, ": ", sqlstate, ".")
    return
end

res, err, errcode, sqlstate =
    db:query("create table cats "
                .. "(id serial primary key, "
                .. "name varchar(5))")
if not res then
    ngx.say("bad result: ", err, ": ", errcode, ": ", sqlstate, ".")
    return
end

ngx.say("table cats created.")

res, err, errcode, sqlstate =
    db:query("insert into cats (name) "
                .. "values (\'Bob\'),(\'\'),(null)")
if not res then
    ngx.say("bad result: ", err, ": ", errcode, ": ", sqlstate, ".")
    return
end

ngx.say(res.affected_rows, " rows inserted into table cats ",
        "(last insert id: ", res.insert_id, ")")

-- run a select query, expected about 10 rows in
-- the result set:
res, err, errcode, sqlstate =
    db:query("select * from cats order by id asc", 10)
if not res then
    ngx.say("bad result: ", err, ": ", errcode, ": ", sqlstate, ".")
    return
end

local cjson = require "cjson"
ngx.say("result: ", cjson.encode(res))

-- put it into the connection pool of size 100,
-- with 10 seconds max idle timeout
local ok, err = db:set_keepalive(10000, 100)
if not ok then
    ngx.say("failed to set keepalive: ", err)
    return
end

-- or just close the connection right away:
-- local ok, err = db:close()
-- if not ok then
--     ngx.say("failed to close: ", err)
--     return
-- end

local cjson = require "cjson"

if ngx.var.my_headers ~= nil
then
    headers = cjson.decode(ngx.var.my_headers)
    for k, v in pairs(headers) do
        ngx.header[k] = v
    end
end

ngx.header["Content-Type"]="text/plain"

测试一下：

% curl  http://localhost:31307/lua
hello lua file test
10020
connected to mysql.
table cats created.
3 rows inserted into table cats (last insert id: 1)
result: [{"id":"1","name":"Bob"},{"id":"2","name":""},{"id":"3","name":null}]

可以看到我们已经成功从mysql取到内容返回回来了

本文参与腾讯云自媒体同步曝光计划，分享自微信公众号。

原始发表：2022-06-05，如有侵权请联系 cloudcommunity@tencent.com 删除

lua