1. 分享回放
2. 日志很重要
3. 程序员
4. 运维/运营
5. timber
6. u/log

分享回放

分享的视频连接:

分享的视频连接

下文主要是分享的记录

日志很重要

系统的日志非常有用:

1. 从程序员的角度: 帮助解决问题
2. 从运维的角度: 分析系统行为, 优化配置
3. 从运营的角度: 分析用户系统, 提供决策支持

程序员

首先从程序员的角度, 方便打log, 可以控制打不打log是主要诉求, 主要落实为:

1. log记录代码和业代码结合, 不需要'额外'编码.
2. 配置简单, 可以控制级别, 区分测试/发布环境
3. 要比较方便的从多条日志中还原出一个场景.

timber可以比较好的解决问题1,2. 问题3其实和运维/运营面临的问题是一样的.

运维/运营

着眼的是log中的统计信息, 一般需要从多条log中判断趋势,一般不太会关注单条的log.

Java系统中日志框架据说有1000多种, 各自对应独有的xml/属性文件的配置, 相当繁琐. 载着容器化/微服务体系也加剧了这一复杂性, 需要有一套繁琐的日志处理工具链.

以logstash来说, 每个实例需要安装日志agent, 汇总本实例的log到logstash, logstash集中处理后生成json, 交由elasticsearch做搜索, 最终呈现给数据展现端.

这个处理流程中的前半段因为日志是纯文本的, 且分布在不同的服务器上, 需要一个汇总的环节, 把分散的文本结构处理为json. 而实际上这个环节是可以避免的, 我们可以直接生成json, 发送到分析系统.

接下来介绍的u/log着眼于解决这个问题.

timber

;; 引入timber依赖
(require '[taoensso.timbre    :as timbre :refer (tracef debugf infof warnf errorf
                                                        spy trace  debug info)])
;; 初始化配置(指定文件名)
(defn log-config [log-file]
  (timbre/merge-config!
   {:level :info
    :ns-blacklist   []
    :timestamp-opts {:pattern "yy-MM-dd HH:mm:ss",
                     :locale :jvm-default,
                     :timezone (TimeZone/getDefault)}
    :appenders {:spit (appenders/spit-appender {:fname log-file})}}))
;; 可以在repl中动态调整log级别
(comment
  (timbre/set-level! :trace)
  (timbre/set-level! :debug)
  (timbre/set-level! :info)
  )
(trace "abc")
(debug "abc")
(info  "abc")
;; 善用spy, 可以非入侵的进行需要的debug
(->> 10
     inc
     range
     (mapv (fn [e] (* e e)))
     (filter even?))
;; spy 嵌入任意表达式
(filter even? (spy :info (mapv (fn [e] (* e e)) (range (inc 10)))))

u/log

;; 引入依赖
(require '[com.brunobonacci.mulog :as u])
;; 简单的日志条目
(u/log ::hello :key1 "v1" :more-info {:foo 1 :bar "two"})
;; 更简单的日志条目
(u/log ::hello :to "New World!")
;; 设置context, context会是全局的
(u/set-global-context!
 {:app-name "my-app"
  :version "v1.0.1"
  :env (or (System/getenv "ENV") "local")})

;; 只有启动了publisher之后, 生成的日志才会被发布出去
(comment
  ;; 最简单的控制台发布
  (def pub (u/start-publisher! {:type :console :pretty? true}))
  ;; 启动的publisher可以随时停掉, 通过启动时返回的函数
  (pub)

  (def multi (u/start-publisher!
              {:type :multi
               :publishers
               [{:type :console}
                {:type :simple-file :filename "simple-ulog.log"}
                {:type :zipkin
                 :url  "http://localhost:9411/"


                 ;; the maximum number of events which can be sent in a single
                 ;; batch request to Zipkin
                 :max-items     5000

                 ;; Interval in milliseconds between publish requests.
                 ;; μ/log will try to send the records to Zipkin
                 ;; with the interval specified.
                 :publish-delay 5000

                 ;; a function to apply to the sequence of events before publishing.
                 ;; This transformation function can be used to filter, tranform,
                 ;; anonymise events before they are published to a external system.
                 ;; by defatult there is no transformation.  (since v0.1.8)
                 :transform identity}
                #_{:type :elasticsearch :url "http://els-cluster"}]}))

  (multi)
  )


;; 程序启动的时候简单的日志
(u/log :app-started)
;; 异常处理的场景
(try
  (< 1 nil)
  (u/log ::successful-compare :key1 "v1" :key2 "v2")
  (catch Exception x
    (u/log ::failed-compare :exception x :key1 "v1" :key2 "v2"))
  )
;; 全场景的记录
(defn inventory-check [order]
  (u/trace ::inventory-check
           [:order order]
           (Thread/sleep 1000 )))
(defn billing [order discount]
  (u/trace ::billing
           [:order order
            :discount discount]
           (Thread/sleep 1200)))
(defn shipment [order membership]
  (u/trace ::shipment
           [:order order
            :membership membership]
           (Thread/sleep 1300)))
(defn process-order [order discount membership]
  (u/trace ::order-processed
           []
           (-> order
               inventory-check
               (billing discount)
               (shipment membership))))
(process-order {:order "order1"}
               {:discount 9/10}
               {:membership :gloden})
;; 有层级的场景
(comment
  (u/trace ::order-processed

           (u/trace ::inventory-check)

           (u/trace ::billing)

           (u/trace ::shipment)))


(comment
  (u/start-publisher!
   {:type :xxx
    :transform (fn [event]
                 ....)})

  ;; 可以灵活定制发布的内容, 比如把所有的异常打印到单独的文件.
  (u/start-publisher!
   {:type :multi
    :publishers
    [{:type :simple-file :filename "/var/log/mulog/everything.log"}
     {:type :simple-file :filename "/var/log/mulog/errors.mlog"
      :transformx
      (partial filter :exception)}
     {:type :elasticsearch
      :transform
      #(map (fn [event]
              (dissoc
               event
               :email
               :username))
            %)}]})


  ;; publisher 有很多的种类
  ;; Simple Console Publisher
  ;; Simple File Publisher
  ;; Advanced Console Publisher
  ;; Cloudwatch Logs Publisher
  ;; Elasticsearch Publisher
  ;; Jaeger Publisher
  ;; Kafka Publisher
  ;; Kinesis Publisher
  ;; Prometheus Publisher
  ;; Slack Publisher
  ;; Zipkin Publisher
  )

u/log的时间链的可视化表示, 对应上文的订单处理的场景