lua-resty-waf

基于OpenResty堆栈的高性能WAF。(High-performance WAF built on the OpenResty stack)

Github stars Tracking Chart

lua-resty-waf -- 基于 OpenResty 堆栈的高性能 WAF

状态

lua-resty-waf 目前正在积极开发中。在问题跟踪器中打开的新错误和问题将在一两天内得到回答,影响性能的安全相关问题将以高优先级修补。当开发资源可用时,将添加更大的特性集和增强功能(有关计划功能的概述,请参见“路线图”部分)。

lua-resty-waf 与 lua-resty-core 的主分支兼容。最新发行的 OpenResty(>= 1.9.7.4)中提供的 lua-resty-core 绑定版本与 lua-resty-waf 兼容;与较旧的 OpenResty 捆绑包绑定的版本则没有,因此想要利用 resty.core 的用户要么需要用 GitHub 项目 中可用的版本替换本地版本,或者基于此提交的基础上给模块打补丁。

描述

lua-resty-waf 是使用 OpenResty 堆栈构建的反向代理 WAF。它使用 Nginx Lua API 来分析 HTTP 请求信息,并根据灵活的规则结构进行处理。 lua-resty-waf 附带一个模仿 ModSecurity CRS 的规则集,以及在初始开发和测试过程中构建的一些自定义规则,以及一个针对新兴威胁的小型虚拟补丁集。此外,lua-resty-waf 附带了用于自动转换现有 ModSecurity 规则的工具,从而使用户无需学习新的规则语法即可扩展 lua-resty-waf 的实现。

lua-resty-waf 最初是由罗伯特·帕普罗奇(Robert Paprocki)为他在西总督大学的硕士论文而开发的。

要求

lua-resty-waf 需要多个第三方 resty lua 模块,尽管它们都与 lua-resty-waf 打包在一起,因此不需要单独安装。建议在运行 OpenResty 软件包的系统上安装 lua-resty-waf。lua-resty-waf 还没有在使用单独的 Nginx 源代码和 Nginx Lua 模块软件包构建的平台上进行测试。

为了获得最佳的正则表达式编译性能,建议使用支持JIT编译的PCRE版本构建Nginx / OpenResty。如果您的操作系统不提供此功能,则可以直接在Nginx / OpenResty构建中构建支持JIT的PCRE。为此,请在--with-pcre configure标志中引用PCRE源的路径。例如:

为了获得最佳的正则表达式编译性能,建议使用支持 JIT 编译的 PCRE 版本来构建Nginx/OpenResty。如果你的操作系统没有提供这个功能,你可以直接在你的 Nginx/OpenResty 构建中构建带有 jit 功能的 PCRE。为此,在 --with-pcre configure 标志中引用 PCRE 源文件的路径。例如
# ./configure --with-pcre=/path/to/pcre/source --with-pcre-jit

您可以从 PCRE 网站 下载 PCRE 源代码。另请参阅此博客文章,以获取有关使用支持 JIT 的 PCRE 库构建 OpenResty 的分步指南。

性能

lua-resty-waf 在设计时考虑了效率和可伸缩性。它利用 Nginx 的异步处理模型和高效的设计来尽快处理每个事务。负载测试表明,部署实现所有提供的规则集(旨在模仿 ModSecurity CRS 背后的逻辑)的部署,每个请求大约需要处理300-500微秒的事务。这等于Cloudflare的WAF宣传的性能。测试是在合理的硬件堆栈(E3-1230 CPU,32 GB RAM,RAID 0中的2 x 840 EVO)上运行的,最大速度约为每秒15,000个请求。 lua-rest-waf 在设计时考虑到了效率和可伸缩性。它利用了 Nginx 的异步处理模型和高效的设计来尽可能快地处理每一个事务。负载测试表明,部署实现了所有提供的规则集,这些规则集被设计成模仿 ModSecurity CRS 背后的逻辑,处理每个请求大约需要 300-500 微秒;这相当于 Cloudflare 的 WAF 所宣传的性能。测试运行在一个合理的硬件堆栈上(E3-1230 CPU、32 GB RAM、RAID 0 配置的 2x840 EVO),每秒钟最多可以请求 15,000 个请求。有关更多信息,请参见 此博客文章

lua-resty-waf 工作负载几乎完全受 CPU 限制。Lua VM 中的内存占用量(不包括由 lua-shared-dict 支持的持久性存储)约为 2MB。

安装

提供了一个简单的 Makefile:

# make && sudo make install

或者,通过 Luarocks 安装:

# luarocks install lua-resty-waf

lua-resty-waf 使用了 OPM 软件包管理器,该软件包管理器可在现代的 OpenResty 发行版中使用。 客户端 OPM 工具要求在系统的 PATH 环境变量中可以使用 resty 命令行工具。

请注意,默认情况下,lua-resty-waf 在 SIMULATE 模式下运行,以防止立即影响应用程序。希望启用规则操作的用户必须将操作模式显式设置为 ACTIVE。

概要

http {
    init_by_lua_block {
        -- use resty.core for performance improvement, see the status note above
        require "resty.core"
        -- require the base module
        local lua_resty_waf = require "resty.waf"
        -- perform some preloading and optimization
        lua_resty_waf.init()
    }
    server {
        location / {
            access_by_lua_block {
                local lua_resty_waf = require "resty.waf"
                local waf = lua_resty_waf:new()
                -- define options that will be inherited across all scopes
                waf:set_option("debug", true)
                waf:set_option("mode", "ACTIVE")
                -- this may be desirable for low-traffic or testing sites
                -- by default, event logs are not written until the buffer is full
                -- for testing, flush the log buffer every 5 seconds
                --
                -- this is only necessary when configuring a remote TCP/UDP
                -- socket server for event logs. otherwise, this is ignored
                waf:set_option("event_log_periodic_flush", 5)
                -- run the firewall
                waf:exec()
            }
            header_filter_by_lua_block {
                local lua_resty_waf = require "resty.waf"
                -- note that options set in previous handlers (in the same scope)
                -- do not need to be set again
                local waf = lua_resty_waf:new()
                waf:exec()
            }
            body_filter_by_lua_block {
                local lua_resty_waf = require "resty.waf"
                local waf = lua_resty_waf:new()
                waf:exec()
            }
            log_by_lua_block {
                local lua_resty_waf = require "resty.waf"
                local waf = lua_resty_waf:new()
                waf:exec()
            }
        }
    }
}

公共函数 & 公共方法 & 选项

(恕删略。请参考自述文件)

阶段处理

lua-resty-waf 设计为在请求生命周期的多个阶段中运行。可以在以下阶段中处理规则:

  • access:请求信息,例如 URI,请求标头,URI 参数和请求主体在此阶段可用。
  • header_filter:在此阶段可以使用响应头和 HTTP 状态。
  • body_filter:响应主体在此阶段可用。
  • log:事件日志在此阶段完成后自动写入。

这些阶段对应于其适当的 Nginx lua 处理程序(分别为 access_by_lua,header_filter_by_lua,body_filter_by_lua 和 log_by_lua)。请注意,不在此列表中的 lua 阶段处理程序中运行 lua-resty-waf 将导致行为中断。较早阶段中可用的所有数据都可在较后阶段中使用。也就是说,在访问阶段可用的数据在 header_filte r和 body_filter 阶段也可用,反之亦然。

包含的规则集

lua-resty-waf 分发有许多规则集,这些规则集旨在模仿 ModSecurity CRS 的功能。作为参考,以下列出了这些规则集:

  • 11000_whitelist:本地策略白名单
  • 20000_http_violation:HTTP协议冲突
  • 21000_http_anomaly:HTTP协议异常
  • 35000_user_agent:恶意/可疑用户代理
  • 40000_generic_attack:通用攻击
  • 41000_sqli:SQLi
  • 42000_xss:XSS
  • 90000_custom:自定义规则/虚拟补丁
  • 99000_scoring:异常分数处理

规则定义

lua-resty-waf 从存储在磁盘上的 JSON blob 解析规则定义。规则根据目的和严重性分组,定义为规则集。创建包含的规则集是为了模仿 ModSecurity CRS 的某些功能,特别是 base_rules 定义。此外,随附的 modsec2lua-resty-waf.pl 脚本可用于将其他或自定义规则集转换为与 lua-resty-waf 兼容的 JSON blob。

请注意,相对于不受支持的操作、集合和运算符,转换脚本中存在一些限制。请参阅此 Wiki 页面 以获取已知不兼容的最新列表。

笔记

社区

有一个Freenode IRC 频道 #lua-resty-waf。 Travis CI在此处发送通知;也可以在此频道中提问/发表评论。

此外,可以在 CodeWake 上使用 Q/A:

Codewake

拉请求

请将所有拉请求指向开发分支,如果 PR 是重大更改,则将其定向到 feature 分支。对母版的承诺仅应以文档更新或其他更改的形式出现,这些更新不会对模块本身产生任何影响(并且可以完全合并到开发中)。

路线图

扩展的虚拟补丁程序规则集:增加对新兴威胁的覆盖范围。
扩展的集成/验收测试:扩大常见威胁和使用场景的覆盖范围。
扩展的ModSecurity语法翻译:支持更多的运算符,变量和操作。
通用应用程序配置文件:为通用CMS /应用程序调整的规则集。
支持多个套接字/文件记录器目标:可能需要分叉lua-resty-logger-socket项目。

局限性

lua-resty-waf正在不断发展和完善,因此,其功能和性能可能会受到限制。当前已知的限制可以在此发行版的GitHub问题跟踪器中找到。

许可证

该程序是免费软件:您可以根据自由软件基金会发布的GNU通用公共许可的条款(许可的版本3)或(根据您的选择)任何更高版本来重新分发和/或修改它。
分发该程序是希望它会有用,但是没有任何保证;甚至没有对适销性或特定用途适用性的暗示保证。有关更多详细信息,请参见GNU通用公共许可证。

您应该已经与该程序一起收到了GNU通用公共许可证的副本。如果没有,请参阅http://www.gnu.org/licenses/

Bugs

请用 GitHub 问题跟踪器创建一个工单来报告错误。

也可以看看



(The first version translated by vz on 2020.08.05)

Main metrics

Overview
Name With Ownerp0pr0ck5/lua-resty-waf
Primary LanguagePerl
Program languageLua (Language Count: 6)
PlatformLinux, Mac, Windows
License:GNU General Public License v3.0
所有者活动
Created At2014-11-04 06:49:41
Pushed At2024-01-31 14:54:02
Last Commit At2021-10-05 19:12:41
Release Count22
Last Release Namev0.11.1 (Posted on 2017-05-09 09:48:01)
First Release Namev0.0.5 (Posted on 2014-12-03 17:20:49)
用户参与
Stargazers Count1.3k
Watchers Count84
Fork Count305
Commits Count688
Has Issues Enabled
Issues Count250
Issue Open Count29
Pull Requests Count54
Pull Requests Open Count6
Pull Requests Close Count27
项目设置
Has Wiki Enabled
Is Archived
Is Fork
Is Locked
Is Mirror
Is Private

Name

lua-resty-waf - High-performance WAF built on the OpenResty stack

Table of Contents

Status

Build Status
Codewake
CII Best Practices

lua-resty-waf is currently in active development. New bugs and questions opened in the issue tracker will be answered within a day or two, and performance impacting / security related issues will be patched with high priority. Larger feature sets and enhancements will be added when development resources are available (see the Roadmap section for an outline of planned features).

lua-resty-waf is compatible with the master branch of lua-resty-core. The bundled version of lua-resty-core available in recent releases of OpenResty (>= 1.9.7.4) is compatible with lua-resty-waf; versions bundled with older OpenResty bundles are not, so users wanting to leverage resty.core will either need to replace the local version with the one available from the GitHub project, or patch the module based off this commit.

Description

lua-resty-waf is a reverse proxy WAF built using the OpenResty stack. It uses the Nginx Lua API to analyze HTTP request information and process against a flexible rule structure. lua-resty-waf is distributed with a ruleset that mimics the ModSecurity CRS, as well as a few custom rules built during initial development and testing, and a small virtual patchset for emerging threats. Additionally, lua-resty-waf is distributed with tooling to automatically translate existing ModSecurity rules, allowing users to extend lua-resty-waf implementation without the need to learn a new rule syntax.

lua-resty-waf was initially developed by Robert Paprocki for his Master's thesis at Western Governor's University.

Requirements

lua-resty-waf requires several third-party resty lua modules, though these are all packaged with lua-resty-waf, and thus do not need to be installed separately. It is recommended to install lua-resty-waf on a system running the OpenResty software bundle; lua-resty-waf has not been tested on platforms built using separate Nginx source and Nginx Lua module packages.

For optimal regex compilation performance, it is recommended to build Nginx/OpenResty with a version of PCRE that supports JIT compilation. If your OS does not provide this, you can build JIT-capable PCRE directly into your Nginx/OpenResty build. To do this, reference the path to the PCRE source in the --with-pcre configure flag. For example:

# ./configure --with-pcre=/path/to/pcre/source --with-pcre-jit

You can download the PCRE source from the PCRE website. See also this blog post for a step-by-step walkthrough on building OpenResty with a JIT-enabled PCRE library.

Performance

lua-resty-waf was designed with efficiency and scalability in mind. It leverages Nginx's asynchronous processing model and an efficient design to process each transaction as quickly as possible. Load testing has show that deployments implementing all provided rulesets, which are designed to mimic the logic behind the ModSecurity CRS, process transactions in roughly 300-500 microseconds per request; this equals the performance advertised by Cloudflare's WAF. Tests were run on a reasonable hardware stack (E3-1230 CPU, 32 GB RAM, 2 x 840 EVO in RAID 0), maxing at roughly 15,000 requests per second. See this blog post for more information.

lua-resty-waf workload is almost exclusively CPU bound. Memory footprint in the Lua VM (excluding persistent storage backed by lua-shared-dict) is roughly 2MB.

Installation

A simple Makefile is provided:

# make && sudo make install

Alternatively, install via Luarocks:

# luarocks install lua-resty-waf

lua-resty-waf makes use of the OPM package manager, available in modern OpenResty distributions. The client OPM tools requires that the resty command line tool is available in your system's PATH environmental variable.

Note that by default lua-resty-waf runs in SIMULATE mode, to prevent immediately affecting an application; users who wish to enable rule actions must explicitly set the operational mode to ACTIVE.

Synopsis

http {
    init_by_lua_block {
        -- use resty.core for performance improvement, see the status note above
        require "resty.core"

        -- require the base module
        local lua_resty_waf = require "resty.waf"

        -- perform some preloading and optimization
        lua_resty_waf.init()
    }

    server {
        location / {
            access_by_lua_block {
                local lua_resty_waf = require "resty.waf"

                local waf = lua_resty_waf:new()

                -- define options that will be inherited across all scopes
                waf:set_option("debug", true)
                waf:set_option("mode", "ACTIVE")

                -- this may be desirable for low-traffic or testing sites
                -- by default, event logs are not written until the buffer is full
                -- for testing, flush the log buffer every 5 seconds
                --
                -- this is only necessary when configuring a remote TCP/UDP
                -- socket server for event logs. otherwise, this is ignored
                waf:set_option("event_log_periodic_flush", 5)

                -- run the firewall
                waf:exec()
            }

            header_filter_by_lua_block {
                local lua_resty_waf = require "resty.waf"

                -- note that options set in previous handlers (in the same scope)
                -- do not need to be set again
                local waf = lua_resty_waf:new()

                waf:exec()
            }

            body_filter_by_lua_block {
                local lua_resty_waf = require "resty.waf"

                local waf = lua_resty_waf:new()

                waf:exec()
            }

            log_by_lua_block {
                local lua_resty_waf = require "resty.waf"

                local waf = lua_resty_waf:new()

                waf:exec()
            }
        }
    }
}

Public Functions

lua-resty-waf.load_secrules()

Translate and initialize a ModSecurity SecRules file from disk. Note that this still requires the ruleset to be added via add_ruleset (the basename of the file must be given as the key).

Example:

http {
    init_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        -- this translates and calculates a ruleset called 'ruleset_name'
        local ok, errs = pcall(function()
            lua_resty_waf.load_secrules("/path/to/secrules/ruleset_name")
        end)

        -- errs is an array-like table
        if errs then
            for i = 1, #errs do
                ngx.log(ngx.ERR, errs[i])
            end
        end
    }

    server {
        location / {
            access_by_lua_block {
                local lua_resty_waf = require "resty.waf"

                local waf = lua_resty_waf:new()

                -- in order to use the loaded ruleset, it must be added via
                -- the 'add_ruleset' option
                waf:set_option("add_ruleset", "ruleset_name")
            }
        }
    }
}

Additionally, load_secrules can take an optional second argument as a table of options to pass to various translation functions. The following options are recognized:

  • path: Define a filesystem path to search for data files for operators such as @pmFromFile. If no such key is defined, the current working directory (.) is used
  • force: Do not error and abort when failing to translate a rule variable
  • loose: Do not error and abort when failing to translate a rule action
  • quiet: Do not error or warn when failing to translate a rule action

This function can also take a third option as a table to catch translation errors, for later processing. If this option is not present or a not a table, translation errors will instead be logged to the error log.

lua-resty-waf.init()

Perform some pre-computation of rules and rulesets, based on what's been made available via the default distributed rulesets. It's recommended, but not required, to call this function (not doing so will result in a small performance penalty). This function should never be called outside this scope.

Example:

http {
    init_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        lua_resty_waf.init()
    }
}

Public Methods

lua-resty-waf:new()

Instantiate a new instance of lua-resty-waf. You must call this in every request handler phase you wish to run lua-resty-waf, and use the return result to call further object methods.

Example:

location / {
    access_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        local waf = lua_resty_waf:new()
    }
}

lua-resty-waf:set_option()

Configure an option on a per-scope basis.

Example:

location / {
    access_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        local waf = lua_resty_waf:new()

        -- enable debug logging only for this scope
        waf:set_option("debug", true)
    }
}

lua-resty-waf:set_var()

Define a transaction variable (stored in the TX variable collection) before executing the WAF. This can be used to define variables used by complex rulesets such as the OWASP CRS.

Example:

location / {
    access_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        local waf = lua_resty_waf:new()

        waf:set_var("FOO", "bar")
    }
}

Note that as with any other ModSecurity rule, the existence of a variable bears no functional change to WAF processing; it is the responsibility of the rule author to understand and use TX variables.

lua-resty-waf:sieve_rule()

Define a collection exclusion for a given rule.

Example:

location / {
    access_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        local waf = lua_resty_waf:new()

        local sieves = {
            {
                type   = "ARGS",
                elts   = "foo",
                action = "ignore",
            }
        }

        waf:sieve_rule("12345", sieves)
    }
}

See the rule sieves wiki page for details and advanced usage examples.

lua-resty-waf:exec()

Run the rule engine. By default, the engine is executed according to the currently running phase. An optional table may be passed, allowing users to "mock" execution of a different phase.

Example:

location / {
    access_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        local waf = lua_resty_waf:new()

        -- execute according to access phase collections and rules
        waf:exec()
    }

    content_by_lua_block {
        local lua_resty_waf = require "waf"

        local waf = lua_resty_waf:new()

        -- execute header_filter rules, passing in a table of additional collections
        -- this assumes the 'request_headers' and 'status' Lua variables were
        -- declared and initialized elsewhere
        local opts = {
            phase = 'header_filter',
            collections = {
                REQUEST_HEADERS = request_headers,
                STATUS = status,
            }
        }

        waf:exec(opts)
    }
}

lua-resty-waf:write_log_events()

Write any audit log entries that were generated from the transaction. This is only optional when exec is called in a log_by_lua handler.

Example:

location / {
    log_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        local waf = lua_resty_waf:new()

        -- write out any event log entries to the
        -- configured target, if applicable
        waf:write_log_events()
    }
}

Options

add_ruleset

Default: none

Adds an additional ruleset to be used during processing. This allows users to implement custom rulesets without stomping over the included rules directory. Additional rulesets must reside within a folder called "rules" that lives within the lua_package_path.

Example:

http {
    -- the rule file 50000.json must live at
    -- /path/to/extra/rulesets/rules/50000.json
    lua_package_path '/path/to/extra/rulesets/?.lua;;';

    server {
        location / {
            access_by_lua_block {
                waf:set_option("add_ruleset", "50000_extra_rules")
            }
        }
    }
}

Multiple rulesets may be added by passing a table of values to set_option. Note that ruleset names are sorted before processing. Rulesets are processed in a low-to-high sorted order.

add_ruleset_string

Default: none

Adds an additional ruleset to be used during processing. This allows users to implement custom rulesets without stomping over the included rules directory. Rulesets are defined inline as a Lua string, in the form of a translated ruleset JSON structure.

Example:

location / {
    access_by_lua_block {
        waf:set_option("add_ruleset_string", "70000_extra_rules", [=[{"access":[{"action":"DENY","id":73,"operator":"REGEX","opts":{},"pattern":"foo","vars":[{"parse":{"values":1},"type":"REQUEST_ARGS"}]}],"body_filter":[],"header_filter":[]}]=])
    }
}

Note that ruleset names are sorted before processing, and must be given as strings. Rulesets are processed in a low-to-high sorted order.

allow_unknown_content_types

Default: false

Instructs lua-resty-waf to continue processing the request when a Content-Type header has been sent that is not in the allowed_content_types table. Such requests will not have their request body processed by lua-resty-waf (the REQUEST_BODY collection will be nil). In this manner, users do not need to explicitly whitelist all possible Content-Type headers they may encounter.

Example:

location / {
    access_by_lua_block {
        waf:set_option("allow_unknown_content_types", true)
    }
}

allowed_content_types

Default: none

Defines one or more Content-Type headers that will be allowed, in addition to the default Content-Types application/x-www-form-urlencoded and multipart/form-data. A request whose content type matches one of allowed_content_types will set the REQUEST_BODY collection to a single string containing (rather than a table); a request whose content type does not match one of these values, or application/x-www-form-urlencoded or multipart/form-data, will be rejected.

Example:

location / {
    access_by_lua_block {
        -- define a single allowed Content-Type value
        waf:set_option("allowed_content_types", "text/xml")

        -- defines multiple allowed Content-Type values
        waf:set_option("allowed_content_types", { "text/html", "text/json", "application/json" })
    }
}

Note that mutiple set_option calls with a parameter of allowed_content_types will simply override the existing options table, so if you want to define multiple allowed content types, you must define them as a Lua table as shown above.

debug

Default: false

Disables/enables debug logging. Debug log statements are printed to the error_log. Note that debug logging is very expensive and should not be used in production environments.

Example:

location / {
    access_by_lua_block {
        waf:set_option("debug", true)
    }
}

debug_log_level

Default: ngx.INFO

Sets the nginx log level constant used for debug logging.

Example:

location / {
    access_by_lua_block {
        waf:set_option("debug_log_level", ngx.DEBUG)
    }
}

deny_status

Default: ngx.HTTP_FORBIDDEN

Sets the status to use when denying requests.

Example:

location / {
    access_by_lua_block {
        waf:set_option("deny_status", ngx.HTTP_NOT_FOUND)
    }
}

disable_pcre_optimization

Default: false

Removes the oj flags from all ngx.re.match, ngx.re.find, and ngx.re.sub calls. This may be useful in some cases where older PCRE libraries are used, but will cause severe performance degradation, so its use is strongly discouraged; users are instead encouraged to build OpenResty with a modern, JIT-capable PCRE library.

Example:

location / {
    access_by_lua_block {
        waf:set_option("disable_pcre_optimization", true)
    }
}

Note: This behavior is deprecated and will be removed in future versions.

event_log_altered_only

Default: true

Determines whether to write log entries for rule matches in a transaction that was not altered by lua-resty-waf. "Altered" is defined as lua-resty-waf acting on a rule whose action is ACCEPT or DENY. When this option is unset, lua-resty-waf will log rule matches even if the transaction was not altered. By default, lua-resty-waf will only write log entries for matches if the transaction was altered.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_altered_only", false)
    }
}

Note that mode will not have an effect on determing whether a transaction is considered altered. That is, if a rule with a DENY action is matched, but lua-resty-waf is running in SIMULATE mode, the transaction will still be considered altered, and rule matches will be logged.

event_log_buffer_size

Default: 4096

Defines the threshold size, in bytes, of the buffer to be used to hold event logs. The buffer will be flushed when this threshold is met.

Example:

location / {
    access_by_lua_block {
        -- 8 KB event log message buffer
        waf:set_option("event_log_buffer_size", 8192)
    }
}

event_log_level

Default: ngx.INFO

Sets the nginx log level constant used for event logging.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_level", ngx.WARN)
    }
}

event_log_ngx_vars

Default: empty

Defines what extra variables from ngx.var are put to the log event. This is a generic way to extend the alert with extra context. The variable name will be the key of the entry under an ngx key in the log entry. If the variable is not present as an nginx variable, no item is added to the event.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_ngx_vars", "host")
        waf:set_option("event_log_ngx_vars", "request_id")
    }
}

The resulting event has these extra items:

{
"ngx": {
    "host": "example.com",
    "request_id": "373bcce584e3c18a"
}
}

event_log_periodic_flush

Default: none

Defines an interval, in seconds, at which the event log buffer will periodically flush. If no value is configured, the buffer will not flush periodically, and will only flush when the event_log_buffer_size threshold is reached. Configure this option for very low traffic sites that may not receive any event log data in a long period of time, to prevent stale data from sitting in the buffer.

Example:

location / {
    access_by_lua_block {
        -- flush the event log buffer every 30 seconds
        waf:set_option("event_log_periodic_flush", 30)
    }
}

event_log_request_arguments

Default: false

When set to true, the log entries contain the request arguments under the uri_args key.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_request_arguments", true)
    }
}

event_log_request_body

Default: false

When set to true, the log entries contain the request body under the request_body key.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_request_body", true)
    }
}

event_log_request_headers

Default: false

The headers of the HTTP request is copied to the log event, under the request_headers key.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_request_headers", true)
    }
}

The resulting event has these extra items:

{
"request_headers": {
    "accept": "*/*",
    "user-agent": "curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3"
}
}

event_log_ssl

Default: false

Enable SSL connections when logging via TCP/UDP.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_ssl", true)
    }
}

event_log_ssl_sni_host

Default: none

Set the SNI host for lua-resty-logger-socket connections.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_ssl_sni_host", "loghost.example.com")
    }
}

event_log_ssl_verify

Default: false

Enable certification verification for SSL connections when logging via TCP/UDP.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_ssl_verify", true)
    }
}

event_log_socket_proto

Default: udp

Defines which IP protocol to use (TCP or UDP) when shipping event logs via a remote socket. The same buffering and recurring flush logic will be used regardless of protocol.

Example:

location / {
    access_by_lua_block {
        -- send logs via TCP
        waf:set_option("event_log_socket_proto", "tcp")
    }
}

event_log_target

Default: error

Defines the destination for event logs. lua-resty-waf currently supports logging to the error log, a separate file on the local file system, or a remote TCP or UDP server. In the latter two cases, event logs are buffered and flushed when a defined threshold is reached (see below for further options regarding event logging options).

Example:

location / {
    access_by_lua_block {
        -- send event logs to the server's error_log location (default)
        waf:set_option("event_log_target", "error")

        -- send event logs to a local file on disk
        waf:set_option("event_log_target", "file")

        -- send event logs to a remote server
        waf:set_option("event_log_target", "socket")
    }
}

Note that, due to a limition in the logging library used, only a single target socket can be defined. This is to say, you may only configure one socket target with a specific host/port combination; if you configure a second host/port combination, data will not be properly logged.

event_log_target_host

Default: none

Defines the target server for event logs that target a remote server.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_target_host", "10.10.10.10")
    }
}

event_log_target_path

Default: none

Defines the target path for event logs that target a local file system location.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_target_path", "/var/log/lua-resty-waf/event.log")
    }
}

This path must be in a location writeable by the nginx user. Note that, by nature, on-disk logging can cause significant performance degredation in high-concurrency environments.

event_log_target_port

Default: none

Defines the target port for event logs that target a remote server.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_target_port", 9001)
    }
}

hook_action

Default: none

Override the functionality of actions taken when a rule is matched. See the example for more details

Example:


    location / {
        access_by_lua_block {
            local deny_override = function(waf, ctx)
                ngx.log(ngx.INFO, "Overriding DENY action")
                ngx.status = 404
            end

            -- override the DENY action with the function defined above
            waf:set_option("hook_action", "DENY", deny_override)
        }
    }

ignore_rule

Default: none

Instructs the module to ignore a specified rule ID. Note that ignoring a rule in a chain will result in the entire chain being ignored, and processing will continue to the next rule following the chain.

Example:

location / {
    access_by_lua_block {
        waf:set_option("ignore_rule", 40294)
        waf:set_option("ignore_rule", {40002, 41036})
    }
}

Multiple rules can be ignored by passing a table of rule IDs to set_option.

ignore_ruleset

Default: none

Instructs the module to ignore an entire ruleset. This can be useful when some rulesets (such as the SQLi or XSS CRS rulesets) are too prone to false positives, or aren't applicable to your application.

Example:

location / {
    access_by_lua_block {
        waf:set_option("ignore_ruleset", "41000_sqli")
    }
}

mode

Default: SIMULATE

Sets the operational mode of the module. Options are ACTIVE, INACTIVE, and SIMULATE. In ACTIVE mode, rule matches are logged and actions are run. In SIMULATE mode, lua-resty-waf loops through each enabled rule and logs rule matches, but does not complete the action specified in a given run. INACTIVE mode prevents the module from running.

By default, SIMULATE is selected if a mode is not explicitly set; this requires new users to actively implement blocking by setting the mode to ACTIVE.

Example:

location / {
    access_by_lua_block {
        waf:set_option("mode", "ACTIVE")
    }
}

nameservers

Default: none

Sets the DNS resolver(s) to be used for RBL lookups. Currently only UDP/53 traffic is supported. This option must be defined as a numeric address, not a hostname. If this option is not defined, all RBL lookup rules will return false.

Example:

location / {
    access_by_lua_block {
        waf:set_option("nameservers", "10.10.10.10")
    }
}

process_multipart_body

Default true

Enable processing of multipart/form-data request bodies (when present), using the lua-resty-upload module. In the future, lua-resty-waf may use this processing to perform stricter checking of upload bodies; for now this module performs only minimal sanity checks on the request body, and will not log an event if the request body is invalid. Disable this option if you do not need this checking, or if bugs in the upstream module are causing problems with HTTP uploads.

Example:

location / {
    access_by_lua_block {
        -- disable processing of multipart/form-data requests
        -- note that the request body will still be sent to the upstream
        waf:set_option("process_multipart_body", false)
    }
}

req_tid_header

Default: false

Set an HTTP header X-Lua-Resty-WAF-ID in the upstream request, with the value as the transaction ID. This ID will correlate with the transaction ID present in the debug logs (if set). This can be useful for request tracking or debug purposes.

Example:

location / {
    access_by_lua_block {
        waf:set_option("req_tid_header", true)
    }
}

res_body_max_size

Default: 1048576 (1 MB)

Defines the content length threshold beyond which response bodies will not be processed. This size of the response body is determined by the Content-Length response header. If this header does not exist in the response, the response body will never be processed.

Example:

location / {
    access_by_lua_block {
        -- increase the max response size to 2 MB
        waf:set_option("res_body_max_size", 1024 * 1024 * 2)
    }
}

Note that by nature, it is required to buffer the entire response body in order to properly use the response as a collection, so increasing this number significantly is not recommended without justification (and ample server resources).

res_body_mime_types

Default: "text/plain", "text/html"

Defines the MIME types with which lua-resty-waf will process the response body. This value is determined by the Content-Type header. If this header does not exist, or the response type is not in this list, the response body will not be processed. Setting this option will add the given MIME type to the existing defaults of text/plain and text/html.

Example:

location / {
    access_by_lua_block {
        -- mime types that will be processed are now text/plain, text/html, and text/json
        waf:set_option("res_body_mime_types", "text/json")
    }
}

Multiple MIME types can be added by passing a table of types to set_option.

res_tid_header

Default: false

Set an HTTP header X-Lua-Resty-WAF-ID in the downstream response, with the value as the transaction ID. This ID will correlate with the transaction ID present in the debug logs (if set). This can be useful for request tracking or debug purposes.

Example:

location / {
    access_by_lua_block {
        waf:set_option("res_tid_header", true)
    }
}

score_threshold

Default: 5

Sets the threshold for anomaly scoring. When the threshold is reached, lua-resty-waf will deny the request.

Example:

location / {
    access_by_lua_block {
        waf:set_option("score_threshold", 10)
    }
}

storage_backend

Default: dict

Define an engine to use for persistent variable storage. Current available options are dict (ngx_lua shared memory zone), memcached, amd redis.

Example:

location / {
    acccess_by_lua_block {
        waf:set_option("storage_backend", "memcached")
    }
}

storage_keepalive

Default: true

Enable or disable TCP keepalive for connections to remote persistent storage hosts.

Example:

location / {
    acccess_by_lua_block {
        waf:set_option("storage_keepalive", false)
    }
}

storage_keepalive_timeout

Default: 10000

Configure (in milliseconds) the timeout for the cosocket keepalive pool for remote persistent storage hosts.

Example:

location / {
    acccess_by_lua_block {
        waf:set_option("storage_keepalive_timeout", 30000)
    }
}

storage_keepalive_pool_size

Default: 100

Configure the pool size for the cosocket keepalive pool for remote persistent storage hosts.

Example:

location / {
    acccess_by_lua_block {
        waf:set_option("storage_keepalive_pool_size", 50)
    }
}

storage_memcached_host

Default: 127.0.0.1

Define a host to use when using memcached as a persistent variable storage engine.

Example:

location / {
    acccess_by_lua_block {
        waf:set_option("storage_memcached_host", "10.10.10.10")
    }
}

storage_memcached_port

Default: 11211

Define a port to use when using memcached as a persistent variable storage engine.

Example:

location / {
    acccess_by_lua_block {
        waf:set_option("storage_memcached_port", 11221)
    }
}

storage_redis_host

Default: 127.0.0.1

Define a host to use when using redis as a persistent variable storage engine.

Example:

location / {
    acccess_by_lua_block {
        waf:set_option("storage_redis_host", "10.10.10.10")
    }
}

storage_redis_port

Default: 6379

Define a port to use when using redis as a persistent variable storage engine.

Example:

location / {
    acccess_by_lua_block {
        waf:set_option("storage_redis_port", 6397)
    }
}

storage_zone

Default: none

Defines the lua_shared_dict that will be used to hold persistent storage data. This zone must be defined in the http{} block of the configuration.

Example:_

http {
    -- define a 64M shared memory zone to hold persistent storage data
    lua_shared_dict persistent_storage 64m;
}

location / {
    access_by_lua_block {
        waf:set_option("storage_zone", "persistent_storage")
    }
}

Multiple shared zones can be defined and used, though only one zone can be defined per configuration location. If a zone becomes full and the shared dictionary interface cannot add additional keys, the following will be entered into the error log:

Error adding key to persistent storage, increase the size of the lua_shared_dict

Phase Handling

lua-resty-waf is designed to run in multiple phases of the request lifecycle. Rules can be processed in the following phases:

  • access: Request information, such as URI, request headers, URI args, and request body are available in this phase.
  • header_filter: Response headers and HTTP status are available in this phase.
  • body_filter: Response body is available in this phase.
  • log: Event logs are automatically written at the completion of this phase.

These phases correspond to their appropriate Nginx lua handlers (access_by_lua, header_filter_by_lua, body_filter_by_lua, and log_by_lua, respectively). Note that running lua-resty-waf in a lua phase handler not in this list will lead to broken behavior. All data available in an earlier phase is available in a later phase. That is, data available in the access phase is also available in the header_filter and body_filter phases, but not vice versa.

Included Rulesets

lua-resty-waf is distributed with a number of rulesets that are designed to mimic the functionality of the ModSecurity CRS. For reference, these rulesets are listed here:

  • 11000_whitelist: Local policy whitelisting
  • 20000_http_violation: HTTP protocol violation
  • 21000_http_anomaly: HTTP protocol anomalies
  • 35000_user_agent: Malicious/suspect user agents
  • 40000_generic_attack: Generic attacks
  • 41000_sqli: SQLi
  • 42000_xss: XSS
  • 90000_custom: Custom rules/virtual patching
  • 99000_scoring: Anomaly score handling

Rule Definitions

lua-resty-waf parses rules definitions from JSON blobs stored on-disk. Rules are grouped based on purpose and severity, defined as a ruleset. The included rulesets were created to mimic some functionality of the ModSecurity CRS, particularly the base_rules definitions. Additionally, the included modsec2lua-resty-waf.pl script can be used to translate additional or custom rulesets to a lua-resty-waf-compatible JSON blob.

Note that there are several limitations in the translation script, with respect to unsupported actions, collections, and operators. Please see this wiki page for an up-to-date list of known incompatibilities.

Notes

Community

There is a Freenode IRC channel #lua-resty-waf. Travis CI sends notifications here; feel free to ask questions/leave comments in this channel as well.

Additionally, Q/A is available on CodeWake:

Codewake

Pull Requests

Please target all pull requests towards the development branch, or a feature branch if the PR is a significant change. Commits to master should only come in the form of documentation updates or other changes that have no impact of the module itself (and can be cleanly merged into development).

Roadmap

  • Expanded virtual patch ruleset: Increase coverage of emerging threats.
  • Expanded integration/acceptance testing: Increase coverage of common threats and usage scenarios.
  • Expanded ModSecurity syntax translations: Support more operators, variables, and actions.
  • Common application profiles: Tuned rulesets for common CMS/applications.
  • Support multiple socket/file logger targets: Likely requires forking the lua-resty-logger-socket project.

Limitations

lua-resty-waf is undergoing continual development and improvement, and as such, may be limited in its functionality and performance. Currently known limitations can be found within the GitHub issue tracker for this repo.

License

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see http://www.gnu.org/licenses/

Bugs

Please report bugs by creating a ticket with the GitHub issue tracker.

See Also