Read-Sitadel-Source

  1. Sitadel源码阅读
    1. 0x00 功能介绍
      1. 1. 指纹识别
      2. 2. 攻击
    2. 0x01 整体结构
    3. 0x02 入口文件
    4. 0x03 lib/request 文件夹
      1. User Agent: lib/request/ragent.py
      2. 类:lib/request/requestfactory.py
      3. 请求:lib/requests/request.py
    5. 0x04 lib/utils 文件夹
      1. 工具显示的baner:lib/utils/banner.py
      2. container.py
      3. datastore.py
      4. manage.py
      5. output.py
      6. validator.py
    6. 0x05 lib/config
      1. settings.py
    7. 0x06 lib/modules/
      1. __init__.py
      2. crawler
        1. 爬虫:lib/crawler/crawler.py
    8. 0x07 lib/modules/fingerprints
      1. _init_.py
      2. server
      3. header
        1. cookie.py
        2. headers.py
      4. cdn指纹识别
        1. cdn基础
        2. akamai
        3. azure.py
        4. cloudflare.py
        5. cloudfront.py
        6. fastly.py
      5. cms指纹识别
        1. Drupal
        2. joomla
        3. magento
        4. wordpress
      6. framework指纹识别
        1. cakephp
        2. cherrypy
        3. dancer
        4. django
        5. flask
        6. fuelphp
        7. grails
        8. laravel
        9. mvc
        10. nette
        11. phalcon
        12. rails
        13. symfony
        14. yii
        15. zend
      7. fronted指纹识别
        1. angularjs
        2. emberjs
        3. jquery
        4. knockout
        5. meteorjs
        6. mootools
        7. prototype
        8. reactjs
        9. vuejs
      8. lang指纹识别
        1. asp
        2. java
        3. perl
        4. php
        5. python
        6. ruby
      9. system指纹识别
        1. bsd
        2. linux
        3. mac
        4. solaris
        5. unix
        6. windows
      10. waf指纹识别
        1. airlock
        2. anquanboa
        3. aws
        4. baidu
        5. barracuda
        6. bigip
        7. binarysec
        8. blockdos
        9. chinacache
        10. ciscoacexml
        11. cloudflare
        12. cloudfront
        13. dotdefender
        14. edgecast
        15. fortiweb
        16. hyperguard
        17. incapsula
        18. isaserver
        19. kona
        20. modsecurity
        21. netcontinuum
        22. paloalt
        23. profense
        24. radware
        25. requestvalidationmode
        26. safedog
        27. secureiis
        28. sengnix
        29. sitelock
        30. sonicwall
        31. sucuri
        32. trafficshield
        33. varnish
        34. wallarm
        35. webknight
    9. 0x08 lib/modules/attacks
      1. bruteforce
        1. admin.py
        2. backdoor.py
        3. bdir.py
        4. bfile.py
        5. dir.py
        6. file.py
        7. init.py
        8. log.py
        9. init.py
      2. injection
        1. html.py
        2. init.py
        3. ldap.py
        4. php.py
        5. rfi.py
        6. sql.py
        7. xpath.py
        8. xss.py
      3. other
        1. allow_method.py
        2. dav.py
        3. htmlobject.py
        4. init.py
        5. listing.py
        6. multipleindex.py
        7. phpinfo.py
        8. robots.py
        9. xst.py
      4. vulns
        1. anonymous.py
        2. crime.py
        3. init.py
        4. shellshock.py
        5. strutsshock.py

Sitadel源码阅读

项目地址:https://github.com/shenril/Sitadel

Web应用程序安全扫描程序

[TOC]

0x00 功能介绍

1. 指纹识别

a) 服务器

b) Web框架(CakePHP、CheeryPy……)

c) 前端框架(AngularJS、MeteorJS、VueJS……)

d) Web应用程序防火墙(Waf)

e) 内容管理系统(CMS)

f) 操作系统(Linux、Unix……)

g) 编程语言(PHP、Ruby……)

h) Cookie安全

i) 内容分发网络(CDN)

2. 攻击

(1)暴力破解

  • 管理接口
  • 常用后门
  • 常用备份目录
  • 常用备份文件
  • 常用目录
  • 常用文件
  • 日志文件

(2)注入攻击

  • HTML注入
  • SQL注入
  • LDAP注入
  • XPath注入
  • 跨站脚本(XSS)
  • 远程文件披露(RFI)
  • PHP代码注入

(3)其他攻击

  • HTTPAllow方法
  • HTML对象
  • 多重引用
  • Robots路径
  • WebDav
  • 跨站追踪(XST)
  • PHPINFO
  • Listing

(4)漏洞利用

  • ShellShock
  • 匿名密码(CVE-2007-1858)
  • SPDY(CVE-2012-4929)
  • Struts-Shock

0x01 整体结构

类型 作用
dir config 配置
dir lib 主要攻击文件,扩展,攻击用到的一些字典等等
dir tests 测试文件
file CHANGELOG 更新日志
file Dockerfile docker文件
file LICENSE 许可证
file Makefile 使用tests文件夹做测试
file README.md 介绍
file setup.py 安装文件
file sitadel.py 主入口文件

所有文件

├── CHANGELOG
├── config
│   └── config.yml
├── Dockerfile
├── lib
│   ├── config
│   │   ├── __init__.py
│   │   └── settings.py
│   ├── data
│   │   ├── admin.txt
│   │   ├── allowmethod.txt
│   │   ├── backdoor.txt
│   │   ├── bdir.txt
│   │   ├── bfile.txt
│   │   ├── cdir.txt
│   │   ├── cfile.txt
│   │   ├── index.txt
│   │   ├── ldap.txt
│   │   ├── log.txt
│   │   ├── phpinfo.txt
│   │   ├── rfi.txt
│   │   ├── sql.txt
│   │   ├── xpath.txt
│   │   └── xss.txt
│   ├── __init__.py
│   ├── modules
│   │   ├── attacks
│   │   │   ├── bruteforce
│   │   │   │   ├── admin.py
│   │   │   │   ├── backdoor.py
│   │   │   │   ├── bdir.py
│   │   │   │   ├── bfile.py
│   │   │   │   ├── dir.py
│   │   │   │   ├── file.py
│   │   │   │   ├── __init__.py
│   │   │   │   └── log.py
│   │   │   ├── __init__.py
│   │   │   ├── injection
│   │   │   │   ├── html.py
│   │   │   │   ├── __init__.py
│   │   │   │   ├── ldap.py
│   │   │   │   ├── php.py
│   │   │   │   ├── rfi.py
│   │   │   │   ├── sql.py
│   │   │   │   ├── xpath.py
│   │   │   │   └── xss.py
│   │   │   ├── other
│   │   │   │   ├── allow_method.py
│   │   │   │   ├── dav.py
│   │   │   │   ├── htmlobject.py
│   │   │   │   ├── __init__.py
│   │   │   │   ├── listing.py
│   │   │   │   ├── multipleindex.py
│   │   │   │   ├── phpinfo.py
│   │   │   │   ├── robots.py
│   │   │   │   └── xst.py
│   │   │   └── vulns
│   │   │       ├── anonymous.py
│   │   │       ├── crime.py
│   │   │       ├── __init__.py
│   │   │       ├── shellshock.py
│   │   │       └── strutsshock.py
│   │   ├── crawler
│   │   │   ├── crawler.py
│   │   │   └── __init__.py
│   │   ├── fingerprints
│   │   │   ├── cdn
│   │   │   │   ├── akamai.py
│   │   │   │   ├── azure.py
│   │   │   │   ├── cloudflare.py
│   │   │   │   ├── cloudfront.py
│   │   │   │   ├── fastly.py
│   │   │   │   └── __init__.py
│   │   │   ├── cms
│   │   │   │   ├── drupal.py
│   │   │   │   ├── __init__.py
│   │   │   │   ├── joomla.py
│   │   │   │   ├── magento.py
│   │   │   │   └── wordpress.py
│   │   │   ├── framework
│   │   │   │   ├── cakephp.py
│   │   │   │   ├── cherrypy.py
│   │   │   │   ├── dancer.py
│   │   │   │   ├── django.py
│   │   │   │   ├── flask.py
│   │   │   │   ├── fuelphp.py
│   │   │   │   ├── grails.py
│   │   │   │   ├── __init__.py
│   │   │   │   ├── laravel.py
│   │   │   │   ├── mvc.py
│   │   │   │   ├── nette.py
│   │   │   │   ├── phalcon.py
│   │   │   │   ├── rails.py
│   │   │   │   ├── symfony.py
│   │   │   │   ├── yii.py
│   │   │   │   └── zend.py
│   │   │   ├── frontend
│   │   │   │   ├── angularjs.py
│   │   │   │   ├── emberjs.py
│   │   │   │   ├── __init__.py
│   │   │   │   ├── jquery.py
│   │   │   │   ├── knockout.py
│   │   │   │   ├── meteorjs.py
│   │   │   │   ├── mootools.py
│   │   │   │   ├── prototype.py
│   │   │   │   ├── reactjs.py
│   │   │   │   └── vuejs.py
│   │   │   ├── header
│   │   │   │   ├── cookie.py
│   │   │   │   ├── headers.py
│   │   │   │   └── __init__.py
│   │   │   ├── __init__.py
│   │   │   ├── lang
│   │   │   │   ├── asp.py
│   │   │   │   ├── __init__.py
│   │   │   │   ├── java.py
│   │   │   │   ├── perl.py
│   │   │   │   ├── php.py
│   │   │   │   ├── python.py
│   │   │   │   └── ruby.py
│   │   │   ├── server
│   │   │   │   ├── __init__.py
│   │   │   │   └── server.py
│   │   │   ├── system
│   │   │   │   ├── bsd.py
│   │   │   │   ├── __init__.py
│   │   │   │   ├── linux.py
│   │   │   │   ├── mac.py
│   │   │   │   ├── solaris.py
│   │   │   │   ├── unix.py
│   │   │   │   └── windows.py
│   │   │   └── waf
│   │   │       ├── airlock.py
│   │   │       ├── anquanboa.py
│   │   │       ├── aws.py
│   │   │       ├── baidu.py
│   │   │       ├── barracuda.py
│   │   │       ├── bigip.py
│   │   │       ├── binarysec.py
│   │   │       ├── blockdos.py
│   │   │       ├── chinacache.py
│   │   │       ├── ciscoacexml.py
│   │   │       ├── cloudflare.py
│   │   │       ├── cloudfront.py
│   │   │       ├── dotdefender.py
│   │   │       ├── edgecast.py
│   │   │       ├── fortiweb.py
│   │   │       ├── hyperguard.py
│   │   │       ├── incapsula.py
│   │   │       ├── __init__.py
│   │   │       ├── isaserver.py
│   │   │       ├── kona.py
│   │   │       ├── modsecurity.py
│   │   │       ├── netcontinuum.py
│   │   │       ├── paloalto.py
│   │   │       ├── profense.py
│   │   │       ├── radware.py
│   │   │       ├── requestvalidationmode.py
│   │   │       ├── safedog.py
│   │   │       ├── secureiis.py
│   │   │       ├── sengnix.py
│   │   │       ├── sitelock.py
│   │   │       ├── sonicwall.py
│   │   │       ├── sucuri.py
│   │   │       ├── trafficshield.py
│   │   │       ├── varnish.py
│   │   │       ├── wallarm.py
│   │   │       └── webknight.py
│   │   └── __init__.py
│   ├── request
│   │   ├── __init__.py
│   │   ├── ragent.py
│   │   ├── requestfactory.py
│   │   └── request.py
│   └── utils
│       ├── banner.py
│       ├── container.py
│       ├── datastore.py
│       ├── __init__.py
│       ├── manager.py
│       ├── output.py
│       └── validator.py
├── LICENSE
├── Makefile
├── README.md
├── setup.py
├── sitadel.py
└── tests
    └── lib
        ├── config
        │   ├── good-config.yml
        │   ├── __init__.py
        │   ├── test_attack_config.yml
        │   ├── test_config.py
        │   └── test_fingerprint_config.yml
        ├── modules
        │   ├── attacks
        │   │   ├── __init__.py
        │   │   └── test_attack.py
        │   ├── fingerprints
        │   │   ├── __init__.py
        │   │   └── test_fingerprint.py
        │   └── __init__.py
        ├── request
        │   ├── __init__.py
        │   ├── test_ragent.py
        │   └── test_request.py
        └── utils
            ├── __init__.py
            ├── test_container.py
            └── test_manager.py

31 directories, 180 files

0x02 入口文件

sitadel.py:

主入口文件。会先初始化一些Usage,接受命令行参数并进行相关的前期处理。然后根据参数开始进行扫描。

if __name__ == "__main__":
    try:
        Sitadel().main()
    except KeyboardInterrupt:
        sys.exit(output.Output().error('Interruption by the user, Quitting...'))

定义一个Sitadel类,通过parser.add_argument接受命令行参数

class Sitadel(object):
    bn = banner.Banner()
    ma = manager
    url = None

    def main(self):
        parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter,
                                         usage=self.bn.banner())
        # 准备风险等级的可能值
        risk_values = [r.value for r in Risk]
        # 添加命令行参数:
            #设定url选项字符串的名字url, help:参数的帮助信息
        parser.add_argument("url", help="URL of the website to scan")
            #设定危险等级选项字符串的名字-r或者-risk,参数可允许的值为risk_values
        parser.add_argument("-r", "--risk", type=int, help="Level of risk allowed for the scan",
                            choices=risk_values)
            #设定user-agent选项字符串的名字-ua,默认为"Sitadel " + 版本
        parser.add_argument("-ua", "--user-agent", default="Sitadel " + __version__,
                            help="User-agent to set for the scan requests")
            #设定遵循重定向选项字符串的名字redirect  解析后的参数名称为redirect,命令行遇到参数时的动作为store_true
        parser.add_argument("--redirect", dest='redirect',
                            help="Whether or not the scan should follow redirection",
                            action="store_true")
            #设定不遵循重定向选项字符串的名字--no-redirect  解析后的参数名称为redirect
        parser.add_argument("--no-redirect", dest='redirect',
                            help="Whether or not the scan should follow redirection",
                            action="store_false")
            #设定默认遵循重定向
        parser.set_defaults(redirect=True)
            #设定timeout选项字符串的名字-t,类型为整形
        parser.add_argument("-t", "--timeout", type=int, help="Timeout to set for the scan HTTP requests")
            #设定cookie选项字符串的名字-c
        parser.add_argument("-c", "--cookie", help="Cookie to set for the scan HTTP requests")
            #设定代理选项字符串的名字-p
        parser.add_argument("-p", "--proxy", help="Proxy to set for the scan HTTP requests")
            #设定指纹识别选项字符串的名字-f,nargs读取的指纹识别参数个数为1或多个
        parser.add_argument("-f", "--fingerprint", nargs='+', help="Fingerprint modules to activate")
            #设定攻击选项字符串的名字-a,nargs读取的攻击方式参数个数为1或多个
        parser.add_argument("-a", "--attack", nargs='+', help="Attack modules to activate")
            #设定配置选项字符串的名字-config,默认为config/config.yml
        parser.add_argument("--config", help="Path to the config file", default="config/config.yml")
             #设定verbosity选项字符串的名字--v,命令行遇到参数时的动作为存储遇到的次数,默认为0
        parser.add_argument("-v", "--verbosity", action="count", default=0, help="Increase output verbosity")
            # 输出版本信息
        parser.add_argument('--version', action='version', version=self.bn.version())
        args = parser.parse_args()

运行扫描必要配置

class Sitadel(object):
    bn = banner.Banner()
    ma = manager
    url = None

    def main(self):
       ...省略...
        # 验证目标URL
        self.url = validator.validate_target(args.url)

        # 阅读配置
        settings.from_yaml(args.config)
        if args.risk is not None:
            settings.risk = Risk(args.risk)

        # 注册服务数据、日志、输出、请求
        Services.register("datastore", Datastore(settings.datastore))
        Services.register("logger", logging.getLogger("sitadelLog"))
        Services.register("output", Output())
        Services.register("request_factory",
                          SingleRequest(url=self.url, agent=args.user_agent, proxy=args.proxy, redirect=args.redirect,
                                  timeout=args.timeout))

        # 显示目标和扫描开始时间
        self.bn.preamble(self.url)

        # 运行指纹识别模块
        self.ma.fingerprints(args.fingerprint,
                             args.user_agent,
                             args.proxy,
                             args.redirect,
                             args.timeout,
                             self.url,
                             args.cookie)

        # 运行爬虫程序发现URL
        discovered_urls = self.ma.crawler(self.url, args.user_agent)

        # 在发现的URL上运行攻击模块
        self.ma.attacks(args.attack, self.url, discovered_urls)

0x03 lib/request 文件夹

│   ├── request
│   │   ├── __init__.py
│   │   ├── ragent.py
│   │   ├── requestfactory.py
│   │   └── request.py

主要是定义一些跟请求相关的方法/类/功能

User Agent: lib/request/ragent.py

生成随机的 User-Agent。命令行选项wascan.py --ragent开启。

import random

#定义随机User-Agent
def RandomUserAgent():
    agents = (
        'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1',
        'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)',
        'Opera/9.80 (X11; Linux i686; Ubuntu/14.10) Presto/2.12.388 Version/12.16',
        'Mozilla/5.0 (Windows; U; MSIE 9.0; Windows NT 9.0; en-US',
        'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FSL 7.0.6.01001)',
        'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0',
        'Mozilla/5.0 (Windows NT 5.1; rv:13.0) Gecko/20100101 Firefox/13.0.1',
        'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0) Gecko/20100101 Firefox/11.0',
        'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705)',
        'Mozilla/5.0 (Windows NT 5.1; rv:13.0) Gecko/20100101 Firefox/13.0.1',
        'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)',
        'Opera/9.80 (Windows NT 5.1; U; en) Presto/2.10.289 Version/12.01',
        'Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1',
        'Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko',
        'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0',
        'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0)'
    )
    #返回agents中的随机一个
    return str(agents[random.randint(0, len(agents) - 1)])

类:lib/request/requestfactory.py

定义了几种可能出现的不同协议的request:

class MultipleHTTPRequests:

    def __init__(self, url):
        self.url = url


class SingleHTTPRequest:
    pass


class DNSRequest:
    pass


class RequestFactory:
    def __init__(self):
        pass
    #**kwargs表示关键字参数,它是一个 dict
    def make_multiple_requests(self, **kwargs):
        return MultipleHTTPRequests(**kwargs)

    def make_single_request(self, **kwargs):
        return SingleHTTPRequest(**kwargs)

    def make_dns_request(self, **kwargs):
        return DNSRequest(**kwargs)

请求:lib/requests/request.py

基本请求。包括请求/代理认证,请求,重定向,响应的处理。

class SingleRequest:
    # 接受参数
    def __init__(self, **kwargs):
        # 获取各项值,保存到kwargs的dict中,后期进一步处理
        self.url = None if "url" not in kwargs else kwargs["url"]
        self.agent = "Sitadel" if "agent" not in kwargs else kwargs["agent"]
        self.proxy = None if "proxy" not in kwargs else kwargs["proxy"]
        self.redirect = True if "redirect" not in kwargs else kwargs["redirect"]
        self.timeout = None if "timeout" not in kwargs else kwargs["timeout"]
        self.ruagent = ragent.RandomUserAgent()
    # 发送请求
    def send(self, url, method="GET", payload=None, headers=None, cookies=None):
        #请求会话
        output = Services.get('output')
        request = Session()
        prepped=self.prepare_request(url,method,payload,headers,cookies)
        urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
        try:
            resp=request.send(
                prepped,
                timeout=self.timeout,
                proxies={
                    'http': self.proxy,
                    'https': self.proxy,
                    'ftp': self.proxy,
                },
                allow_redirects=self.redirect,
                verify=False)
            return resp
        except RequestException as err:
            output.error("Error while trying to contact the website: \n {0}\n".format(err))
            raise(err)
    # 准备的请求 获取各项参数
    def prepare_request(self, url, method, payload, headers, cookies):
        if payload is None:
            payload = {}
        if headers is None:
            headers = {}
        if cookies is not None:
            cookies = {cookies: ''}
        if "--random-agent" in sys.argv:
            headers['User-Agent'] = self.ruagent
        else:
            headers['User-Agent'] = self.agent
        # get请求方式处理
        if method.upper() == "GET": # upper() 小写字母转为大写字母
            req = Request(
                method=method.upper(),
                url=url,
                headers=headers,
                cookies=cookies,
            ).prepare() #发送请求之前的额外处理
        # post请求方式处理
        elif method.upper() == "POST":
            req = Request(
                method=method.upper(),
                url=url,
                data=payload,
                headers=headers,
                cookies=cookies,
            ).prepare()
        # 其他请求方式
        else:
            req = Request(
                method=method.upper(),
                url=url,
                data=payload,
                headers=headers,
                cookies=cookies,
            ).prepare()
        #返回所有的req属性字典
        return req

0x04 lib/utils 文件夹

主要是定义一些小功能、小工具

│   └── utils
│       ├── banner.py
│       ├── container.py
│       ├── datastore.py
│       ├── __init__.py
│       ├── manager.py
│       ├── output.py
│       └── validator.py

工具显示的baner:lib/utils/banner.py

#定义Banner类
class Banner:
    r = Fore.RED
    y = Fore.YELLOW
    ny = Fore.YELLOW
    nw = Fore.WHITE
    g = Fore.GREEN
    e = Style.RESET_ALL
    #定义banner形状和字体颜色
    def banner(self):
        print(self.ny + "   _   _   _        ______ _                 _       _  " + self.e)
        print(self.ny + "  | |_| |_| |      / _____|_)  _            | |     | | " + self.e)
        print(self.ny + "  |         |     ( (____  _ _| |_ _____  __| |_____| | " + self.e)
        print(self.ny + "  |    _    |      \____ \| (_   _|____ |/ _  | ___ | | " + self.e)
        print(self.ny + "  |   |_|   |      _____) ) | | |_/ ___ ( (_| | ____| | " + self.e)
        print(self.ny + "  |         |     (______/|_|  \__)_____|\____|_____)\_)" + self.r + " " + version + "\n" + self.e)
        print(self.g + "~/#" + self.e + " Sitadel - Web Application Security Scanner" + self.g + " #\\~" + self.e)
        print(self.g + "~/#" + self.e + " Shenril (@shenril)" + self.g + " #\\~" + self.e)
        print(self.g + "~/#" + self.e + " https://github.com/shenril/Sitadel" + self.g + " #\\~" + self.e)
        print("\n\n")
    #前言
    def preamble(self, url):
        print('URL: %s' % url)
        print('Started: %s' % (time.strftime('%d/%m/%Y %H:%M:%S')))#返回以可读字符串表示的当地时间

    def version(self):
        return self.g + "~/#" + self.e + " Sitadel (" + version + ")\n"

container.py

定义Services类,获取cls、key参数

class Services(object):
    services = {}

    @classmethod

    def get(cls, key):
        #异常处理,如果服务是空报错
        try:
            if cls.services[key] is None:
                raise NameError("No service registered with this name")
            else:
                return cls.services[key]
        except KeyError:
            raise NameError("No service registered with this name")

    @classmethod
    def register(cls, name, instance) -> None:
        cls.services[name] = instance

datastore.py

用于访问插件数据的公共文件夹的实用程序

import os

class Datastore:

    def __init__(self, rootpath):
        self.rootpath = rootpath
    #python跨路径调用/lib/data/字典文件
    def open(self, filename, mode):
        return open(os.path.join(self.rootpath, filename), mode,encoding="utf-8")

manage.py

#指纹识别传递参数
def fingerprints(modules, agent, proxy, redirect, timeout, url, cookie):
    plugins = settings.fingerprint_plugins
    if modules is not None:
        plugins = modules
    Fingerprints(
        agent=agent,
        proxy=proxy,
        redirect=redirect,
        timeout=timeout,
        url=url,
        cookie=cookie
    ).run(plugins)

#获取爬虫url
def crawler(start_url, agent):
    return crawl(start_url, agent)

#调用plugins攻击执行
def attacks(modules, url, crawled_urls):
    plugins = settings.attack_plugins
    if modules is not None:
        plugins = modules
    Attacks(url, crawled_urls).run(plugins)

output.py

定义了各种打印输出方法,基本的格式化字符串、颜色。略过

validator.py

import sys
from urllib.parse import urlparse

#验证url是否有效
def validate_target(url):
    try:
        u = urlparse(url)
        #判断协议与域名
        if u.scheme and u.netloc:
            return u.geturl()#geturl方法获取真实的url
        else:
            raise ValueError('Url not valid, please try with a valid target url!')
    except ValueError as e:
        print(e)
        sys.exit(2)

0x05 lib/config

settings.py

基本设置

import os.path
from enum import IntEnum

import yaml


class Risk(IntEnum):
    """
    枚举插件的风险
    0 NO_DANGER几乎没有检测到的风险
    1 NOISY生成可能检测到的大量请求和模式
    2危险进行开发阶段,可能对目标有潜在危害

    """
    NO_DANGER = 0
    NOISY = 1
    DANGEROUS = 2

#定义settings类
class Settings(object):
    cfg = {}

    _setters = ['risk', 'dns_resolver', 'datastore']

    def __getattr__(self, item):
        return Settings.cfg[item]

    def __setattr__(self, key, value):
        if key in Settings._setters:
            Settings.cfg[key] = value
        else:
            raise NameError("You cannot redefine the value of %s dynamically\nPlease use the config file" % key)

    @classmethod
    def from_yaml(cls, filepath):
        """
        从yaml文件生成配置字典
        :param filepath:配置文件路径
        :return:无
        """
        # 检查提供的文件路径是否存在
        if not os.path.isfile(filepath):
            raise FileNotFoundError("Invalid path for the configuration file")

        # 解析配置并将其合并到dict中
        with open(filepath, 'r') as yamlfile:
            try:
                # 从文件中获取配置
                config = yaml.load(yamlfile)
                # 合并字典并获得结果
                cls.cfg = {**cls.cfg, **config}
            except yaml.YAMLError as e:
                print(e)

0x06 lib/modules/

__init__.py

class IPlugin(type):
    def __init__(cls, name, bases, dct):
        #判断cls是否不包含plugins属性
        if not hasattr(cls, 'plugins'):
            # 一个基类,创建一个空注册表
            cls.plugins = []
        else:
            # 这是派生类,将cls添加到注册表
            if hasattr(cls, 'level') and getattr(cls, 'level') <= settings.risk:
                cls.plugins.append(cls)
        #调用父类方法
        super(IPlugin, cls).__init__(name, bases, dct)

crawler

爬虫,爬取页面上的所有连接。

│   │   ├── crawler
│   │   │   ├── crawler.py
│   │   │   └── __init__.py

爬虫:lib/crawler/crawler.py

urls = []

#爬虫类SitadelSpider,它继承自CrawlSpider类
class SitadelSpider(CrawlSpider):
    name = "sitadel"
    #定义规则
    rules = [
        Rule(
            LinkExtractor(
                canonicalize=True,
                unique=True
            ),
            follow=True,
            callback="parse_items"
        )
    ]

    # 解析项目的方法
    def parse_items(self, response):
        links = LinkExtractor(canonicalize=True, unique=True).extract_links(response)
        # 对于links中的所有链接
        for link in links:
            #对于允许的域名allowed_domains中所有的域名
            for allowed_domain in self.allowed_domains:
                #判断拆分url的netloc是否等于所允许的域名
                if urlparse(link.url).netloc == allowed_domain:
                    urls.append(link.url)
                    #yield 是一个类似 return的关键字,迭代一次遇到yield时就返回yield后面爬虫下来的url
                    yield scrapy.Request(link.url, callback=self.parse)

#定义crawl函数,传入url和user-agent
def crawl(url, user_agent):
    output = Services.get('output')

    # 爬虫工具的设置
    settings = get_project_settings()
    settings.set("USER_AGENT", user_agent)
    settings.set("LOG_LEVEL", "CRITICAL")

    # 创建将执行爬虫的进程
    output.info('Start crawling the target website')
    process = CrawlerProcess(settings)
    domain = urlparse(url).hostname
    process.crawl(SitadelSpider, start_urls=[str(url)], allowed_domains=[str(domain)])
    process.start()

    # 清理结果
    clean_urls = []
    for u in urls:
        try:
            new_url = urlparse(u).geturl()
            if new_url not in clean_urls:
                clean_urls.append(new_url)
        except ValueError:
            continue

    return clean_urls

0x07 lib/modules/fingerprints

指纹识别模式

目前针对Web应用指纹识别的研究工作主要通过对大量HTML数据的分析来识别Web应用, 包括HTML源码关键字和特殊文件及路径, 由于需要处理较多HTML数据和文件, 降低了其执行速度; 而主流工具主要利用HTML源码关键字实现Web应用的快速识别, 但是由于关键字易被删除而导致识别率低.

为解决Web应用指纹识别的研究工作与实际应用中存在的上述问题, 并提高Web应用的识别率, 本文以主流的开源Web应用[15]为研究对象, 以源码审计为研究方法, 通过对Web应用的静态文件、源码以及结构设计的详细分析, 并从以下4个方面选取Web应用指纹.

1)结构特征. Web应用的类型可由其特殊的结构设计准确识别, 该指纹只需从HTML数据的头部获取. 例如, 如果Web应用的文件路径包含/wp-includes/, 则该Web应用是Wordpress.

2)静态文件. Web应用的类型及版本可由其未被修改而被直接使用的静态文件准确识别. 例如, 用于Web应用前端布局的文件style.css, 通过与目标style.css比较可准确识别Web应用的类型及版本.

3)Cookie设计. Web应用的类型可由开发者为其设计特殊的Cookie名准确识别. 例如, 如果Cookie名为django, 则该Web应用是Django-CMS.

4)关键字. HTML源码关键字是主流工具选取的Web应用指纹, 存在与功能无关且易被删除的缺点, 但可作为Web应用的补充指纹. 通过对关键字在HTML源码中的分析, 选取以下4个关键字指纹:

·<meta name="generator" content="*application*" />

·Powered by *application*

·<meta name="author" content="*author*" />

·<meta name="copyright" content="*copyright*" />

其中, 结构特征、静态文件和Cookie设计这3类指纹都与Web应用功能相关, 有不易被修改或删除的特点, 我们设计并构建了包含这4类Web应用指纹的Web指纹库, 只需按照Web指纹库定义的格式向其中增加新类型Web指纹, 即可实现对新增Web应用的识别, 对Web应用的识别具有良好的可扩展性.

_init_.py

_init_.py代码中Fingerprint类如下:

class FingerprintPlugin(metaclass=IPlugin):
    #指纹模块的默认风险等级为NO DANGER,因为它仅分析一个请求响应
    level = Risk.NO_DANGER

    def process(self, headers, content):
        raise NotImplementedError(str(self) + ": Process method not found")

    def __repr__(self):
        parent_module = self.__class__.__module__.split('.')[-2]
        return parent_module.title()


class Fingerprints:
    def __init__(self, agent, proxy, redirect, timeout, url, cookie):
        # 相关参数 初始化
        self.url = url
        self.cookie = cookie
        self.output = Services.get('output')
        self.request = Services.get('request_factory')

    def run(self, plugins_activated):
        self.output.info('Launching fingerprints modules...')
        # 从配置中注册插件
        for p in plugins_activated:
            #
            currentdir = os.path.dirname(os.path.realpath(__file__))
            pkgpath = os.path.dirname(currentdir + "/%s/" % p)
            modules = [name for _, name, _ in pkgutil.iter_modules([pkgpath])]
            for module in modules:
                importlib.import_module(".{pkg}.{mod}".format(pkg=p, mod=module), __package__)
        try:
            # 发送侦听请求
            # 首先发送HTTP GET请求
            resp = self.request.send(
                url=self.url,
                method="GET",
                payload=None,
                headers=None,
                cookies=self.cookie
            )

            # 循环遍历将结果传递到指纹模块上进行处理
            fingerprints = (
                [(p(), p().process(resp.headers, resp.text)) for p in FingerprintPlugin.plugins])

            # 显示每个模块类别的结果
            for category, result in fingerprints:
                if result is not None:
                    self.output.finding('{category} detected: {result}'.format(category=category, result=result))

        except Exception as e:
            self.output.error("Error occured\nAborting fingerprint...\n")
            return

server

指纹识别模式服务开启 ,代码重点server类如下

#定义Server类 传入指纹识别插件
class Server(FingerprintPlugin):
    def process(self, headers, content):
        server = None
        try:
            for item in headers.items():
                if re.search(r'server', item[0], re.I):
                    server = item[1]
            # FIXME修复对请求URL的访问
            # if server is None:
            #     resp = Request().send(Request().url, headers={'Expect': 'Linguini'})
            #     for item in resp.headers.items():
            #         if re.search(r'server', item[0], re.I): server = item[1]
            return server
        except Exception as e:
            print(e)
class Cookie(FingerprintPlugin):
    def process(self, headers, content):
        #如果被测url的header存在set-cookie,那么取过来放在cookie变量中
        if 'set-cookie' in headers:
            cookie = headers['set-cookie']
        else:
            cookie = None
        #如果cookie不为空
        if cookie is not None:
            #如果cookie中存在domain=*,输出cookie中匹配到的domain=(.+?)[\;]
            if re.search(r'domain=\S*', cookie, re.I):
                Output().finding(
                    'Cookies are only accessible to this domain: %s' % re.findall(r'domain=(.+?)[\;]', cookie, re.I)[0])
            #如果在cookie中未找到httponly则提示cookie创建时没有使用hhtponly技术
            if not re.search('httponly', cookie, re.I):
                Output().finding('Cookies created without HTTPOnly Flag.')
            if not re.search('secure', cookie, re.I):
                Output().finding('Cookies created without Secure Flag.')

headers.py

#定义header类,传入指纹识别插件的参数
class Headers(FingerprintPlugin):
      #定义process函数
    def process(self, headers, content):
      #将各个请求header头字典作为list存入fields参数
        fields = ('Accept',
                  'Accept-Charset',
                  'Accept-Encoding',
                  略。。。。。。
                  'X-UA-Compatible'
                  )
        #如何在headers的key中找到X-Frame-Options则输出提示
        if not re.search(r'X-Frame-Options', str(headers.keys()), re.I):
            Output().finding('X-Frame-Options header is not present.')

        if not re.search(r'Strict-Transport-Security', str(headers.keys()), re.I):
            Output().finding('Strict-Transport-Security header is not present.')

        if not re.search(r'x-xss-protection', str(headers.keys()), re.I):
            Output().finding('X-XSS-Protection header is not present.')
        try:
            #循环遍历key 如果实际情况key都不在扫描器存储的字典key中,则输出提示不在文件中
            for key in headers.keys():
                if key not in fields:
                    Output().finding('Uncommon header "%s" found, with contents: %s' % (key, headers[key]))
        except Exception as e:
            print(e)

cdn指纹识别

cdn基础

接下来我们通过图解来看一下当访问http://www.apple.com 时,apple主页通过CDN获取html页面的全过程

img

(1) 用户向电信运营商的本地DNS递归查询服务器(以下简称local telcom DNS)询问 www.apple.com 的ip地址。

(2,3) local telcom DNS向根域名服务器和.com顶级域名服务器进行递归查询apple.com的权威域名服务器地址。

(4,5) local telcom DNS向apple.com权威域名服务器查询www.apple.com 的ip,得到指向Akamai域名的CNAME记录:www.isg-apple.com.akadns.net,该域名由Akamai拥有。接着local telcom DNS向www.isg-apple.com.akadns.net发起DNS查询请求,得到另一个CNAME记录 www.apple.com.edgekey.net ,该域名同样由Akamai拥有。和上一步一样,接着local telcom DNS向www.apple.com.edgekey.net 发起域名查询请求,得到另一个CNAME记录 e3191.dscc.akamaiedge.net ,该域名还是由Akamai拥有(akadns.net, edgekey, akamaiedge.net 都是Akamai控制的域名)。(由上面的dig截图我们知道,e3191.dscc.akamaiedge.net 是最后一次CNAME记录的域名,也是akamai的智能DNS服务器,它能根据用户所处的地理位置和当前网络负载情况给出最合适的服务器ip地址。)

(6,7) local telcom DNS向根域名服务器和.net顶级域名服务器进行递归查询akamaiedge.net的权威域名服务器地址。

(8,9) lcoal telcom DNS向akamaiedge.net的权威域名服务器查询e3191.dscc.akamaiedge.net 的ip, 得到的返回结果为Akamai CDN edge server的ip,一般是离用户地理位置最近的那个edge server服务器。

(10) local telcom DNS将Akamai CDN edge server的ip返回给用户的计算机。

(11) 用户向Akamai CDN edge server发送请求。

(12) Akamai CDN edge server会进行判断:如果是静态资源请求且缓存已经过期,或者是动态资源请求,会向apple源服务器发送请求,否则直接转到(13)。

(13) Akamai CDN edge server将缓存的静态资源或者作为代理转发的动态资源返回给用户。

我们平时看到的对CDN的使用基本就是按照上图的流程来进行的。

akamai

域名拼接上edgekey.net(akamai独有),如果A记录存在,A记录解析成功,说明这个域名存在有CDN,若不存在则无cdn。

from urllib.parse import urlparse

from dns.resolver import NXDOMAIN, NoAnswer, Resolver, Timeout

from lib.config import settings
from lib.config.settings import Risk
from lib.modules.fingerprints import FingerprintPlugin
from lib.utils.container import Services

#定义Akamai类
class Akamai(FingerprintPlugin):
    level = Risk.NO_DANGER


    def process(self, headers, content):
        request = Services.get('request_factory')
        hostname = urlparse(request.url).hostname
        try:
            resolver = Resolver(configure=False)
            resolver.nameservers = [settings.dns_resolver]
            resolver.timeout = 2
            resolver.lifetime = 2

            #dns询问指定查询类型为A记录 
            dns_query = resolver.query(hostname + ".edgekey.net", 'A')

            #域名解析正确且至少要返回一个IP,则判断是Akamai CDN
            if len(dns_query) > 0:
                return "Akamai CDN"

        except NXDOMAIN:
            pass
        except NoAnswer:
            pass
        except Timeout:
            pass

azure.py

DNS解析CNAME 记录的查询是否存在azureedge.net

class Azure(FingerprintPlugin):
    level = Risk.NO_DANGER

    def process(self, headers, content):
        request = Services.get('request_factory')
        hostname = urlparse(request.url).hostname
        _ = False

        try:
            resolver = Resolver(configure=False)
            resolver.nameservers = [settings.dns_resolver]
            resolver.timeout = 2
            resolver.lifetime = 2

            dns_query = resolver.query(hostname, 'CNAME')
            if len(dns_query) > 0:
                for answer in dns_query:
                    #扫描answer字符串中azureedge.net,r代表了原字符串不用加转义字符,re.I 忽略大小写 
                    #search是在整体搜索,而match是从开始搜索
                    _ |= re.search(r'azureedge\.net', str(answer), re.I) is not None
            #如果成功判断是Azure  cdn
            if _:
                return "Azure CDN"
        except NoAnswer:
            pass
        except NXDOMAIN:
            pass
        except Timeout:
            pass

cloudflare.py

域名拼接上.cdn.cloudflare.net(cloudflare独有),如果A记录存在,A记录解析成功,说明这个域名存在有cloudflare CDN

class Cloudflare(FingerprintPlugin):
    level = Risk.NO_DANGER

    def process(self, headers, content):
        request = Services.get('request_factory')
        hostname = urlparse(request.url).hostname
        try:
            resolver = Resolver(configure=False)
            resolver.nameservers = [settings.dns_resolver]
            resolver.timeout = 2
            resolver.lifetime = 2
            # 域名拼接上.cdn.cloudflare.net,查询A记录
            dns_query = resolver.query(hostname + ".cdn.cloudflare.net", 'A')
            #域名解析正确且至少要返回一个IP,则判断是Akamai CDN
            if len(dns_query) > 0:
                return "Cloudflare CDN"

cloudfront.py

请求hostname查看cname返回包中师傅存在cloudfront.net

class CloudFront(FingerprintPlugin):
    level = Risk.NO_DANGER

    def process(self, headers, content):
        request = Services.get('request_factory')
        hostname = urlparse(request.url).hostname
        _ = False
        try:
            resolver = Resolver(configure=False)
            resolver.nameservers = [settings.dns_resolver]
            resolver.timeout = 2
            resolver.lifetime = 2
            #
            dns_query = resolver.query(hostname, 'CNAME')

            if len(dns_query) > 0:
                #扫描answer字符串中cloudfront.net
                for answer in dns_query:
                    _ |= re.search(r'cloudfront\.net', str(answer), re.I) is not None
            if _:
                return "CloudFront CDN (Amazon)"

fastly.py

请求hostname查看cname返回包中师傅存在fastly.net

def process(self, headers, content):
        request = Services.get('request_factory')
        hostname = urlparse(request.url).hostname
        _ = False
        try:
            resolver = Resolver(configure=False)
            resolver.nameservers = [settings.dns_resolver]
            resolver.timeout = 2
            resolver.lifetime = 2

            dns_query = resolver.query(hostname, 'CNAME')
            if len(dns_query) > 0:
                for answer in dns_query:
                    _ |= re.search(r'fastly\.net', str(answer), re.I) is not None
            if _:
                return "Fastly CDN"

cms指纹识别

CMS识别原理就是得到一些CMS的一些固有特征,通过得到这个特征来判断CMS的类别。

Drupal

三个search任意一个结果为1 _就是true,然后返回时drupal cms

但是这里问题很大:有部分开发者会除自带的Drupal.settings

drupal 7 移除自带的 css 和 javascript 及jQuery.extend(Drupal.settings,

drupal7后第二条匹配规则就算是失效了。

三条匹配规则还需要添加

class Drupal(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        try:
            #使用 Python 的原始字符串,只需加一个 r 前缀
            # | 指明两项之间的一个选择
            #在url中寻找/misc/drupal.js或者content中是否有power by Drupal这样去检测drupal
            _ = re.search(r'src="\S*/misc/drupal.js*|Powered by Drupal, an open source content management system',
                          content) is not None
            #\S匹配任何非空白字符 jQuery中Drupal.extend
            #扫描content中匹配存在/misc/drupal.css或jQuery中的Drupal.settings或Drupal.extend
            _ |= re.search(r'\S*/misc/drupal.css"|jQuery.extend\WDrupal.settings|Drupal.extend\W', content) is not None
            #匹配meta标签存在name="Generator" content="Drupal'
            _ |= re.search(r'<meta name="Generator" content="Drupal', content) is not None
            if _:
                return "Drupal"
        except Exception as e:
            print(e)

joomla

两重判断

class Joomla(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        try:
            #匹配url中的/index.php?option= 或 meta标签name="generator" content="Joomla 或匹配 源码存在Powered by <a href="http://www.joomla.org">Joomla!</a>
            _ = re.search(
                r'/index.php?option=(\S*)|<meta name="generator" content="Joomla*|Powered by <a href="http://www.joomla.org">Joomla!</a>*',
                content) is not None
            if _:
                #二次判断是否存在/templates/目录,re.I忽略大小写
                if re.search('/templates/*', content, re.I):
                    return "Joomla"
        except Exception as e:
            print(e)

magento

class Magento(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        try:
          #页面特征较多,做一个循环判断,主要原理是网页中发现关键字
            for x in ('x-magento-init', 'Magento_*', 'images/logo.gif" alt="Magento Commerce" /></a></h1>',
                      '<a href="http://www.magentocommerce.com/bug-tracking" id="bug_tracking_link"><strong>Report All Bugs</strong></a>',
                      '<meta name="keywords" content="Magento, Varien, E-commerce" />', 'mage/cookies.js" ></script>',
                      '<div id="noscript-notice" class="magento-notice">'):
                #判断_是true
                _ = re.search(x, content) is not None
                if _:
                    return "Magento"
        except Exception as e:
            print(e)

wordpress

结构特征. Web应用的类型可由其特殊的结构设计准确识别, 该指纹只需从HTML数据的头部获取. 例如, 如果Web应用的文件路径包含/wp-includes/, 则该Web应用是Wordpress.

class Wordpress(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        try:
            #循环寻找指定URL的关键字来判断是否存在各个Wordpress的url文件夹
            for x in ('/wp-admin/', '/wp-content/', '/wp-includes/', '<meta name="generator" content="WordPress'):
                _ = re.search(x, content) is not None
            if _:
                return "Wordpress"
        except Exception as e:
            print(e)

framework指纹识别

在framework下面对更多种类的框架进行了识别。这里倒是可以自己增加一些额外的框架识别代码,进行扩充。

│   │   │   ├── framework
│   │   │   │   ├── cakephp.py
│   │   │   │   ├── cherrypy.py
│   │   │   │   ├── dancer.py
│   │   │   │   ├── django.py
│   │   │   │   ├── flask.py
│   │   │   │   ├── fuelphp.py
│   │   │   │   ├── grails.py
│   │   │   │   ├──` __init__.py`
│   │   │   │   ├── laravel.py
│   │   │   │   ├── mvc.py
│   │   │   │   ├── nette.py
│   │   │   │   ├── phalcon.py
│   │   │   │   ├── rails.py
│   │   │   │   ├── symfony.py
│   │   │   │   ├── yii.py
│   │   │   │   └── zend.py

cakephp

原理: header特别字段cakephp=的正则匹配

class Cakephp(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中的value值是否有cakephp=
            _ = re.search(r'cakephp=', item[1], re.I) is not None
            if _:
                return "CakePHP (PHP)"

cherrypy

原理: header特别字段CherryPy的正则匹配

class Cherrypy(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配headers中value值是否有CherryPy
        for item in headers.items():
            _ = re.search(r'CherryPy', item[1], re.I) is not None
            if _:
                return "CherryPy (Python)"

dancer

原理:header特别字段Dancer和dancer.session的正则匹配

class Dancer(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有Dancer或存在dancer.session
            _ |= re.search(r'Dancer|dancer\.session=.*', item[1], re.I) is not None
        if _:
            return "Dancer (Perl)"

django

原理:header特别字段csrftoken的正则匹配

class Django(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #csrftoken是django中防止csrf的机制
        for item in headers.items():
            _ = re.search(r'csrftoken=', item[1], re.I) is not None
            if _:
                return "Django (Python)"

flask

原理: header特别字段flask的正则匹配

class Flask(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配header头value值是否有flask
            _ = re.search(r'flask', item[1], re.I) is not None
            if _:
                return "Flask (Python)"

fuelphp

原理: header特别字段fuelcid= 或者 content特别字段Powered by <a href="http://fuelphp.com">FuelPHP</a>的正则匹配

class Fuelphp(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            ##header头value值匹配fuelcid=
            _ = re.search(r'fuelcid=', item[1], re.I) is not None
            #两个匹配规则 按位或运算符,如果content中匹配存在Powered by <a href="http://fuelphp.com">FuelPHP</a> 那么_判定为true
            _ |= re.search(r'Powered by <a href="http://fuelphp.com">FuelPHP</a>', content) is not None
            if _:
                return "FuelPHP (PHP)"

grails

原理:header key值和value值的正则匹配

headers的代码形式{‘k1’:’v1’, ‘k2’:’v2’}

item是个list,遍历出来的item是个二元数组 (‘k1’, ‘v1’)

item[0]是key值,item[1]是value值

class Grails(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #遍历header头匹配value值是Grails
            _ = re.search(r'Grails', item[1], re.I) is not None
            #遍历header头匹配key值是X-Grails或X-Grails-Cached
            _ |= re.search(r'X-Grails|X-Grails-Cached', item[0], re.I) is not None
            if _:
                return "Grails (Java)"

laravel

原理: header字段的value值特别为laravel_session=的正则匹配

class Laravel(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #header头value值匹配laravel_session=
            _ = re.search(r'laravel_session=', item[1], re.I) is not None
            if _:
                return "Laravel (PHP)"

mvc

原理:header特别字段的正则匹配

class Mvc(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配response headers中是否有x-aspnetmvc-version或__requestverificationtoken
            _ = re.search(r'x-aspnetmvc-version|__requestverificationtoken', str(item), re.I) is not None
            if _:
                return "ASP.NET MVC"

nette

原理: header特别字段nette*或nette-browser=*的正则匹配

class Nette(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers头中value值是否有nette*或nette-browser=*
            _ = re.search(r'nette*|nette-browser=*', item[1], re.I) is not None
            if _:
                return "Nette (PHP)"

phalcon

原理:header特别字段phalcon-auth-*或Phalcon [(https://phalconphp.com/)]*的正则匹配

class Phalcon(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers头中value值是否有phalcon-auth-*或Phalcon [(https://phalconphp.com/)]*
            _ = re.search(r'phalcon-auth-*', item[1], re.I) is not None
            _ |= re.search(r'Phalcon [(https://phalconphp.com/)]*', item[1]) is not None
            if _:
                return "Phalcon (C and PHP)"

rails

原理:header key值和value值的正则匹配

class Rails(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #遍历header头匹配value值是Grails
            _ = re.search(r'rails*|_rails_admin_session=*|x-rails', item[1], re.I) is not None
            #遍历header头匹配key值是rails*或x-rails
            _ |= re.search(r'rails*|x-rails', item[0], re.I) is not None
            if _:
                return "Rails (Ruby)"

symfony

原理: header特别字段symfony=*的正则匹配

class Symfony(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中的value值是否有symfony=*
            _ = re.search(r'symfony=*', item[1], re.I) is not None
            if _:
                return "Symfony PHP"

yii

原理: header特别字段_csrf=* 或者 匹配content特别字段Powered by <a href="http://fuelphp.com">FuelPHP</a>的正则匹配

class Yii(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中的value值是否有_csrf=*
            _ = re.search(r'_csrf=*', item[1], re.I) is not None
            #或者匹配content中存在Powered by <a href="http://www.yiiframework.com/" rel="external">Yii Framework</a>
            _ |= re.search(r'Powered by <a href="http://www.yiiframework.com/" rel="external">Yii Framework</a>',
                           content) is not None
            if _:
                return "Yiiframework (PHP)"

zend

原理: header特别字段Zend的正则匹配

class Zend(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中的value值是否有Zend
            _ = re.search(r'Zend', item[1], re.I) is not None
            if _:
                return "Zend (PHP)"

fronted指纹识别

前端框架指纹识别

│   │   │   ├── frontend
│   │   │   │   ├── angularjs.py
│   │   │   │   ├── emberjs.py
│   │   │   │   ├── __init__.py
│   │   │   │   ├── jquery.py
│   │   │   │   ├── knockout.py
│   │   │   │   ├── meteorjs.py
│   │   │   │   ├── mootools.py
│   │   │   │   ├── prototype.py
│   │   │   │   ├── reactjs.py
│   │   │   │   └── vuejs.py

angularjs

原理:三条匹配规则 有一则规则触发就是AngularJS,主要依靠content特别字段的正则匹配

class Angularjs(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #re.I:忽略大小写 匹配content中是否有ng-app
        _ = re.search(r'ng-app', content, re.I) is not None
        #或者匹配content中是否有ng-version
        _ |= re.search(r'ng-version', content, re.I) is not None
        #或者匹配content中是否有app-root
        _ |= re.search(r'app-root', content, re.I) is not None
        if _:
            return "AngularJS (Google)"

emberjs

原理:主要依靠原理: header特别字段的正则匹配

class Emberjs(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中的value值是否有ember-view
            _ = re.search(r'ember-view', item[1], re.I) is not None
            if _:
                return "EmberJs (Javascript)"

jquery

原理:主要依靠content特别字段的正则匹配

class Jquery(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配content中是否有jquery-[0-9\-\.]+.js或者存在jquery
        #举例子:jquery-5.js
        _ = re.search(r'jquery-[0-9\-\.]+.js|jquery', content, re.I) is not None
        if _:
            return "JQuery (Javascript)"

knockout

原理:主要依靠content特别字段的正则匹配

class Knockout(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #举例子knockout-5-..js
        #匹配content中是否有knockout-[0-9\-\.]+.js
        _ = re.search(r'knockout-[0-9\-\.]+.js', content, re.I) is not None
        if _:
            return "Knockout (Javascript)"

meteorjs

原理:主要依靠content特别字段的正则匹配

class Meteorjs(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配content中是否有__meteor_runtime_config__
        _ = re.search(r'__meteor_runtime_config__', content, re.I) is not None
        if _:
            return "MeteorJS (Javascript)"

mootools

原理:主要依靠content特别字段的正则匹配

class Mootools(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配content中是否有MooTools-[0-9\-\.]+.js
        _ = re.search(r'MooTools-[0-9\-\.]+.js', content, re.I) is not None
        if _:
            return "MooTools (Javascript)"

prototype

原理:主要依靠content特别字段的正则匹配

class Prototype(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配content中是否有prototype-[0-9\-\.]+.js或者prototype.js
        _ = re.search(r'prototype.js', content, re.I) is not None
        _ |= re.search(r'prototype-[0-9\-\.]+.js', content, re.I) is not None
        if _:
            return "Prototype (Javascript)"

reactjs

原理:主要依靠content特别字段的正则匹配

class Reactjs(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配content中是否有react-[0-9\-\.]+.js或者reactroot或者reactid
        _ = re.search(r'react-[0-9\-\.]+.js', content, re.I) is not None
        _ = re.search(r'reactroot|reactid', content, re.I) is not None
        if _:
            return "ReactJS (Javascript)"

vuejs

原理:主要依靠content特别字段的正则匹配

class Vuejs(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配content中是否有v-bind或者v-for或者v-if
        _ |= re.search(r'v-bind|v-for|v-if', content, re.I) is not None
        if _:
            return "VueJS (Javascript)"

lang指纹识别

对采用何种编程语言的识别

│   │   │   ├── lang
│   │   │   │   ├── asp.py
│   │   │   │   ├── __init__.py
│   │   │   │   ├── java.py
│   │   │   │   ├── perl.py
│   │   │   │   ├── php.py
│   │   │   │   ├── python.py
│   │   │   │   └── ruby.py

对采用何种编程语言的识别有点意思,也是response header和content部分特别字段的正则匹配。

看下python语言的识别。lang/python.py:

asp

原理:主要依靠header和content特别字段的正则匹配

class Asp(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            ##匹配header中是否有ASP.NET或X-AspNet-Version或x-aspnetmvc-version
            _ = re.search(r'ASP.NET|X-AspNet-Version|x-aspnetmvc-version', str(item), re.I) is not None
            #\W匹配非字母字符,即匹配特殊字符。匹配content是否有__VIEWSTATE, \W匹配非字母字符,即匹配特殊字符
            if not _:
                _ |= re.search(r'(__VIEWSTATE\W*)', content) is not None
            #$:匹配字符串末尾 匹配content是否有.asp或者aspx
            if not _:
                _ |= re.search(r'\.asp$|\.aspx$', content) is not None
            if _:
                return "ASP.NET"

java

原理:主要依靠header和content特别字段的正则匹配

class Java(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            ##匹配headers中的是否有Java或Servlet或JSP或JBoss或Glassfish或Oracle或JRE或JDK|或SESSIONID
            _ = re.search(r'Java|Servlet|JSP|JBoss|Glassfish|Oracle|JRE|JDK|JSESSIONID', str(item)) is not None
            if not _:
                #匹配content中字符串的末尾存在.jsp或.jspx或.do或.wss或.action
                _ |= re.search(r'\.jsp$|\.jspx$|.do$|\.wss$|\.action$', content) is not None
            if _:
                return "Java"

perl

原理:主要依靠content特别字段的正则匹配

class Perl(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配content中字符串的末尾存在.pl或.cgi
        _ = re.search(r'\.pl$|\.cgi$', content) is not None
        if _:
            return "Perl"

php

原理:主要依靠header和content特别字段的正则匹配

class Php(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中的是否有X-PHP-PID或PHP2、php3、php4等 或 PHPSESSID
            _ = re.search(r'X-PHP-PID|PHP\S*|PHPSESSID', str(item)) is not None
            #匹配content中字符串的末尾存在.php或.phtml
        _ |= re.search(r'\.php$|\.phtml$', content) is not None
        if _:
            return "PHP"

python

原理:主要依靠 header和content特别字段的正则匹配

class Python(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中的是否有python或zope或zserver或wsgi或plone或_ZopeId
            _ = re.search(r'python|zope|zserver|wsgi|plone|_ZopeId', item[1], re.I) is not None
        #匹配content中字符串的末尾存在.py
        _ |= re.search(r'\.py$', content) is not None
        if _:
            return "Python"

ruby

原理:主要依靠response header和content特别字段的正则匹配

class Ruby(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中的是否有mod_rack或phusion或passenger
            _ = re.search(r"mod_rack|phusion|passenger", item[1], re.I) is not None
        #匹配content中字符串的末尾存在.rb或.rhtml
        _ |= re.search(r'\.rb$|\.rhtml$', content) is not None
        if _:
            return "Ruby"

system指纹识别

这个识别还是很粗糙的,因为很多时候http response里面server部分有时候没内容,然后os部分有时候也没有内容。

bsd

原理:主要依靠header特别字段的正则匹配

class Bsd(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配headers中的是否有任何非空白字符+BSD
        for item in headers.items():
            _ = re.search(r'\S*BSD', str(item), re.I) is not None
            if _:
                return "BSD"

linux

原理:主要依靠header特别字段的正则匹配

class Linux(FingerprintPlugin):
    def process(self, headers, content):
        for item in headers.items():
            #re.findall匹配的全部字串,返回形式为数组
            _ = re.findall(          r'linux|ubuntu|gentoo|debian|dotdeb|centos|redhat|sarge|etch|lenny|squeeze|wheezy|jessie|red hat|scientific linux',
                str(item), re.I)
            #如果匹配到两个则返回第一个匹配到的,如果匹配一个或者两个以上也返回第一个匹配到的
            if _:
                if len(_) == 2:
                    return _[0]
                else:
                    return _[0]

mac

原理:主要依靠header特别字段的正则匹配

class Mac(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中的是否有Mac或MacOS或非空白字符MacOS*
            _ = re.search(r'Mac|MacOS|MacOS\S*', str(item)) is not None
            if _:
                return "MacOS"

solaris

原理:主要依靠header特别字段的正则匹配

class Solaris(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中的是否有solaris或sunos或opensolaris或sparc64或sparc
            _ = re.search(r'solaris|sunos|opensolaris|sparc64|sparc', str(item), re.I) is not None
            if _:
                return "Solaris"

unix

原理:主要依靠header特别字段的正则匹配

class Unix(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中的是否有unix
            _ = re.search(r'unix', str(item), re.I) is not None
            if _:
                return "Unix"

windows

原理:主要依靠header特别字段的正则匹配

class Windows(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中的是否有windows或win32
            _ = re.search(r'windows|win32', str(item), re.I) is not None
            if _:
                return "Windows"

waf指纹识别

│   │   │   └── waf
│   │   │       ├── airlock.py
│   │   │       ├── anquanboa.py
│   │   │       ├── aws.py
│   │   │       ├── baidu.py
│   │   │       ├── barracuda.py
│   │   │       ├── bigip.py
│   │   │       ├── binarysec.py
│   │   │       ├── blockdos.py
│   │   │       ├── chinacache.py
│   │   │       ├── ciscoacexml.py
│   │   │       ├── cloudflare.py
│   │   │       ├── cloudfront.py
│   │   │       ├── dotdefender.py
│   │   │       ├── edgecast.py
│   │   │       ├── fortiweb.py
│   │   │       ├── hyperguard.py
│   │   │       ├── incapsula.py
│   │   │       ├── __init__.py
│   │   │       ├── isaserver.py
│   │   │       ├── kona.py
│   │   │       ├── modsecurity.py
│   │   │       ├── netcontinuum.py
│   │   │       ├── paloalto.py
│   │   │       ├── profense.py
│   │   │       ├── radware.py
│   │   │       ├── requestvalidationmode.py
│   │   │       ├── safedog.py
│   │   │       ├── secureiis.py
│   │   │       ├── sengnix.py
│   │   │       ├── sitelock.py
│   │   │       ├── sonicwall.py
│   │   │       ├── sucuri.py
│   │   │       ├── trafficshield.py
│   │   │       ├── varnish.py
│   │   │       ├── wallarm.py
│   │   │       └── webknight.py

airlock

原理:主要依靠header特别字段的正则匹配

class Airlock(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有^AL[_-]SESS[_-]S=\S*
            _ = re.search(r'^AL[_-]SESS[_-]S=\S*', item[1], re.I) is not None
            #匹配headers中key值的是否有X-Airlock-Test
            _ |= re.search(r'X-Airlock-Test', item[0], re.I) is not None
            if _:
                return "InfoGuard Airlock (Phion/Ergon)"

anquanboa

原理:主要依靠header特别字段的正则匹配

class Anquanboa(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中key值是否有X-Powered-By-Anquanbao
            _ = re.search(r'X-Powered-By-Anquanbao', item[0], re.I) is not None
            if _:
                return "Anquanbao Firewall"

aws

原理:三条匹配规则,主要依靠header特别字段的正则匹配

class Aws(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有aws*
            _ = re.search(r'aws*', item[1], re.I) is not None
            #匹配headers中key值是否有x-amz-id-0、x-amz-id-1、x-amz-id-2
            _ |= re.search(r'x-amz-id-[0-2]', item[0], re.I) is not None
            #匹配headers中key值是否有x-amz-request-id
            _ |= re.search(r'x-amz-request-id', item[0], re.I) is not None
            if _:
                return 'Amazon Web Services Web Application Firewall (Amazon)'

baidu

原理:主要依靠header特别字段的正则匹配

class Baidu(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有yunjiasu-nginx
            _ = re.search(r'yunjiasu-nginx', item[1], re.I) is not None
            #匹配headers中value值是否有YJS-ID
            _ |= re.search(r'YJS-ID', item[1], re.I) is not None
            #匹配headers中value值是否有fhl
            _ |= re.search(r'fhl', item[1], re.I) is not None
            if _:
                return "Yunjiasu Web Application Firewall (Baidu)"

barracuda

原理:主要依靠header特别字段的正则匹配

class Barracuda(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有barracuda*
            _ = re.search(r'barracuda*', item[1], re.I) is not None
            #匹配headers中value值是否有barra_counter_session=
            _ |= re.search(r'barra_counter_session=', item[1], re.I) is not None
            #匹配headers中value值是否有barracuda_
            _ |= re.search(r'barracuda_', item[1], re.I) is not None
            if _:
                return "Barracuda Web Application Firewall (Barracuda Networks)"

bigip

原理:主要依靠header特别字段的正则匹配

class Bigip(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有BigIP或BIGipServer
            _ = re.search(r'BigIP|BIGipServer', item[1], re.I) is not None
            #匹配headers中value值是否有TS+至少匹配4次+=
            _ |= re.search(r'TS\w{4,}=', item[1], re.I) is not None
            #匹配headers中value值是否有F5
            _ |= re.search(r'F5', item[1], re.I) is not None
            if _:
                return "BIG-IP Application Security Manager (F5 Networks)"

binarysec

原理:主要依靠header特别字段的正则匹配

class Binarysec(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有BinarySec
            _ = re.search(r'BinarySec', item[1], re.I) is not None
            #匹配headers中key值是否有x-binarysec-via或x-binarysec-nocahe
            _ |= re.search(r'x-binarysec-[via|nocahe]', item[0], re.I) is not None
            if _:
                return "BinarySEC Web Application Firewall (BinarySEC)"

blockdos

原理:主要依靠header特别字段的正则匹配

class Blockdos(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有BlockDos[\.net]*
            _ = re.search(r'BlockDos[\.net]*', item[1], re.I) is not None
            if _:
                return "BlockDoS"

chinacache

原理:主要依靠header特别字段的正则匹配

class Chinacache(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中key值是否有Powered-By-ChinaCache
            _ = re.search(r'Powered-By-ChinaCache', item[0], re.I) is not None
            if _:
                return "ChinaCache-CDN"

ciscoacexml

原理:主要依靠header特别字段的正则匹配

class Ciscoacexml(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有ACE XML Gateway
            _ = re.search(r'ACE XML Gateway', item[1], re.I) is not None
            if _:
                return "Cisco ACE XML Gateway (Cisco Systems)"

cloudflare

原理:主要依靠header特别字段的正则匹配

class Cloudflare(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有cloudflare[-nginx]
            _ = re.search(r'cloudflare[-nginx]', item[1], re.I) is not None
            #匹配headers中value值是否有__cfduid=
            _ |= re.search(r'__cfduid=', item[1], re.I) is not None
            #匹配headers中key值是否有cf-ray
            _ |= re.search(r'cf-ray', item[0], re.I) is not None
            if _:
                return "CloudFlare Web Application Firewall (CloudFlare)"

cloudfront

原理:主要依靠header特别字段的正则匹配

class Cloudfront(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有(cloudfront)
            _ = re.search(r'(cloudfront)', item[1], re.I) is not None
            #匹配headers中key值是否有x-amz-cf-id
            _ |= re.search('x-amz-cf-id', item[0], re.I) is not None
            if _:
                return "CloudFront Web Application Firewall (Amazon)"

dotdefender

原理:主要依靠header特别字段的正则匹配

class Dotdefender(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中key值是否有X-dotDefender-denied
            _ = re.search(r'X-dotDefender-denied', item[0], re.I) is not None
            if _:
                return "dotDefender Web Application Firewall (Applicure Technologies)"

edgecast

原理:主要依靠header特别字段的正则匹配

class Edgecast(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有ECDF
            _ = re.search(r'ECDF', item[1], re.I) is not None
            if _:
                return "EdgeCast Web Application Firewall (Verizon)"

fortiweb

原理:主要依靠header特别字段的正则匹配

class Fortiweb(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有FORTIWAFSID=
            _ = re.search(r'FORTIWAFSID=', item[1], re.I) is not None
            if _:
                return "FortiWeb Web Application Firewall (Fortinet)"

hyperguard

原理:主要依靠header特别字段的正则匹配

class Hyperguard(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有 开始位置+WODSESSION=
            _ = re.search(r'^WODSESSION=', item[1], re.I) is not None
            if _:
                return "Hyperguard Web Application Firewall (art of defence)"

incapsula

原理:主要依靠header特别字段的正则匹配

class Incapsula(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有 incap_ses或visid_incap或Incapsula
            _ = re.search(r'incap_ses|visid_incap|Incapsula', item[1], re.I) is not None
            if _:
                return "Incapsula Web Application Firewall (Incapsula/Imperva)"

isaserver

原理:主要依靠content特别字段的正则匹配

class Isaserver(FingerprintPlugin):
    def process(self, headers, content):
        try:
            _ = False
            #匹配content中是否有The server denied the specified Uniform Resource Locator (URL). Contact the server administrator.
            _ = re.search(
                r'The server denied the specified Uniform Resource Locator (URL). Contact the server administrator.',
                content, re.I) is not None
            #匹配content中是否有The ISA Server denied the specified Uniform Resource Locator (URL)
            _ |= re.search(r'The ISA Server denied the specified Uniform Resource Locator (URL)', content,
                           re.I) is not None
            if _:
                return "ISA Server (Microsoft)"
        #若未检测到则抛出异常
        except Exception as e:
            print("IsaServer Problem ", e)

kona

原理:主要依靠header特别字段的正则匹配

class Kona(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
       for item in headers.items():
            #匹配headers中value值是否有AkamaiGHost或Kona
            _ = re.search(r'AkamaiGHost|Kona', item[1], re.I) is not None
            if _:
                return "Kona Web Application Firewall (Akamai)"

modsecurity

原理:主要依靠header特别字段的正则匹配

class Modsecurity(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有d_Security或NOYB
            _ = re.search(r'Mod_Security|NOYB', item[1], re.I) is not None
            if _:
                return "ModSecurity Web Application Firewall (Trustwave)"

netcontinuum

原理:主要依靠header特别字段的正则匹配

class Netcontinuum(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有NCI__SessionId=
            _ = re.search(r'NCI__SessionId=', item[1], re.I) is not None
            if _:
                return "NetContinuum Web Application Firewall (NetContinuum/Barracuda Networks)"

paloalt

原理:主要依靠header特别字段的正则匹配

class Paloalto(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有MISS from PaloAlto
            _ = re.search(r'MISS from PaloAlto', item[1], re.I) is not None
            if _:
                return "Palo Alto Firewall (Palo Alto Networks)"

profense

原理:主要依靠header特别字段的正则匹配

class Profense(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有PLBSID=
            _ = re.search(r'PLBSID=', item[1], re.I) is not None
            #匹配headers中value值是否有Profense
            _ = re.search(r'Profense', item[1], re.I) is not None
            if _:
                return "Profense Web Application Firewall (Armorlogic)"

radware

原理:主要依靠header特别字段的正则匹配

class Radware(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有X-SL-CompState
            _ = re.search(r'X-SL-CompState', item[0], re.I) is not None
            if _:
                return "AppWall Web Application Firewall (Radware)"

requestvalidationmode

原理:主要依靠content特别字段的正则匹配

class Requestvalidationmode(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配content中是否有ASP.NET has detected data in the request that is potentially dangerous
        _ = re.search(r'ASP.NET has detected data in the request that is potentially dangerous', content,
                      re.I) is not NoneRequest Validation has detected a potentially dangerous client input value
        #匹配content中是否有Request Validation has detected a potentially dangerous client input value
        _ |= re.search(r'Request Validation has detected a potentially dangerous client input value', content,
                       re.I) is not None
        if _:
            return "ASP.NET RequestValidationMode (Microsoft)"

safedog

原理:主要依靠header特别字段的正则匹配

class Safedog(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有WAF/2.0
            _ = re.search(r'WAF/2\.0', item[1], re.I) is not None
            #匹配headers中value值是否有Safedog
            _ |= re.search(r'Safedog', item[1], re.I) is not None
            if _:
                return "Safedog Web Application Firewall (Safedog)"

secureiis

原理:主要依靠content特别字段的正则匹配

class Secureiis(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配content中是否有SecureIIS+开始字符<+Web Server Protection
        _ = re.search(r'SecureIIS[^<]+Web Server Protection', content, re.I) is not None
        #匹配content中是否有http://www.eeye.com/SecureIIS/
        _ |= re.search(r'http://www.eeye.com/SecureIIS/', content, re.I) is not None
        #匹配content中是否有?subject=[^>]*SecureIIS Error
        _ |= re.search(r'\?subject=[^>]*SecureIIS Error', content, re.I) is not None
        if _:
            return "SecureIIS Web Server Security (BeyondTrust"

sengnix

原理:主要依靠content特别字段的正则匹配

class Senginx(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配content中是否有SENGINX-ROBOT-MITIGATION
        _ = re.search(r'SENGINX-ROBOT-MITIGATION', content, re.I) is not None
        if _:
            return "SEnginx (Neusoft Corporation)"

sitelock

原理:主要依靠content特别字段的正则匹配

class Sitelock(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配content中是否有SiteLock Incident ID
        _ = re.search(r'SiteLock Incident ID', content, re.I) is not None
        if _:
            return "TrueShield Web Application Firewall (SiteLock)"

sonicwall

原理:主要依靠header特别字段的正则匹配

class Sonicwall(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        #匹配content中是否有This request is blocked by the SonicWALL
        _ = re.search(r'This request is blocked by the SonicWALL', content, re.I) is not None
        #匹配content中是否有Web Site Blocked.+\bnsa_banner      \b匹配一个单词边界
        _ |= re.search(r'Web Site Blocked.+\bnsa_banner', content, re.I) is not None
        if _:
            return "SonicWALL (Dell)"

sucuri

原理:主要依靠header特别字段的正则匹配

class Sucuri(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有Sucuri或Cloudproxy
            _ = re.search(r'Sucuri|Cloudproxy', item[1], re.I) is not None
            #匹配headers中key值是否有X-Sucuri-ID
            _ |= re.search(r'X-Sucuri-ID', item[0], re.I) is not None
            if _:
                return "CloudProxy WebSite Firewall (Sucuri)"

trafficshield

原理:主要依靠header特别字段的正则匹配

class Trafficshield(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有F5-TrafficShield
            _ = re.search(r'F5-TrafficShield', item[1], re.I) is not None
            #或匹配到headers中value值是否有ASINFO=
            _ |= re.search(r'ASINFO=', item[1], re.I) is not None
            if _:
                return "TrafficShield (F5 Networks)"

varnish

原理:主要依靠header特别字段的正则匹配

class Varnish(FingerprintPlugin):

    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中key值是否有X-Varnish
            _ = re.search(r'X-Varnish', item[0], re.I) is not None
            #匹配headers中value值是否有varnish*
            _ |= re.search(r'varnish*', item[1], re.I) is not None
            if _:
                return "Varnish FireWall (OWASP)"

wallarm

原理:主要依靠header特别字段的正则匹配

class Wallarm(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有nginx-wallarm
            _ = re.search(r'nginx-wallarm', item[1], re.I) is not None
            if _:
                return "Wallarm Web Application Firewall (Wallarm)"

webknight

原理:主要依靠header特别字段的正则匹配

class Webknight(FingerprintPlugin):
    def process(self, headers, content):
        _ = False
        for item in headers.items():
            #匹配headers中value值是否有Webknight
            _ = re.search(r'Webknight', item[1], re.I) is not None
            if _:
                return "WebKnight Application Firewall (AQTRONIX)"

0x08 lib/modules/attacks

主要的攻击模块

│   │   ├── attacks
│   │   │   ├── bruteforce
│   │   │   │   ├── admin.py
│   │   │   │   ├── backdoor.py
│   │   │   │   ├── bdir.py
│   │   │   │   ├── bfile.py
│   │   │   │   ├── dir.py
│   │   │   │   ├── file.py
│   │   │   │   ├── __init__.py
│   │   │   │   └── log.py
│   │   │   ├── __init__.py
│   │   │   ├── injection
│   │   │   │   ├── html.py
│   │   │   │   ├── __init__.py
│   │   │   │   ├── ldap.py
│   │   │   │   ├── php.py
│   │   │   │   ├── rfi.py
│   │   │   │   ├── sql.py
│   │   │   │   ├── xpath.py
│   │   │   │   └── xss.py
│   │   │   ├── other
│   │   │   │   ├── allow_method.py
│   │   │   │   ├── dav.py
│   │   │   │   ├── htmlobject.py
│   │   │   │   ├── __init__.py
│   │   │   │   ├── listing.py
│   │   │   │   ├── multipleindex.py
│   │   │   │   ├── phpinfo.py
│   │   │   │   ├── robots.py
│   │   │   │   └── xst.py
│   │   │   └── vulns
│   │   │       ├── anonymous.py
│   │   │       ├── crime.py
│   │   │       ├── __init__.py
│   │   │       ├── shellshock.py
│   │   │       └── strutsshock.py

bruteforce

admin.py

代更

backdoor.py

bdir.py

bfile.py

dir.py

file.py

init.py

log.py

init.py

injection

注入类的漏洞检测

html.py

检查HTML代码注入。思路即:在参数值中添加进html代码,然后检查返回的响应,直接用search(payload,resq.text) 来看能否检测到相应的模式,。若存在则保存URLDATAPAYLOAD,然后输出。

init.py

ldap.py

php.py

rfi.py

sql.py

xpath.py

xss.py

other

allow_method.py

dav.py

htmlobject.py

init.py

listing.py

multipleindex.py

phpinfo.py

robots.py

xst.py

vulns

anonymous.py

crime.py

init.py

shellshock.py

strutsshock.py


转载请注明来源,欢迎对文章中的引用来源进行考证,欢迎指出任何有错误或不够清晰的表达。可以在下面评论区评论,也可以邮件至 951207194@qq.com

文章标题:Read-Sitadel-Source

文章字数:12,538

本文作者:Mang0

发布时间:2019-03-15, 00:00:08

最后更新:2019-03-15, 00:02:38

原始链接:http://mang0.me/archis/f4173bf8/

版权声明: "署名-非商用-相同方式共享 4.0" 转载请保留原文链接及作者。

目录
×

喜欢就点赞,疼爱就打赏