试用clusterfuzzlite

用户1423082

发布于 2024-12-31 20:14:16

5400

代码可运行

文章被收录于专栏：giantbranch's bloggiantbranch's blog

运行总次数：0

代码可运行

clusterfuzzlite是是一种持续的模糊测试解决方案，作为持续集成 (CI) 工作流的一部分运行，比如我们一旦push代码，便可以自动build，之后自动fuzz。

比如它支持GitHub Actions，GitLab，Google Cloud Build和Prow，我们最常见到的应该是GitHub Actions

如何让项目支持clusterfuzzlite呢

ClusterFuzzLite 重用了 OSS-Fuzz 工具链来简化构建。这意味着 ClusterFuzzLite 将在 docker 容器中构建您的项目。所以加入你熟悉OSS-Fuzz，这就看着很像了。只不过多了一些参数，比如–external

有一个问题就是他这个只支持libfuzzer！！！，所以局限性还是有的。

环境配置及文件编辑

首先环境的配置好：

python什么的必备
安装好docker
下载oss-fuzz： git clone https://github.com/google/oss-fuzz.git

首先需要在项目根目录新建.clusterfuzzlite文件夹，包含下面三个文件

.clusterfuzzlite/project.yaml
.clusterfuzzlite/Dockerfile
.clusterfuzzlite/build.sh

上面的文件不用我们新建，可以使用命令新建模板

$ cd /path/to/oss-fuzz
$ export PATH_TO_PROJECT=<path_to_your_project>
$ python infra/helper.py generate --external --language=c++ $PATH_TO_PROJECT

以https://github.com/libexpat/libexpat.git为例

cd / && git clone https://github.com/libexpat/libexpat.git expat
export PATH_TO_PROJECT=/expat
python infra/helper.py generate --external --language=c $PATH_TO_PROJECT

主要编辑build.sh文件即可，主要最后编译出来的fuzzer需要复制到$OUT即可

cd $SRC/expat/

: ${LD:="${CXX}"}
: ${LDFLAGS:="${CXXFLAGS}"}  # to make sure we link with sanitizer runtime

cmake_args=(
    # Specific to Expat
    -DEXPAT_BUILD_FUZZERS=ON
    -DEXPAT_OSSFUZZ_BUILD=ON
    -DEXPAT_SHARED_LIBS=OFF

    # C compiler
    -DCMAKE_C_COMPILER="${CC}"
    -DCMAKE_C_FLAGS="${CFLAGS}"

    # C++ compiler
    -DCMAKE_CXX_COMPILER="${CXX}"
    -DCMAKE_CXX_FLAGS="${CXXFLAGS}"

    # Linker
    -DCMAKE_LINKER="${LD}"
    -DCMAKE_EXE_LINKER_FLAGS="${LDFLAGS}"
    -DCMAKE_MODULE_LINKER_FLAGS="${LDFLAGS}"
    -DCMAKE_SHARED_LINKER_FLAGS="${LDFLAGS}"
)

mkdir -p build
cd build
cmake ../expat "${cmake_args[@]}"
make -j$(nproc)

for fuzzer in fuzz/*;
do
  cp $fuzzer $OUT
done

本地测试

上搞完就可以本地测试了

构建您的 docker 镜像和 fuzz 目标

模板命令

$ python infra/helper.py build_image --external $PATH_TO_PROJECT
$ python infra/helper.py build_fuzzers --external $PATH_TO_PROJECT --sanitizer <address/undefined/memory>

实际命令

python infra/helper.py build_image --external $PATH_TO_PROJECT
python infra/helper.py build_fuzzers --external $PATH_TO_PROJECT --sanitizer address

假如没问题说明build.sh写得没问题，环境库依赖也没问题

通过运行 check_build 命令查找要修复的常见构建问题

这将检查您的模糊测试目标是否使用正确的sanitizer编译，并且在模糊测试几秒钟后不会崩溃。

$ python infra/helper.py check_build --external $PATH_TO_PROJECT --sanitizer <address/undefined/memory>

运行特定的模糊测试目标，使用 run_fuzzer

$ python infra/helper.py run_fuzzer --external --corpus-dir=<path-to-temp-corpus-dir> $PATH_TO_PROJECT <fuzz_target>

<path-to-temp-corpus-dir>就是宿主机你自己准备好的种子文件路径 <fuzz_target>是编译出来的fuzzer的名字

如果您打算使用 ClusterFuzzLite 的代码覆盖率报告功能，最好测试覆盖率报告生成是否有效。这将使用本地语料库目录中上一个 run_fuzzer 步骤生成的语料库。

$ python infra/helper.py build_fuzzers --external --sanitizer coverage $PATH_TO_PROJECT
$ python infra/helper.py coverage --external $PATH_TO_PROJECT --fuzz-target=<fuzz_target> --corpus-dir=<path-to-temp-corpus-dir>

步骤4是可选的，run_fuzzer没问题即可就行了

Running ClusterFuzzLite

ClusterFuzzLite可以以pull request为触发，也可以cron定时触发

下面就以GitHub Actions为例了，需要在.github/workflows目录新建文件

.github/workflows/cflite_pr.yml (for PR fuzzing, pull request为触发)
.github/workflows/cflite_build.yml (for continuous builds, 用于持续构建，push代码就执行build)
.github/workflows/cflite_batch.yml (for batch fuzzing，用于批量模糊测试)
.github/workflows/cflite_cron.yml(for tasks done on a cron schedule: pruning and coverage)

之后展示了一些默认的配置设置，默认配置已经适用于大多数项目

.github/workflows/cflite_pr.yml

name: ClusterFuzzLite PR fuzzing
on:
  pull_request:
    paths:
      - '**'
permissions: read-all
jobs:
  PR:
    runs-on: ubuntu-latest
    concurrency:
      group: ${{ github.workflow }}-${{ matrix.sanitizer }}-${{ github.ref }}
      cancel-in-progress: true
    strategy:
      fail-fast: false
      matrix:
        sanitizer:
        - address
        # Override this with the sanitizers you want.
        # - undefined
        # - memory
    steps:
    - name: Build Fuzzers (${{ matrix.sanitizer }})
      id: build
      uses: google/clusterfuzzlite/actions/build_fuzzers@v1
      with:
        language: c++ # Change this to the language you are fuzzing.
        github-token: ${{ secrets.GITHUB_TOKEN }}
        sanitizer: ${{ matrix.sanitizer }}
        # Optional but recommended: used to only run fuzzers that are affected
        # by the PR.
        # See later section on "Git repo for storage".
        # storage-repo: https://${{ secrets.PERSONAL_ACCESS_TOKEN }}@github.com/OWNER/STORAGE-REPO-NAME.git
        # storage-repo-branch: main   # Optional. Defaults to "main"
        # storage-repo-branch-coverage: gh-pages  # Optional. Defaults to "gh-pages".
    - name: Run Fuzzers (${{ matrix.sanitizer }})
      id: run
      uses: google/clusterfuzzlite/actions/run_fuzzers@v1
      with:
        github-token: ${{ secrets.GITHUB_TOKEN }}
        fuzz-seconds: 600
        mode: 'code-change'
        sanitizer: ${{ matrix.sanitizer }}
        # Optional but recommended: used to download the corpus produced by
        # batch fuzzing.
        # See later section on "Git repo for storage".
        # storage-repo: https://${{ secrets.PERSONAL_ACCESS_TOKEN }}@github.com/OWNER/STORAGE-REPO-NAME.git
        # storage-repo-branch: main   # Optional. Defaults to "main"
        # storage-repo-branch-coverage: gh-pages  # Optional. Defaults to "gh-pages".

字段解析：

language：更改为目标代码的语言 sanitizers：更改或启用更多消毒剂。 fuzz-seconds：更改模糊测试的时间。 parallel-fuzzing：使用并行模糊测试。

接下是.github/workflows/cflite_batch.yml

name: ClusterFuzzLite batch fuzzing
on:
  schedule:
    - cron: '0 0/6 * * *'  # Every 6th hour. Change this to whatever is suitable.
permissions: read-all
jobs:
  BatchFuzzing:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        sanitizer:
        - address
        # Override this with the sanitizers you want.
        # - undefined
        # - memory
    steps:
    - name: Build Fuzzers (${{ matrix.sanitizer }})
      id: build
      uses: google/clusterfuzzlite/actions/build_fuzzers@v1
      with:
        language: c++ # Change this to the language you are fuzzing.
        sanitizer: ${{ matrix.sanitizer }}
    - name: Run Fuzzers (${{ matrix.sanitizer }})
      id: run
      uses: google/clusterfuzzlite/actions/run_fuzzers@v1
      with:
        github-token: ${{ secrets.GITHUB_TOKEN }}
        fuzz-seconds: 3600
        mode: 'batch'
        sanitizer: ${{ matrix.sanitizer }}
        # Optional but recommended: For storing certain artifacts from fuzzing.
        # See later section on "Git repo for storage".
        # storage-repo: https://${{ secrets.PERSONAL_ACCESS_TOKEN }}@github.com/OWNER/STORAGE-REPO-NAME.git
        # storage-repo-branch: main   # Optional. Defaults to "main"
        # storage-repo-branch-coverage: gh-pages  # Optional. Defaults to "gh-pages".

.github/workflows/cflite_build.yml

name: ClusterFuzzLite continuous builds
on:
  push:
    branches:
      - main  # Use your actual default branch here.
permissions: read-all
jobs:
  Build:
   runs-on: ubuntu-latest
   concurrency:
     group: ${{ github.workflow }}-${{ matrix.sanitizer }}-${{ github.ref }}
     cancel-in-progress: true
   strategy:
     fail-fast: false
     matrix:
        sanitizer:
        - address
        # Override this with the sanitizers you want.
        # - undefined
        # - memory
   steps:
   - name: Build Fuzzers (${{ matrix.sanitizer }})
     id: build
     uses: google/clusterfuzzlite/actions/build_fuzzers@v1
     with:
        language: c++ # Change this to the language you are fuzzing.
        sanitizer: ${{ matrix.sanitizer }}
        upload-build: true

.github/workflows/cflite_cron.yml

name: ClusterFuzzLite cron tasks
on:
  schedule:
    - cron: '0 0 * * *'  # Once a day at midnight.
permissions: read-all
jobs:
  Pruning:
    runs-on: ubuntu-latest
    steps:
    - name: Build Fuzzers
      id: build
      uses: google/clusterfuzzlite/actions/build_fuzzers@v1
      with:
        language: c++ # Change this to the language you are fuzzing
    - name: Run Fuzzers
      id: run
      uses: google/clusterfuzzlite/actions/run_fuzzers@v1
      with:
        github-token: ${{ secrets.GITHUB_TOKEN }}
        fuzz-seconds: 600
        mode: 'prune'
        # Optional but recommended.
        # See later section on "Git repo for storage".
        # storage-repo: https://${{ secrets.PERSONAL_ACCESS_TOKEN }}@github.com/OWNER/STORAGE-REPO-NAME.git
        # storage-repo-branch: main   # Optional. Defaults to "main"
        # storage-repo-branch-coverage: gh-pages  # Optional. Defaults to "gh-pages".
  Coverage:
    runs-on: ubuntu-latest
    steps:
    - name: Build Fuzzers
      id: build
      uses: google/clusterfuzzlite/actions/build_fuzzers@v1
      with:
        language: c++ # Change this to the language you are fuzzing.
        sanitizer: coverage
    - name: Run Fuzzers
      id: run
      uses: google/clusterfuzzlite/actions/run_fuzzers@v1
      with:
        github-token: ${{ secrets.GITHUB_TOKEN }}
        fuzz-seconds: 600
        mode: 'coverage'
        sanitizer: 'coverage'
        # Optional but recommended.
        # See later section on "Git repo for storage".
        # storage-repo: https://${{ secrets.PERSONAL_ACCESS_TOKEN }}@github.com/OWNER/STORAGE-REPO-NAME.git
        # storage-repo-branch: main   # Optional. Defaults to "main"
        # storage-repo-branch-coverage: gh-pages  # Optional. Defaults to "gh-pages".

之后将代码都提交github，之后在Actions那里可以看到Workflow