Fuzz in sixty seconds

Note that this blog post was updated on the 21st of November to address issues in the original fuzz.cmd file and to update it to work with the latest version of Bug­Id. You can download the latest version here.

Introduction

In this blog post I am going to help you get started with fuzzing browsers as fast as possible. The goal is to be up and running with minimal thought and effort. It'll be quick and dirty, using publicly available tools to save time.

You will be able to fuzz all browsers common on the Windows Desktop: Google Chrome, Microsoft Edge, Mozilla Firefox, or Microsoft Internet Explorer. I may have exagerate a little when I said sixty seconds in the title but it'll be less than sixty minutes for sure.

I will show you how to create a script that can fuzz a browser continuously. It will generate HTML files and load then in a browser to detect any crashes while they are rendered. For each unique issue detected, a report will be generated that contains information about the type of crash, location and its potential security impact. It will also save the HTML files that triggered the crash, so you can reproduce the issue later for further manual analysis. If the browser survives loading all generated HTML files it will be terminated automatically. After all this the script will start over, generating new files and testing them continuously.

Required Software and Tools

To generate HTML files, I'll be using Domato. Domato is a Python script that can generate HTML files for the express purpose of triggering security issues while rendering them. It is written and maintained by Ivan Fratric of Google Project Zero.

To run the browser and detect crashes, I'll be using Bug­Id. Bug­Id is a Python script that can run an application in a debugger to detect and analyze crashes and recognize security issues. It is written and maintained by myself.

Our target browser will be any the four browsers most commonly used on Windows: Google Chrome, Microsoft Edge, Mozilla Firefox, or Microsoft Internet Explorer. We will be using the stock browser with as little changes in the configuration as possible to guarantee that any issue we find will actually affect users of these browsers and not just our special build or configuration.

Setup

I will assume you have a freshly installed copy of Windows 10 running in a VM without any software installed. Both Domato and Bug­Id are written in Python 2, so we need to download that. At the time of this writing, the download page for Python was available at https://www.python.org/downloads/. Make sure you download Python 2 and not 3; the later is newer but not entirely compatible with Python 2. If you are running a 64-bit version of Windows, make sure you download the 64-bit version of Python too or you will not be able to debug 64-bit applications. After downloading the installer, you can run it and leave all settings to default.

Next up is Debugging Tools for Windows from Microsoft. This is used by Bug­Id to run the browser in a debugger. It comes as part of the Windows 10 Software Development Kit (SDK), which you can download from https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk. After downloading the installer, you can run it and disable everything except the "Debugging Tools for Windows"; it is all we need from the SDK. You can leave all other settings to default.

Finally we need to get Domato and Bug­Id. You do not need to install these; you can simply download them and then run them directly. We will create a folder C:\Fuzzing in which we'll store everything.

You can download the most recent version of Domato from https://github.com/googleprojectzero/domato/archive/master.zip. After downloading unzip the file into C:\Fuzzing. You should now have a folder C:\Fuzzing\domato-master which contains generate.py among many other files.

You can download the most recent release version of Bug­Id from https://github.com/Sky­Lined/Bug­Id/releases. Make sure you download the first "Asset"; a zip file called Bug­Id.####-##-##.zip and not the source code; Bug­Id has some dependencies that are included in the release zip but not the source code zip. Create a folder C:\Fuzzing\Bug­Id and unzip the downloaded file there. You should now have a file C:\Fuzzing\Bug­Id\Bug­Id.py among the files in that folder.

You may want to install Google Chrome and/or Mozilla Firefox if you want to fuzz them; Microsoft Edge and Internet Explorer come installed by default. If you are installing Firefox on a 64-bit version of Windows, you may want to make sure you download the 64-bit version of Firefox from https://www.mozilla.org/en-US/firefox/all/ because debugging 32-bit applications on 64-bit Windows using page-heap will produce less reliable results.

Configure

Many security issues cause corruption of data inside a browser process. This may not result in a crash immediately if the browser does not use that data until much later. To increase the chances of detecting such issues, Bug­Id uses a feature called page-heap. This needs to be enabled manually for each binary that is part of a browser, which requires administrator privileges. Bug­Id comes with a script that can do this so you don't have to worry about the details. Open a cmd.exe prompt as Administrator, go to C:\Fuzzing\Bug­Id and run any of the following commands to enable page-heap in various browsers:

Page­Heap.cmd edge ON
Page­Heap.cmd chrome ON
Page­Heap.cmd firefox ON
Page­Heap.cmd msie ON

You do not need to enable it for all browsers if you are not planning to fuzz them but it does not hurt either. The output should look like this:

Note that Bug­Id itself does not require Administrator privileges; just Page­Heap.cmd.

We should now have everything we need on the machine to start fuzzing. Let's make sure everything is working by running the following commands:

"c:\Python27\python.exe" -c "print 'ok!'"
"c:\Python27\python.exe" "domato-master\generator.py"
"Bug­Id\Bug­Id.cmd" "%Win­Dir%\system32\cmd.exe" --c­Bug­Id.b­Ensure­Page­Heap=false -- /C ECHO ok!

The output should look like this:

Fuzzing script

We'll create the batch script C:\Fuzzing\Fuzz.cmd to do our fuzzing. This batch script will continously loop through commands to run Domato to generate HTML files and run Bug­Id to test them in the browser. If a crash is detected, it will save the generated HTML files and crash report in a separate folder for later manual analysis. You can download it [here][(xxx). It consists of four sections.

The first section of our batch script contains some variables that allow us to configure fuzzing:

@ECHO OFF
SET BASE_­FOLDER=C:\Fuzzing
SET PYTHON_­EXE=C:\Python27\python.exe

:: What browser do we want to fuzz? ("chrome" | "edge" | "firefox" | "msie")
SET TARGET_­BROWSER=edge
:: How many HTML files shall we teach during each loop?
SET NUMBER_­OF_­FILES=100
:: How long does it take Bug­Id to start the browser and load an HTML file?
SET BROWSER_­LOAD_­TIMEOUT_­IN_­SECONDS=30
:: How long does it take the browser to render each HTML file?
SET AVERAGE_­PAGE_­LOAD_­TIME_­IN_­SECONDS=2

:: Optionally configurable
SET BUGID_­FOLDER=%BASE_­FOLDER%\Bug­Id
SET DOMATO_­FOLDER=%BASE_­FOLDER%\domato-master
SET TESTS_­FOLDER=%BASE_­FOLDER%\Tests
SET REPORT_­FOLDER=%BASE_­FOLDER%\Report
SET RESULT_­FOLDER=%BASE_­FOLDER%\Results
:: Store our results in a folder named after the target:
IF NOT EXIST "%RESULT_­FOLDER%\%TARGET_­BROWSER%" MKDIR "%RESULT_­FOLDER%\%TARGET_­BROWSER%"

This should all be reasonably self explanatory. You may want to change the browser you are targetting and modify the timeout and average page load time based on your experience, as they depend greatly on the hardware you are running on. Tests are generated in the TESTS_­FOLDER folder and if we find any crashes, we store information about them in the RESULT_­FOLDER folder.

Next we'll add the main loop to our batch script:

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Repeatedly generate tests and run them in the browser.
:LOOP
  CALL :GENERATE
  IF ERRORLEVEL 1 EXIT /B 1
  CALL :TEST
  IF ERRORLEVEL 1 EXIT /B 1
  GOTO :LOOP

In the :GENERATE label we'll ask Domato to generate test HTML files in the "Tests" folder.

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Generate test HTML files
:GENERATE
  REM Delete old files.
  DEL "%TESTS_­FOLDER%\fuzz-*.html" /Q >nul 2>nul
  REM Generate new HTML files.
  "%PYTHON_­EXE%" "%DOMATO_­FOLDER%\generator.py" --output_­dir "%TESTS_­FOLDER%" --no_­of_­files %NUMBER_­OF_­FILES%
  IF ERRORLEVEL 1 EXIT /B 1
  EXIT /B 0

In the :TEST label we'll start the browser in Bug­Id and get it to load the test HTML files. Bug­Id will exit when the application crashes, or has ran without crashing for a given number of seconds. When can check Bug­Id's exit code to determine what happened and then look for the HTML report file:

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Run browser in Bug­Id and load test HTML files
:TEST
  REM Delete old report if any.
  IF NOT EXIST "%REPORT_­FOLDER%" (
    MKDIR "%REPORT_­FOLDER%"
  ) ELSE (
    DEL "%REPORT_­FOLDER%\*.html" /Q >nul 2>nul
  )
  REM Guess how long the browser needs to run to process all tests.
  REM This is used by Bug­Id to terminate the browser in case it survives all tests.
  SET /A MAX_­BROWSER_­RUN_­TIME=%BROWSER_­LOAD_­TIMEOUT_­IN_­SECONDS% + %AVERAGE_­PAGE_­LOAD_­TIME_­IN_­SECONDS% * %NUMBER_­OF_­FILES%
  REM Start browser in Bug­Id...
  "%PYTHON_­EXE%" "%BUGID_­FOLDER%\Bug­Id.py" "%TARGET_­BROWSER%" "--s­Report­Folder­Path=\"%REPORT_­FOLDER:\=\\%\"" --n­Application­Max­Run­Time=%MAX_­BROWSER_­RUN_­TIME% -- "file://%TESTS_­FOLDER%\index.html"

  IF ERRORLEVEL 2 (
    ECHO - ERROR %ERRORLEVEL%.
    REM ERRORLEVEL 2+ means something went wrong.
    ECHO Please fix the issue before continuing...
    EXIT /B 1
  ) ELSE IF NOT ERRORLEVEL 1 (
    EXIT /B 0
  )
  ECHO Crash detected!

  REM Create results sub-folder based on report file name and copy test files
  REM and report.
  FOR %%I IN ("%REPORT_­FOLDER%\*.html") DO (
    CALL :COPY_­TO_­UNIQUE_­CRASH_­FOLDER "%RESULT_­FOLDER%\%%~nx­I"
    EXIT /B 0
  )
  ECHO Bug­Id reported finding a crash, but not report file could be found!?
  EXIT /B 1

Finally, in the :COPY_­TO_­UNIQUE_­CRASH_­FOLDER label we will copy the HTML report, which has a name that is unique to the bug, to a folder with the same name. We will also copy the tests files that triggered the crash to the "Repro" sub-folder. If Bug­Id detect the same issue during a previous fuzzing run, the folder already exists and the result will be ignored.

:COPY_­TO_­UNIQUE_­CRASH_­FOLDER
  SET REPORT_­FILE=%~nx1
  REM We want to remove the ".html" extension from the report file name to get
  REM a unique folder name:
  SET UNIQUE_­CRASH_­FOLDER=%RESULT_­FOLDER%\%TARGET_­BROWSER%\%REPORT_­FILE:~0,-5%
  IF EXIST "%UNIQUE_­CRASH_­FOLDER%" (
    ECHO Repro and report already saved after previous test detected the same issue.
    EXIT /B 0
  )
  ECHO Copying report and repro to %UNIQUE_­CRASH_­FOLDER% folder...
  REM Move report to unique folder
  MKDIR "%UNIQUE_­CRASH_­FOLDER%"
  MOVE "%REPORT_­FOLDER%\%REPORT_­FILE%" "%UNIQUE_­CRASH_­FOLDER%\report.html"
  REM Copy repro
  MKDIR "%UNIQUE_­CRASH_­FOLDER%\Repro"
  COPY "%TESTS_­FOLDER%\*.html" "%UNIQUE_­CRASH_­FOLDER%\Repro"
  ECHO Report and repro copied to %UNIQUE_­CRASH_­FOLDER% folder.
  EXIT /B 0

This completes our fuzzing script fuzz.cmd. You can download the complete script here.

Domato simply generates a bunch of HTML files. In order to get the browser to load them all in sequence, we'll create a separate HTML file in C:\Fuzzing\Tests\index.html. Here's an example of an HTML file that will load the tests generated by Domato in iframes and give each one 5 seconds to finish loading. Some pages may load a lot faster, others will time out. After letting it run a few times, I've set AVERAGE_­PAGE_­LOAD_­TIME_­IN_­SECONDS to 2 in the script above as that seems to be a reasonable average.

<!doctype html>
<!-- saved from url=(0014)about:internet -->
<html>
  <head>
    <script>
      var o­IFrame­Element = document.get­Element­By­Id("IFrame"),
          n­Page­Load­Timeout­In­Seconds = 5,
          u­Index = 0;
      onload = function f­Load­Next() {
        var s­Index = "" + u­Index++;
        while (s­Index.length < 5) s­Index = "0" + s­Index;
        var s­Test­URL = "fuzz-" + s­Index + ".html";
        document.title = "Loading test " + s­Test­URL + "...";
        // Add iframe element that loads the next test case.
        var o­IFrame = document.body.append­Child(document.create­Element("iframe")),
            b­Finished = false;
        o­IFrame.set­Attribute("sandbox", "allow-scripts");
        o­IFrame.set­Attribute("src", s­Test­URL);
        // Hook load event handler and add timeout to remove the iframe when the test is finished.
        try {
          o­IFrame.content­Window.add­Event­Listener("load", f­Cleanup­And­Load­Next);
        } catch (e) {
          // This may cause an exception because some browsers treat different files loaded from the
          // local file system as comming from different origins.
        };
        var x­Timeout = set­Timeout(f­Cleanup­And­Load­Next, n­Page­Load­Timeout­In­Seconds * 1000);
        function f­Cleanup­And­Load­Next() {
          // Both the load event and the timeout can call this function; make sure we only execute once:
          if (!b­Finished) {
            b­Finished = true;
            console.log("Finished test " + s­Test­URL + "...");
            // Let's give the page another 5 seconds to render animations etc.
            set­Timeout(function() {
              // Remove the iframe from the document to delete the test.
              try {
                document.body.remove­Child(o­IFrame);
              } catch (e) {};
            }, 5000);
            f­Load­Next();
          };
        };
      };
    </script>
  </head>
  <body>
  </body>
</html>

You can download this file here. (Note: this file was updated on April 2nd, 2019 to add a try ... catch-block around the add­Event­Listener call to prevent an exception in certain browsers.)

That's it! You are now ready to start fuzzing. Simply run the script we created and watch it generate HTML files and load them in the browser over and over...

fuzz.cmd

Now what?

Once a crash is found, a folder is created by Bug­Id that contains a HTML report with details about the type of crash and the location in the code where the crash occured, along with some other useful information. Don't worry if you do not understand all the details in this report. The most important part from a security standpoint is in the top section, after the heading "Security impact". If this says "Potentially exploitable security issue", you have most likely found a security issue. If this says "Denial of service" it is most likely just a regular bug with no serious security implications. You are likely to find a lot of Access Violation while attempting to Read memory using a NULL pointer, which will show up as AVR@NULL. This is a common bug, but not a security issue. Other common bugs that are not security issues include OOM (Out Of Memory), Assert (The application noticed something was not as expected and terminated itself), *Recursive­Call (a function calls itself over and over until it runs out of stack memory), and Integer­Divide­By­Zero (the application tried to divide by zero).

If you do find security security issue, it is likely to be reported as RAF/WAF (Read/Write After Free; the application thought it was done using a chunk of memory and freed it but continued to use it afterwards) or OOBR/OOBW (Out Of Bounds Read/Write; the application try to read/write beyond the bounds of a memory chunk).

Please report all security issues to the browser vendor as soon as possible. If you don't know exactly what to report, zipping the report and repro files in the results folder and sending it to the vendor should provide them with enough information to reproduce, analyze and fix the issue. They may even decide to give you a nice bug bounty as a thank you for helping them secure their users!

You can report security issues in the following locations:

• Google Chrome Go to https://bugs.chromium.org/p/chromium/issues/list and click "New issue" at the top right. You will have to create/sign in with a Google account. Make sure that you select "Security (Issues related to the security of the browser)" during step 2.
• Microsoft Edge and Internet Explorer Please send an email to the Microsoft Security Response Center at secure@microsoft.com. If you want, you can use PGP to encrypt your conversation, the key is available at https://www.microsoft.com/en-us/msrc/pgp-key-msrc.
• Mozilla Firefox Go to https://bugzilla.mozilla.org/enter_­bug.cgi?product=Firefox. You will have to create an account or sign in with an existing account. Make sure you check the "Security" check-box at the bottow of the form.

Unattended distributed fuzzing

The best thing about fuzzing is that you get to let machines do all the hard work. The more machines you use, the bigger your chances of finding bugs. It is quite easy to use the script created in this blog post to fuzz on multiple machines at the same time and save results in a single location. All we need to do is create a shared folder on a server to store the results and change the RESULT_­FOLDER in our script to point to this network share. This allows you to run the script on as many machines as you want and collect have all of them report the crashes they find to a single folder. For example:

SET RESULT_­FOLDER=\\server\Fuzzing\Results

If you are going to run this script on multiple machines, you may want to have these machines automatically log-in as a particular user and start the script whenever they are started. You can do that by executing the following commands in an Administrator command prompt:

SET USERNAME=<Username>
SET PASSWORD=<Password>
SET WINLOGON=HKLM\Software\Microsoft\Windows NT\Current­Version\Winlogon
SET RUN=HKLM\Software\Microsoft\Windows\Current­Version\Run
REG ADD "%WINLOGON%" /v Default­User­Name /t REG_­SZ /d "%USERNAME%" /f >nul
REG ADD "%WINLOGON%" /v Default­Password /t REG_­SZ /d "%PASSWORD%" /f >nul
REG ADD "%WINLOGON%" /v Auto­Admin­Logon /t REG_­SZ /d 1 /f >nul
REG ADD "%RUN%" /v Fuzz /d "\"%COMSPEC%\" /K \"C:\Fuzzing\fuzz.cmd\"" /f >nul

Note that you need to modify these command to provide the username and password for your machine. When you are done, you can test if it works by rebooting the machine: it should automatically log in and start fuzzing.

Afterword

The above set-up will get you started quickly, but not very efficiently. In a future blog post I will show you how to improve on this basic concept.

Also, we are using a stock public fuzzer, which has been run extensively on many, many machines for a very long time. This means your chances of finding a vulnerability are small. But you can improve this fuzzer by adding to the "Grammar" it uses to generate tests and start looking for bugs in new features where no-one may have looked before.

If you want to create your own fuzzers or improve Domato, have a look at the files it generates. It should give you a good idea of the kind of tests you will want to generate yourself in order to find bugs.

For now, I wish you happy fuzzing and good luck finding vulnerabilities!