Testing build rate with more refined header file info #641
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
**Statement**: Most modifications to the original code of `oss-fuzz-gen` (including items 2. and 3. below) are for the convenience of current testing and performance comparison based on Gemini. The current changes will not be the final merge changes. Modifications to be noted： 1. Changed c projects like **picotls**, **libvnc**'s language setting in yaml from c++ to c. Otherwise, the prompt will provide an example of c++, and the large model imitation the c++ example to include FuzzydDataProvider. h (c++). 2. Added `headerfiles `project as a module in the oss-fuzz-gen project. Therefore, the include statement changed from _import headerfiles.api as headerfiles_ to _from headerfiles.headerfiles import api as headerfiles_. (This facilitates us to adjust the code in the `headerfiles` at any time, and eventually, we will package it as an external library) 3. Changed the function: https://github.com/occia/oss-fuzz-gen/blob/e71091bab8b4ac20a2e575ee9f7cbce91a987fdd/data_prep/project_src.py#L238 to avoid the bug: "docker: Error response from daemon: Conflict." 4. Project **bind9**: Execute _make "-j\$(nproc)"_ in original build.sh sometimes cause link errors, related to the setting of multithreading in the project. Execute _make_ produce no errors (by headersfile_updated_script). 5. Project **openexr**: The header files introduced by `headerfiles` will be part of the prompt, occasionally affecting the generation of LLM. Haven't found a solution yet. _"We have prepared the following list of headers which covers all target project APIs and will prepend them as #include statments at the beginning of your generated fuzz target. Therefore, you only need to include the headers of non-target-project APIs used in your fuzz target. \<code\> dns/acl.h..."_. Overall Results: (Based on GPT-4o) PROJ | ORI | FIX -- | -- | -- avahi | 16 | 18 bind9 | 21 | 14 bluez | 0 | 0 brotli | 0 | 1 capstone | 35 | 50 coturn | 16 | 18 croaring | 42 | 50 igraph | 0 | 0 kamailio | 6 | 28 krb5 | 0 | 0 lcms | 0 | 37 libbpf | 0 | 39 libcoap | 0 | 0 libevent | 4 | 28 libfido2 | 0 | 50 libical | 6 | 16 libjpeg-turbo | 39 | 50 libpcap | 50 | 50 librdkafka | 0 | 0 libsndfile | 39 | 45 libsodium | 0 | 0 libssh2 | 26 | 25 libssh | 22 | 32 libtpms | 40 | 40 libusb | 1 | 27 libvnc | 0 | 24 libxls | 0 | 33 libyang | 1 | 0 lwan | 0 | 8 mbedtls | 0 | 16 mdbtools | 0 | 0 minizip | 50 | 50 ndpi | 1 | 7 njs | 2 | 0 oniguruma | 20 | 20 openexr | 17 | 0 opusfile | 23 | 36 picotls | 43 | 41 pjsip | 14 | 19 proftpd | 23 | 44 pupnp | 29 | 29 sleuthkit | 0 | 0 tidy-html5 | 35 | 39 unicorn | 0 | 0 unit | 15 | 13 utf8proc | 20 | 20 vlc | 6 | 9 w3m | 34 | 30 wasm3 | 5 | 15 zydis | 0 | 0 --------- Co-authored-by: Cen Zhang <[email protected]>
|
/gcbrun exp -n dg |
|
/gcbrun exp -n dg -b headerfiles |
|
/gcbrun exp -n hd -b headerfiles |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.