I mentioned last week that I’m a big fan of “bad guy” unit tests to improve software security. To recap, these adversarial unit tests check for security edge cases in source code (“what if I call that function with a null value at the end of the filename?”). In my experience, even developers that are fans of unit testing rarely write “bad guy” ones.

ChatGPT is a super fast way to generate regular and adversarial unit tests from the source code you feed in.

However, the apparent fly in the ointment is companies with proprietary source code will not want to copypasta their intellectual property to OpenAI’s - or anyone else’s - public AI.

My initial reaction was to drop this use case into the on-premise AI bucket. But then the insight came: what if I extract just the function metadata from the source code I want to generate unit tests for?

Translation: take the information that describes how a programmer would call into a unit of code (a function) and what the programmer would receive back, rather than the actual source itself.

Would that be sufficient to generate meaningful security unit tests?

Naturally, I had ChatGPT generate the 50-line python script to derive that information. For the geeks, I generated an Abstract Syntax Tree (AST) from a sample python script and extracted the function metadata and docstrings into a JSON file. This is all executed client-side, i.e. no exposure of source code.

That was a mouthful; what does that look like? Here is a sample:

{     "name": "find_enclosing_rectangle",     "args": [      "points"     ],     "docstring": "Find the smallest rectangle that encloses a list of points.\n\nArgs:\n  points (List[Point]): The list of points to enclose.\n\nReturns:\n  Rectangle: The smallest rectangle that encloses the points.\n\nRaises:\n  ValueError: If no points are given.",     "returns": {      "type": null,      "value": null     },     "filename": "samples/sample.py"    }

The next step was to write a suitable prompt for ChatGPT and paste in just the JSON data.

ChatGPT then quickly got to work generating 10 adversarial and regular unit tests...all without access to the "secret" source code. I reviewed the unit tests and the output was solid. I pasted the code generated by ChatGPT into a test_sample_test.py file and executed it using command provided by ChatGPT.

All tests passed bar an injection test. My sample function had a defect. A positive result for testing - I fixed the input handling, and all tests passed.

Now, this is just an interactive MVP for Python code. It doesn't handle opaque object passing and the like...but the beauty of AST means this approach can work with PHP, Java, Go etc.

In practice, a risk-based approach would lean towards confining this effort to sensitive functions, i.e. those that receive untrusted input and implement key security controls and security features.

To me, this is evidence that with a little creativity we will likely find more security use cases suitable for public AI today.

P.S. unit tests are generally not shipped to customers, which conveniently sidesteps a potential licensing or intellectual property infringement problem that prevents some companies from shipping AI-generated code to users or devices, as previously noted by a reader (Hi A!).