What is state of the art today around input pre-processing?

In March, OpenAI introduced ChatML version 0, a way to structure input messages to an LLM (for the geeks, think IRC to XMPP).

ChatML segregates conversation into different layers or roles (system, assistant, user) which makes it possible for a developer to clearly express who is saying what; i.e. if implemented securely, an untrusted prompt can’t syntactically override that.

This is welcome and establishes with confidence - at the syntax layer - who is saying what in conversational AI.

I can’t help but note two things:

• currently fielded OpenAI models don’t place much emphasis on the “system” messages; which means developers need to provide more message context to avoid fresh user messages overriding the system prompt (!). This situation will improve with new model versions as they will place more weight on the system message
• OpenAI is setting low expectations: they are not claiming this version solves prompt injection, but rather it’s an eventual goal. It may be helpful to think of this as helping defeat syntax-level prompt injections rather than content payloads that exploit particular models' unique emergent properties.