Very insightful! Dr. Chierici, since you ended with a point on orchestration and pipelines: we know that most big companies do not integrate GenAI models via direct API download (from OpenAI, Anthropic etc).
They do so via AI Orchestration Middleware Platforms like Vertex AI or Microsoft Azure.
Such companies are reputable an claim to protect clients with enhanced security and compliance by design, but the underlying foundation models are the same: GPT-4o, Sonnet 3.5, Gemini 2.5 Flash...).
How much can we really trust these Managed AI Orchestration Platforms? Do the same jailbreak techniques work post deployment?
To what extent does Middleware introduce added technical opacity to deployers / integrators downstream?
And, most importantly: if "emergent behaviours" (such as sycophancy and deception) are inherent to the foundation model, can we really do much at the application/ integration level to correct them?
Thank you for reading! Great question, Katalina, and one of the central practical worries for firms buying “managed AI”: how much safety, control and interpretability do orchestration/middleware layers actually add? And where are they just lipstick on a lamprey? I’ll be blunt: managed orchestration helps a lot on engineering, (theoretical) compliance and operational risk (read: satisfying the corporate checklists), but it does not magically remove the core behavioural risks that live inside the foundation model. You still need layered mitigations, continuous testing, and honest risk budgets, including red teaming and evals.
If interested in a deep dive, I can l break that down into what middleware does, what it cannot realistically fix, how jailbreaks behave post-deployment, and practical controls you can actually rely on. But it’ll take me some good time :)
Very insightful! Dr. Chierici, since you ended with a point on orchestration and pipelines: we know that most big companies do not integrate GenAI models via direct API download (from OpenAI, Anthropic etc).
They do so via AI Orchestration Middleware Platforms like Vertex AI or Microsoft Azure.
Such companies are reputable an claim to protect clients with enhanced security and compliance by design, but the underlying foundation models are the same: GPT-4o, Sonnet 3.5, Gemini 2.5 Flash...).
How much can we really trust these Managed AI Orchestration Platforms? Do the same jailbreak techniques work post deployment?
To what extent does Middleware introduce added technical opacity to deployers / integrators downstream?
And, most importantly: if "emergent behaviours" (such as sycophancy and deception) are inherent to the foundation model, can we really do much at the application/ integration level to correct them?
Thank you! 🙏🏼
Thank you for reading! Great question, Katalina, and one of the central practical worries for firms buying “managed AI”: how much safety, control and interpretability do orchestration/middleware layers actually add? And where are they just lipstick on a lamprey? I’ll be blunt: managed orchestration helps a lot on engineering, (theoretical) compliance and operational risk (read: satisfying the corporate checklists), but it does not magically remove the core behavioural risks that live inside the foundation model. You still need layered mitigations, continuous testing, and honest risk budgets, including red teaming and evals.
If interested in a deep dive, I can l break that down into what middleware does, what it cannot realistically fix, how jailbreaks behave post-deployment, and practical controls you can actually rely on. But it’ll take me some good time :)
Thank you! Sounds like an amazing (paid subscriber) post to look forward to :D.
Good call, I will certainly follow up ;)