AI readiness

Adopt AI on a stable foundation

AI readiness is your organization's ability to safely, compliantly, and productively build, ship, and operate AI systems, scaling value while minimizing risk.

This guide walks you through how to configure Cortex for AI readiness. Each step below builds on the last, but you can adopt them incrementally based on your organization's priorities.

Step 1: Ingesting data and establishing ownership

Every other step depends on having your services in Cortex with clear ownership assigned. Without this, Scorecards can't surface the right teams, Workflows can't route correctly, and Eng Intelligence won't reflect reality.

What to do

Import your services, infrastructure, and other entities into Cortex. Assign an owner (team or individual) to each entity. Then connect the integrations that give Cortex visibility into how those entities are built, tested, and deployed.

For AI-specific entities like models or pipelines, consider creating a custom entity type (e.g. "AI Model") to group and track them separately.

Integrations to prioritize

Category	Tools
Version control	GitHub, GitLab, Bitbucket, Azure DevOps
Project management	Jira, GitHub, ClickUp, Azure DevOps
Code quality and security	Snyk, SonarQube, Codecov, Checkmarx, Veracode, Wiz
Documentation	Link runbooks and external docs to each entity

You'll know this step is done when...

Every entity in Cortex has an owner assigned, and your key integrations are returning data (green on the Integrations settings page).

Common issues

• Ownership gaps - Use the Cortex ownership report to find entities with no owner set. Unowned entities will silently fail Scorecard rules that check ownership.

• Integration not returning data - Check that your API token has the required scopes. Most integrations list required permissions in their setup docs.

Step 2: Creating an AI readiness Scorecard

Scorecards automate checking whether your services meet your AI readiness standards. Cortex provides an AI readiness template structured across three levels—Bronze, Silver, and Gold—each representing increasing maturity.

What to do

Create a Scorecard using the AI readiness template. Review and customize the rules for your environment (see below).

Default template rules

Level	Rule	What it checks
Bronze	Service ownership defined	An owner (team or individual) is set
Bronze	Runbook linked	An external runbook doc exists
Bronze	On-call rotation configured	An on-call schedule is linked
Silver	SLOs defined	Service level objectives are configured
Silver	CI/CD pipeline active	Deployments are running through a pipeline
Silver	Dependency inventory current	Service dependencies are catalogued
Gold	AI model security scanning	Vulnerability scanning is enabled for AI models
Gold	Test coverage minimum met	Code coverage meets your defined threshold
Gold	DORA metrics within targets	Deployment frequency and MTTR meet benchmarks

These are the template defaults. You can reorder, delete, edit, or add rules using Cortex Query Language (CQL) to match your organization's actual standards.

You'll know this step is done when...

Your Scorecard is live and entities are showing Bronze/Silver/Gold scores (even if most are failing; that's expected at first).

Common issues

• All entities show 0% - Check that the integrations required by your rules are connected and returning data. A rule checking for SLOs can't pass if your monitoring integration isn't configured.

• Rules feel too strict or too lenient - Start with Bronze rules only and expand once teams are consistently passing. Adjust point weights to signal which rules matter most.

Step 3: Automating processes with Workflows

A Workflow is an automated, multi-step process that runs entirely within Cortex. It lets teams define tasks, trigger actions, collect input, and route approvals in one place, turning complex, multi-tool operations into a single repeatable experience. For AI readiness, this means you can enforce standards automatically rather than relying on manual checklists or Slack threads.

Identifying good candidates for Workflow automation

Before building, look at where friction exists in your AI readiness program. Scorecards and Eng Intelligence are useful here: if a team consistently fails the same rules, or cycle time data shows work getting stuck at a particular handoff, that's a strong signal that the underlying process should be a Workflow.

What to do

Create a Workflow, configure its settings, and add blocks. You can create a Workflow based on a template or create one from scratch.

Two high-value Workflows to set up

Scaffolding new AI services with baseline standards

When engineers create a new AI service, use a Scaffolder template to pre-populate it with required files like incident runbook, SLO configuration, and ownership metadata. This makes compliance the default rather than a retrofit.

Requiring manual approval before production

Add a manual approval block to your deployment Workflow to require sign-off from a senior engineer or security reviewer before a service is marked production-ready. This creates an audit trail alongside your Scorecard data.

Workflows can be managed in the Cortex UI or via GitOps. If you're starting fresh, build in the UI first as it's faster to test and iterate before committing to code. See Workflows as code

You'll know this step is done when...

At least one Workflow is actively running—either scaffolding new services or gating production sign-off.

Common issues

• The HTTP Request block is pointing to the wrong Scorecard - The deployment-blocking Workflow template references a Production readiness Scorecard by default. Update the HTTP Request block to point to your AI readiness Scorecard before activating it, or it will gate deployments against the wrong criteria.

• Missing API key - The HTTP Request block requires cortex_api_key configured as a workspace secret. If it isn't set up before the Workflow runs, every execution will fail at that step with an auth error.

• Activating the deployment gate too early - If only a fraction of your services are passing at Gold level, a hard block will immediately stop legitimate deployments and create pressure to roll the Workflow back. Start with a Slack notification on failure instead, then switch to blocking once pass rates are high enough.

• Approval blocks with no assigned approver - If the approver field is left unassigned or pointed at a team that doesn't monitor it, requests stall silently.

Step 4: Configuring Cortex MCP

Cortex MCP gives engineers conversational access to Cortex data directly from their AI coding tool. Instead of navigating dashboards, they can ask natural language questions and get real-time answers drawn from your Cortex workspace.

Example questions engineers can ask:

• What are the quick wins for my AI Readiness Scorecard? → Returns a prioritized list of failing rules for their services

• Who owns the model-inference-service? → Returns owner, on-call contact, and links to runbook

• What's the current Gold-level Scorecard score for payments-api? → Returns real-time score with passing/failing rules

What to do

Configure the Cortex MCP. It can be hosted locally or remotely. Refer to Using the Cortex MCP for information on starting a new chat, crafting effective prompts, and using Eng Intelligence metrics in chat.

You'll know this step is done when...

An engineer on your team has successfully queried Cortex MCP from their coding environment and received a meaningful answer about a service they own.

Common issues

• MCP not connected to the right workspace - If the MCP is pointed at the wrong Cortex workspace or authenticated with credentials that have limited permissions, queries will return incomplete or no results. Verify the connection returns accurate data for a service you know well before rolling it out.

• Questions that are too vague - MCP works best with specific, scoped questions. How are my AI services doing? returns less useful results than What are quick wins for my AI Governance Scorecard? Train teams on the kinds of questions that get actionable answers.

• Teams not aware it exists - MCP adoption tends to be low when it's announced once and forgotten. Engineers default to existing habits (Slack, wikis, asking a colleague) unless MCP is introduced in the context of a workflow they already use, like a Scorecard remediation or an incident investigation.

Step 5: Reviewing Eng Intelligence metrics

Use the DORA dashboard, Velocity dashboard, and Data Explorer to establish a baseline before your AI readiness program is in full effect, then track how metrics shift as teams meet Scorecard standards.

Key metrics to baseline:

• Deployment frequency

• Mean time to recovery (MTTR)

• Change failure rate

• Cycle time (PR open → merge)

What to do

Analyze metrics in the Data Explorer, the Cortex dashboards, and in reports.

You'll know this step is done when...

You've captured a baseline snapshot of your key DORA metrics and noted the date your AI readiness Scorecard went live. This gives you a before/after comparison point.

Common issues

• No baseline before rollout - Once services start passing Scorecard rules and behaviors change, the before state is gone. Pull baseline metrics before your AI Readiness Scorecard goes live.

• Missing security and code quality metrics - If the relevant integrations (Snyk, SonarQube, Checkmarx, etc.) weren't connected in Step 1, the metrics that would show improvement in security scanning and code quality simply won't appear in Eng Intelligence. Verify those integrations are pulling data before establishing your baseline.

• Watching deployment frequency without change failure rate - Deployment frequency tends to increase as teams gain confidence, but if change failure rate is climbing alongside it, your readiness standards aren't holding. Track both together.

Next steps

Once your workspace is configured, see AI readiness in action for how to maintain standards and drive continuous improvement.