How an AI Application Change Goes Live

My last post covered what the SuiteResources AI Application Platform is — the roadmap side, the SDK, all the things every app gets for free. This one covers something a lot less glamorous: what actually happens between somebody deciding on a change and that change running on a production server.

There are six steps. Two of them involve a human on purpose. The rest are automated, and the automation is deliberately not allowed anywhere near the production site.

Here's the whole thing.

The Cast

Five things are involved, and nothing below makes sense until these do.

The AI Application Platform. It holds each application’s definition, features, feature amendments, and decides who may see what, sets up new apps, and hands each app its build instructions. Call it the control tower.

The SDK. The shared parts bin. Login, usage reporting, health checks — written once so that five apps don't each build their own slightly-wrong version. There is a long list of the common features the SDK provides each Platform app.

The apps. The CMS behind this blog, the SuiteScript Migration Tool, the deadline trackers. Each one is its own codebase and its own website.

GitHub. Stores the code, and runs the automated checks every single time the code changes.

Forge. Runs the actual web servers. When told to deploy, it pulls the newest code onto a server and restarts things.

The Six Steps

1. Decide, and write it down

Every feature or feature amendment for every app starts as a record in the AI Application Platform — what's changing and why — before anyone opens an editor.

Why bother? Because six months later the code will tell you exactly what it does and absolutely nothing about why it was done that way. The record is the only place the reasoning survives.

2. Write the code, on the dev server using the dev branch

Work lands on a branch called dev. The main branch is what the public site runs, and nothing gets there by accident.

Two branches means there's always somewhere to try something that isn't the place customers are looking at.

3. GitHub checks it, automatically

Pushing code kicks off a run: install dependencies, run the test suite against a real database, lint the front end, build it, then load the built site in a headless browser to confirm it actually starts.

That last part is the one people skip. Code can compile perfectly and still fall over the moment a page opens. "It built" and "it works" are two different claims, and only one of them is worth anything.

4. Forge deploys it to the test server

If every check passed, GitHub asks Forge to deploy. Forge pulls the code, installs dependencies, runs database migrations, rebuilds the front end, reloads the web server.

Note where it deploys: the test server. Only the test server. This step is automatic, and automatic things should never touch the site customers use.

5. Prove it actually landed

GitHub doesn't stop at "deploy requested." It goes back and asks the website itself which version it's running, over and over, and fails the entire run if the new version never shows up.

More on why that's a separate step in a minute. It's the change we're happiest about.

6. Promote to the live site

Moving a change to the public site is a deliberate human action, done from the AI Application Platform. Never a side effect of committing code.

Everything before this point is reversible in a few minutes. This step is the one customers see. It gets a person.

Every App Has Two of Everything

This is the part that trips people up more than anything else, because the two look identical from the outside.

• The test twin — cms-dev.suiteresources.com. Runs the dev branch. Deployed automatically, many times a day. Safe to break. This is where a change gets confirmed.
• The live site — cms.suiteresources.com. Runs the main branch. Deployed only when a person promotes it. What customers see.

Same pattern for every app. Separate servers, separate databases, so data doesn't wander between them.

How to Check a Site Yourself

Two addresses answer the two questions that actually come up. Both are public and safe to hit from anywhere.

"Which version is this site running?"

curl https://cms-dev.suiteresources.com/health/version

Get back JSON with a commit and a branch, and that's the exact code the site is serving right now. Get back a page of HTML instead, and the address doesn't exist on that site — usually an app running a version of the SDK that's too old.

"Are its background jobs running?"

curl -i https://cms-dev.suiteresources.com/health/background

A 200 means scheduled tasks and the job queue are both alive. A 503 means something isn't, and the reply names which: never_run (never set up on the server), stale (was working, stopped), or inline (jobs running inside web requests instead of in a background worker).

A 503 there is the system doing its job. It's meant to be loud, so a monitor notices before a person does.

Three Things That Surprised Us

Each of these cost real time. They're the reason steps 3 and 5 look the way they do.

A page loading doesn't mean the address exists. These sites answer every unknown address with the normal website and a success code. So typing a URL, getting a page back, and concluding the feature is live? Tells you nothing. Look at what came back, not whether it worked. JSON means the feature exists. HTML means it doesn't.

A green tick doesn't mean it deployed. GitHub's job is to ask Forge to deploy. For a long stretch, that request succeeding got recorded as success — even when the deploy then failed on the server. The pipeline said yes every single time. A failed deploy and a perfect one looked identical from the outside. That's what step 5 exists to fix: the run now waits, asks the site which version it's serving, and goes red if the new one never appears.

A server's setup is frozen the day it's created. The instructions Forge follows when deploying a site get written once, when that site is first created, and never updated afterward. Improving them in the codebase changes nothing on any server that already exists. So if a fix "should" be live and isn't, the first question is whether it depended on those deploy instructions changing.

Why Any of This Matters to You

Short version: nothing reaches a customer-facing site because a machine decided it was ready.

The automation runs fast and runs constantly, but it runs against a test twin. A person moves it the rest of the way, and by then the change has been built, tested, loaded in a browser, deployed, and independently confirmed to be running. Six steps that mostly exist because something went wrong.