The integration work nobody scopes
The model is rarely the hard part. Getting clean, current, permissioned data out of the systems of record is the hard part, and it is where most projects quietly stop.
Ask a team what is left before their AI feature ships and you will usually hear about the model: the prompt needs tuning, the accuracy is not quite there, they are waiting on a bigger context window.
Then you look at the burndown and the actual remaining work is a connector to a CRM whose API paginates inconsistently, a mapping between two customer identifiers that disagree for about 3% of records, and a permissions question nobody wants to own.
This is the normal case, not the unlucky one.
Why it goes missing from the estimate
Integration work is invisible in a demo. The demo runs on a CSV somebody exported by hand, and that export took four minutes, so the integration is mentally filed as a four-minute problem. The demo is also where the funding decision gets made.
It is also genuinely hard to estimate before you have credentials. You cannot know that the API returns deleted records as active with a deleted_at timestamp until you have looked, and you cannot look until procurement has finished, which is week five.
The four costs
Authentication and rate limits
Every system of record has its own answer, and several of them are bad. OAuth flows that require a user to be present. Tokens that expire in an hour with no refresh. A rate limit expressed per-endpoint rather than per-account, so a backfill that is well within quota globally still trips one path. Vendors who will raise your limit but need a ticket and two weeks.
None of this is intellectually difficult. All of it is calendar time, and it is calendar time that lands on the critical path because nothing else can be tested until it is done.
Schema drift
The schema you integrate against is a snapshot of what an organisation believed when they configured the system, and organisations keep believing new things. Custom fields get added. A picklist gains a value. A field that was free text becomes a reference. Someone renames a stage in the sales pipeline and every downstream rule that matched on the string stops matching.
The failure mode is quiet: nothing errors, the value is simply absent or wrong, and the system carries on producing confident output from incomplete input. Contract tests against the upstream schema are the cheapest insurance in this entire category of work, and they are almost never in the original scope.
Incremental sync, and deletes
A full reload is easy and does not scale. An incremental sync needs a reliable watermark, and a surprising number of systems do not give you one — updated_at is not set by every write path, or it is set by the ETL that last touched the row, or it is in the local timezone of whoever configured the instance.
Deletes are worse. Many APIs simply stop returning a record, which is indistinguishable from it not matching your filter. If you do not solve this, deleted records live forever in your copy, and the first time anyone notices is when a customer asks why a person who left the company is still receiving output.
Permissions
This is the one that stalls projects outright, because it is not really a technical question.
The source system has a permission model. Your application has a different one. Somebody has to decide what happens when they disagree — and that decision has consequences the engineering team is not authorised to accept. It usually needs the person who owns the data, the person who owns the risk, and a written answer. Getting all three in a room is a scheduling problem, not a coding problem, and it takes as long as it takes.
Start it in week one. It is the single most common reason a technically finished system sits unreleased.
How to de-risk it
Build the connector first. Before the prompt, before the interface, before the evaluation harness. The connector is the part most likely to contain an unpleasant surprise, and a surprise in week two is a scoping conversation while a surprise in month four is a failed project.
Run against real data immediately. Not a sample, not a fixture — a live pull from the actual instance, including the records that look wrong. The nulls, the duplicates, the test records from 2019 that somebody never cleaned up. Those are the inputs your system will have.
Write the contract test on day one. A test that asserts the shape and cardinality of what the upstream returns, running on a schedule against the real system, so a change is a red build rather than a silent behaviour shift.
Put a number on the disagreement. When two systems hold the same entity, measure how often they disagree before designing around it. "The identifiers match for 97% of records" is a scoping input. "They should match" is a hope.
What this means for a plan
If a proposal puts integration in the last phase, it is describing the order the work is comfortable in, not the order it is risky in. The right order is the reverse: do the part that can kill the project while killing it is still cheap.