Generative AI helps most in ERP as an interface and an assistant layer: summarising records, drafting documents, answering questions over grounded data and proposing actions a human approves. It should not be given unattended authority over transactions, and every output must be grounded in retrieved data rather than model memory.
Where generative AI actually helps
The strongest near-term value is in reducing the cost of reading and writing text around structured processes. Summarising a supplier's email thread against a purchase order, drafting an order confirmation, explaining a variance in plain language, or answering an internal question about a policy are all tasks where a language model adds real leverage.
Conversational access to data is genuinely useful when it is grounded. An assistant that can answer which purchase orders are overdue by more than ten days, with a link to each record, saves time even though the underlying query is trivial.
What these have in common is that they are assistive. The model proposes, explains or drafts, and a person or a deterministic process decides. Value comes from reducing effort, not from removing the decision.
- Summarisation of documents, threads and record histories.
- Drafting routine correspondence and portal content.
- Natural language querying over grounded structured data.
- Explaining anomalies and variances in plain language.
Grounding over retrieval and tools
A model asked about your inventory levels will produce a plausible number. Grounding means the answer is constructed from retrieved records supplied in the request, with the retrieval step executed under the requesting user's permissions so the model never sees data that user cannot access.
For questions over structured data, tool calling is more reliable than retrieval alone: the model selects a defined query or endpoint with validated parameters, the system executes it, and the model summarises the result. This keeps the model away from schema internals and makes the operation auditable.
Never let the model generate SQL or arbitrary API calls directly. Define a bounded set of tools with typed parameters, permissions and rate limits, and treat the model's selection as an untrusted input that must be validated.
| Pattern | Value | Risk control |
|---|---|---|
| Summarise a record or thread | High, low effort | Read-only, user permissions applied |
| Draft correspondence | High | Human review before sending |
| Query data in natural language | High | Bounded tools, validated parameters |
| Anomaly explanation | Medium | Grounded in retrieved figures, human decides |
| Autonomous transaction posting | Low | Not recommended without deterministic rules |
| Forecasting | Medium | Existing statistical models usually sufficient |
Governance, privacy and data residency
Before sending enterprise data to any model provider, establish the contractual and technical position: whether inputs are retained or used for training, where processing occurs, whether the provider is a data processor under your agreements, and what happens on deletion requests.
Classify the data that may be sent. Commercial data such as order volumes may be acceptable, whereas personal data, pricing agreements and intellectual property may not. Enforce this with a classification check in the request path rather than relying on user judgement.
Where confidentiality prevents external processing, a locally hosted open-weights model is the alternative, accepting lower capability in exchange for full control. The decision should follow from the data classification, not from model performance benchmarks.
- Confirm retention, training use and processing location contractually.
- Enforce data classification before anything leaves your boundary.
- Apply the requesting user's permissions to every retrieval.
- Log prompts and responses for audit, with appropriate access control.
Evaluation and change control
Model and prompt changes alter behaviour silently, so they need the same change control as code. Build a test set from real enterprise questions with known correct answers, and run it on every change, scoring for correctness, groundedness and refusal behaviour.
Measure groundedness explicitly. A response that is fluent but not supported by the retrieved data is the most dangerous failure mode because it is hardest for a user to detect. Where possible, require citations to the records used.
Provide feedback mechanisms in the interface and review them. User-reported errors are the most efficient way to find the cases your test set missed, and they should feed directly into the evaluation set.
- Maintain a domain test set with known correct answers.
- Score correctness, groundedness and appropriate refusal.
- Require citations to source records where feasible.
- Feed user-reported errors back into the evaluation set.
Frequently asked questions
Should generative AI be allowed to post transactions in ERP?
No, not autonomously. Use it to propose, draft and explain, with a human or a deterministic rule making the transactional decision. Models are not reliable enough to be given transactional authority, and the audit implications are difficult to satisfy.
How do you stop an AI assistant inventing data?
Ground every answer in retrieved records supplied in the request, apply the requesting user's permissions to that retrieval, prefer tool calling over free generation for structured questions, and require citations so unsupported claims are visible.
When should an LLM be hosted locally instead of using a provider?
When data classification or contractual obligations prevent sending the content externally. Local open-weights models trade capability for control, and the decision should follow from the data classification rather than from benchmark performance.
References
Engineer your next strategic platform
Estimate your project investment and delivery timeline instantly, or schedule an architectural discovery call with our engineering leadership.