# Custom software in healthcare with AI: what you can build and where to stop

> Outpatient clinics, diagnostic centres and care homes can use AI to build tools that take hours of repetitive work off their staff. Where it pays off, which rules apply and where to stop.

- Published: 2026-10-21
- Who it is for: Management and IT leads at private healthcare providers
- URL: https://cloudfactory.it/en/articles/custom-software-healthcare-ai/

## You can build a lot, but around clinical data

In an outpatient clinic, a diagnostic centre or a care home, the working day is split between the management system, the phone, diaries, Excel sheets and messages. With AI it is now possible to build, in a few weeks, custom tools that remove hours of repetitive work. In [another article](https://cloudfactory.it/en/articles/custom-software-ai-smes/) we explained when it makes sense to build and when to buy; in healthcare the question changes.

It is not enough to ask whether the tool works. You have to ask whether the organisation can account for how it handles patient data, what happens if the tool stops, and who decides when the AI suggests something. Our position is simple: there is plenty of room for custom software in operations and administration, and very little inside clinical decision-making.

## Why healthcare is different: four rules to know

A tool built in-house at a healthcare provider falls under at least four sets of rules. None of them forbids building; all of them require you to be able to show how you did it.

| Regulation | What it requires | What it means for a custom tool |
| --- | --- | --- |
| GDPR | Health data is a special category of personal data | Access must be restricted, logged and encrypted; assess where the data resides and who can see it, including the AI service provider |
| NIS2 (Legislative Decree 138/2024) | Healthcare providers are among the obliged entities, either essential or important; they need risk management, business continuity, access control, secure development and supply chain security ([Agenda Digitale](https://www.agendadigitale.eu/sanita/nis2-in-sanita-perche-server-obsoleti-e-otp-bloccati-sono-un-rischio/)) | Even an internal tool is within scope: it must be documented, kept up to date and backed up. Fines reach up to 10 million euros or 2% of turnover for essential entities |
| Medical Device Regulation (MDR) | Software is a medical device depending on its intended purpose: if it processes an individual patient's data to support diagnosis or treatment, it is normally at least class IIa ([OpenRegulatory, MDCG 2019-11](https://openregulatory.com/mdcg/mdcg-2019-11)) | Scheduling, administration and billing usually are not. A tool that suggests diagnoses, doses or clinical priorities needs CE marking: it is not something you build "in-house" |
| AI Act and Law 132/2025 | From 2 August 2026 chatbots and voice assistants must disclose that they are AI; obligations for high-risk systems are pushed back to December 2027 and August 2028 ([Agenda Digitale](https://www.agendadigitale.eu/sicurezza/digital-omnibus-ai-cosa-cambia-davvero-per-le-imprese/)). In Italy, AI in healthcare is a support tool and the decision stays with the doctor ([ICT Security Magazine](https://www.ictsecuritymagazine.com/notizie/legge-italiana-sullintelligenza-artificiale/)) | A phone line or chat that talks to patients must introduce itself as AI. No tool should decide in place of the clinician |

The common thread is being able to demonstrate. A tool put together in an afternoon, with no documentation and no one accountable for it, is hard to defend during an inspection or after an incident.

## Cyber risk: healthcare is the target where damage hurts most

According to the Clusit 2026 Report, in 2025 healthcare suffered 11% of serious attacks worldwide, up 19% on 2024. It is also the sector with the highest share of incidents rated critical or extreme in severity: 64% ([01net](https://www.01net.it/rapporto-clusit-2026-cybersecurity-ai-ransomware/)).

In Italy, healthcare accounted for 1.8% of the serious incidents recorded in 2025, down on the previous year. The country as a whole, however, recorded 507 incidents, 42% more than in 2024, and phishing and social engineering, fuelled by generative AI, grew by 66% ([01net](https://www.01net.it/rapporto-clusit-2026-cybersecurity-ai-ransomware/)).

For anyone building tools with AI, the point is practical. Independent studies show that a significant share of generated code contains vulnerabilities, especially around authentication and permissions ([IOActive](https://www.ioactive.com/the-security-gap-in-ai-generated-code/)). In an ordinary company that is a risk; in an organisation that holds test reports and medical histories it is a notifiable incident.

## Where custom software pays off and where it does not

The closer a tool gets to clinical decision-making, the more it makes sense to buy it certified. The more it is about operations and administration, the more custom software pays off.

| Area | Examples | Recommended approach |
| --- | --- | --- |
| Internal operations | Shifts, leave, availability of rooms and equipment, operational checklists | Build: high value, limited risk |
| Administration and management | Payment reconciliation, reminders for overdue payments, checking services against provider agreements, management dashboards | Build on top of the management system, reading data without changing it where possible |
| Front desk and communication | Appointment reminders, a phone line or chat answering questions on opening hours, exam preparation, cancellations | Build with strict rules: it discloses that it is AI, gives no clinical advice, always hands over to a human operator |
| Links between systems | Data exchange between the management system, the laboratory and the online booking calendar | Build with care: this is where data gets lost or duplicated; logs and tests are needed |
| Clinical records, reporting, flows to the national health service (SSN) and electronic health record (FSE) | Handling clinical data and institutional data flows | Buy certified software integrated with the regional systems |
| Diagnosis and treatment support | Triage, image reading, dosage calculation | Do not build in-house: it is a medical device and requires CE-marked products |

On the phone line, Klarna's lesson applies. After handing much of its customer service over to AI, the company started hiring human agents again, admitting that quality had dropped ([Entrepreneur](https://www.entrepreneur.com/business-news/klarna-ceo-reverses-course-by-hiring-more-humans-not-ai/491396)). With patients, a way out to a real person is not optional.

## Checklist before going live

Before a custom tool touches real patient data, it should pass these ten checks.

1. **Written intended purpose.** What it does and what it does not do, with a check that it does not qualify as a medical device.
2. **Privacy assessment.** Update the record of processing activities and, where required, carry out a data protection impact assessment (DPIA).
3. **Where the data lives.** Servers and AI services within the European Union, contracts with suppliers and their appointment as data processors.
4. **Minimal data.** The tool sees only what it needs, pseudonymised wherever possible.
5. **Access.** Personal accounts with multi-factor authentication, no shared accounts, an access log.
6. **Code review.** A security review and testing on dummy data before any real data.
7. **Continuity.** Backups, a recovery plan and a manual procedure if the tool stops.
8. **Owner and documentation.** One person, internal or external, who is accountable for it and keeps the documentation up to date.
9. **Transparency.** If the tool talks to patients, it states that it is AI and always offers a human operator.
10. **Training.** Staff know what the tool does, what it does not do and how to report a problem.

## How we work

At Cloudfactory we have been looking after IT for private healthcare providers for years: multi-site groups, diagnostic centres, care homes for the elderly. We know that in this sector a tool that is useful but not defensible is a liability, not a saving.

That is why we always start from the intended purpose and the data map, before writing any code. We use AI to build internal management tools, dashboards and voice assistants for front-desk phone calls, always alongside certified clinical systems and never in their place. Then we take care of what the prototype does not show: security, backups, documentation, updates.

## Frequently asked questions

### Can a healthcare provider develop custom software with AI?

Yes, for operations, administration and communication with patients. It must be done with documentation, security checks and someone accountable, because it falls under GDPR obligations and often NIS2.

### When does software become a medical device?

When its intended purpose is to support diagnosis, treatment or monitoring of an individual patient. Scheduling, administration and billing normally do not qualify.

### Does a voice assistant for bookings have to say it is AI?

Yes. From 2 August 2026 the AI Act requires that anyone talking to an AI system is informed of it, unless this is obvious from the context.

### Can AI make clinical decisions?

No. Italian law on AI treats it as a support tool: the decision always stays with the doctor.

### Where is the best place to start?

With an operational or administrative process that currently costs hours of manual work and does not touch clinical decision-making.

## Sources

- [01net, Clusit 2026 Report](https://www.01net.it/rapporto-clusit-2026-cybersecurity-ai-ransomware/)
- [Agenda Digitale, NIS2 in healthcare](https://www.agendadigitale.eu/sanita/nis2-in-sanita-perche-server-obsoleti-e-otp-bloccati-sono-un-rischio/)
- [Agenda Digitale, Digital Omnibus AI: what changes for businesses](https://www.agendadigitale.eu/sicurezza/digital-omnibus-ai-cosa-cambia-davvero-per-le-imprese/)
- [ICT Security Magazine, The Italian law on artificial intelligence](https://www.ictsecuritymagazine.com/notizie/legge-italiana-sullintelligenza-artificiale/)
- [OpenRegulatory, MDCG 2019-11 explained](https://openregulatory.com/mdcg/mdcg-2019-11)
- [IOActive, The Security Gap in AI-Generated Code](https://www.ioactive.com/the-security-gap-in-ai-generated-code/)
- [Entrepreneur, Klarna goes back to hiring human agents](https://www.entrepreneur.com/business-news/klarna-ceo-reverses-course-by-hiring-more-humans-not-ai/491396)

---

Cloudfactory S.r.l., Rome. Contact: hello@cloudfactory.it