Docs

Docs

AI Form Filler

Use FormAIController to let users fill any Vaadin form from natural-language input or attached files.

FormAIController populates the fields of a FormLayout (Vaadin’s responsive multi-column form container) or any other layout, using values an LLM extracts from a user prompt or attached files. The controller traverses the layout, discovers every field, and allows the LLM to read the current values, query the available values for selection components like Combo Box or Radio Button Group, and write new values back. Each write is validated through the Binder, or through the component’s built-in validators if Binder is not used. Rejected values are reported back so the model can correct them on the next turn.

The controller works with any combination of standard Vaadin field components, such as TextField, ComboBox, DatePicker, MultiSelectComboBox, and CheckboxGroup. No extra wiring is needed beyond constructing the controller around the layout and attaching it to the orchestrator.

Basic Usage

Build the form, construct a FormAIController for it, and attach the controller to an orchestrator:

Source code
Java
TextField name = new TextField("Name");
EmailField email = new EmailField("Email");
ComboBox<String> country = new ComboBox<>("Country");
country.setItems("Finland", "Germany", "United States");
DatePicker hiredOn = new DatePicker("Hired on");

FormLayout form = new FormLayout(name, email, country, hiredOn);
MessageInput messageInput = new MessageInput();

FormAIController controller = new FormAIController(form);

AIOrchestrator.builder(provider, systemPrompt)
        .withInput(messageInput)
        .withController(controller)
        .build();

add(messageInput, form);

Example prompts:

  • "Fill in John Doe, john@acme.com, started in Germany on March 1, 2026."

  • "Maria from Helsinki, hired last Monday." (relative dates are interpreted by the LLM)

  • "Use the information in the attached resume to fill the form." (when the orchestrator has a file receiver configured)

Tip
Built-In Workflow Instructions
The controller already informs the LLM of the workflow it needs. You can focus your own system prompt on application-specific behavior, such as tone, naming conventions, or which fields the user may leave blank.

Field Discovery

The controller walks the container’s component tree on every LLM turn, so fields added or removed between turns are picked up automatically. The container can be any component that implements HasComponents. Any component that implements HasValue is treated as a field, and any nested HasComponents is walked recursively.

PasswordField is always hidden from the LLM. To hide other fields, for example internal IDs or anything sensitive that the user must fill in manually, call ignoreField():

Source code
Java
controller.ignoreField(internalIdField);

Ignored fields are hidden from the LLM and stay editable while a fill is in progress.

Hidden, Disabled, and Read-Only Fields

The controller checks each field’s state on every turn:

  • A field hidden via setVisible(false), or one that sits inside a hidden container, is dropped from the LLM surface entirely. The model cannot read its value and cannot write to it. It reappears the moment a value-change listener or other application code makes it visible again.

  • A field the application has disabled (setEnabled(false)) or set read-only (setReadOnly(true)) stays in the form state as read-only context. The model sees its current value — useful as context for writes to other fields — but any write is rejected, and the model receives the rejection reason so it can adjust on the next turn.

A common case is a conditional field driven by a ValueChangeListener — a "Cost center" enabled only when "Trip type" is set to "Business", or a renewal date enabled by a "Renews automatically" checkbox. The built-in workflow instructions tell the model that a disabled or read-only field is usually waiting on a controlling field, so it sets the controlling field first and writes the dependent one in the same turn.

Field Descriptions

The LLM sees each field’s label, helper text, and component type. When those don’t fully capture the field’s meaning, for example a numeric field that takes a percentage rather than an absolute amount, or a date that means "renewal date" rather than "purchase date", add an explicit description with describeField():

Source code
Java
controller.describeField(discount, "Discount as a percentage between 0 and 100.")
        .describeField(renewalDate, "When the subscription renews, not when it started.");

Later calls for the same field overwrite earlier ones.

Binder Integration

When the form is backed by a Binder, pass the binder to the controller as well:

Source code
Java
Binder<Employee> binder = new Binder<>(Employee.class);
binder.bindInstanceFields(this);

FormAIController controller = new FormAIController(form, binder);

For every named binding (bind("propertyName"), bindInstanceFields(this), or @PropertyId), the bean property name is used as a default field description, so when the user mentions a field by its bean-side name, the LLM can match the request to the right field. An explicit describeField() call always overrides the default. Lambda-bound bindings have no property name and contribute no default field description.

Options for Selection Fields

Selection fields like ComboBox, Select, MultiSelectComboBox, and CheckboxGroup take a value from a known set. fieldValueOptions() registers that set with the controller.

The registration carries the field’s domain items. The controller renders each item into an LLM-facing label through the field’s own setItemLabelGenerator(). When the LLM picks a label, the controller applies the same labeler to the registered items and writes the matching one back.

Use a fixed list when the values are small and known up front, or a query callback when they come from a service or database.

Fixed List

For a small set of known String values, pass them directly:

Source code
Java
controller.fieldValueOptions(
        ValueOptions.forField(industry)
                .options(List.of("Software", "Manufacturing", "Healthcare")));

For a field over a domain type, set the field’s item-label generator first; the controller picks it up automatically:

Source code
Java
List<Project> projects = projectService.findAll();

ComboBox<Project> projectField = new ComboBox<>("Project");
projectField.setItemLabelGenerator(Project::name);
projectField.setItems(projects);

controller.fieldValueOptions(
        ValueOptions.forField(projectField).options(projects));

In the above example, the LLM sees project names. When it picks one, the controller writes the matching Project instance to the field.

Service-Backed Lookup

When the values come from a service or repository the application already uses, supply a callback that returns the matching items for each search the LLM runs:

Source code
Java
ComboBox<Project> projectField = new ComboBox<>("Project");
projectField.setItemLabelGenerator(Project::name);

controller.fieldValueOptions(
        ValueOptions.forField(projectField)
                .options((filter, limit) ->
                        projectService.search(filter, limit)));

The callback returns domain items; the controller derives the labels through the field’s setItemLabelGenerator() before showing them to the LLM. If the LLM picks a label and the matching item is no longer available (for example, because the application’s data has changed since the search), the write is rejected and the model can try again on the next turn.

projectService here is a stand-in for whatever your application uses to look up projects: a Spring repository, a REST client, an in-memory list. The controller only needs the callback to return the matching items for the given filter and limit.

Note
Eager Items as a Fallback
Single- and multi-select fields configured with setItems(…​) already share their items with the LLM, so the simple fixed-options case often needs no fieldValueOptions() call. Use fieldValueOptions() when items come from a lazy or remote source rather than an in-memory list, or when you want the LLM to fetch options through a filter callback instead of receiving the full set up front.

Multi-Select Fields

MultiSelectComboBox, CheckboxGroup, and any other field that implements MultiSelect are supported. Use the component’s concrete multi-select type for the field reference so the forField(MultiSelect) overload is selected:

Source code
Java
MultiSelectComboBox<Project> projectsField =
        new MultiSelectComboBox<>("Projects");
projectsField.setItemLabelGenerator(Project::name);
projectsField.setItems(projects);

controller.fieldValueOptions(
        ValueOptions.forField(projectsField).options(projects));

The selected items are written to the field as a set, in the order returned by the LLM.

Note
Multi-Value Fields Must Implement MultiSelect
A field whose value type is a Collection must implement MultiSelect. The controller rejects two cases at registration time: a MultiSelect field passed through the single-value forField(HasValue) overload, and a Collection-valued field that doesn’t implement MultiSelect.

Custom Labels for the LLM

By default the LLM sees the same labels the field’s setItemLabelGenerator() produces for the UI. When the LLM should see a different label, for example a code rather than a display name, set an explicit generator on the registration:

Source code
Java
controller.fieldValueOptions(
        ValueOptions.forField(projectField)
                .options(projects)
                .itemLabelGenerator(Project::code));

The explicit generator overrides both the labels surfaced to the LLM and the lookup that resolves a chosen label back to an item. The field’s UI continues to render through its own generator.

Items with the same label, such as two projects sharing a display name, resolve to the first one in registration order. The controller logs a warning at registration time when this happens with a fixed list; supply a unique generator (such as Project::code above) to remove the ambiguity.

Validation

When the model writes back a set of values, the controller commits all of them first and then runs validation once against the resulting form. Each field is checked according to how it is wired:

  • A field that is bound through a Binder is validated through its binding, so the converter and every registered validator run as one unit.

  • An unbound field with a default validator (for example the email-format check on EmailField, or the min and max constraints on NumberField and DatePicker) is validated through that validator.

  • Cross-field rules registered with binder.withValidator((bean, ctx) → …​) — see Binder-Level Validators — are evaluated against the post-write form, but only when the binder has a bean set (setBean(bean)) and every per-field check passes first. A failing rule is reported as a form-level rejection rather than against any single field, so the model can adjust the offending values and try again in the same turn.

If validation fails for a written field, the value stays in the field, the field’s UI error indicator turns on, and the failure is reported back to the model with the field id and the validator’s message. The model can supply a corrected value in the same turn, so users typically see only the final, valid state. Fields the current turn did not write are not flagged invalid as a side effect — a required field the user has not reached yet stays clean.

What the Model Sees

The controller sends only a defined subset of the form to the LLM. Knowing where that boundary lies matters when the form holds sensitive or domain-specific data.

The model sees:

  • Each visible field’s label, helper text, component type, and any describeField() text or Binder property-name default.

  • The current value of every visible, non-ignored field, so it can decide which entries to overwrite. Disabled and application-set read-only fields are included for context, with a flag telling the model not to write to them.

  • The available labels for a selection field, derived from a combo box or select’s eager items, or from the items a fieldValueOptions() query callback returns for the filter the model supplies.

The model does not see:

  • Any field excluded with ignoreField(). Its value, label, and existence are all hidden.

  • Any field the application has hidden via setVisible(false), or that sits inside a hidden container.

  • The contents of PasswordField, which is always excluded.

  • Internal data, services, or beans. The model has access only to what the field components themselves show.

Important
Visible Field Values Are Sent to the Model
Every visible field’s current value is forwarded to the LLM provider on every turn. Hide fields that carry secrets, identifiers, or personally identifiable information with ignoreField(), or keep them out of the layout passed to the controller.

Field Locking During a Fill Turn

While a fill is in progress, every field the AI can write — visible, enabled, and not read-only — shows an "AI is working" shimmer and is guarded against user edits, so the user cannot type into a field the AI is about to overwrite. The guard is applied on the client only: the field’s server-side read-only state is never changed, so it doesn’t affect what the LLM sees or writes, and a field’s application-set read-only state is left untouched. Fields the application had already disabled or set read-only stay as they were, and fields excluded with ignoreField() stay editable throughout. The working state clears automatically when the turn ends, whether it succeeded or failed. A field switched to read-only on the server mid-turn, for example by a ValueChangeListener reacting to one of the model’s writes, stays read-only when the guard is released.

Marking AI Changes

When an AI fill changes several fields at once, users benefit from a visual cue that flags which fields the AI wrote. The controller handles this automatically: when a turn ends, every field whose value changed is marked with an "AI" badge. Selecting the badge opens a popover that explains the value was filled by AI and offers a revert control, which restores the field’s value from before the AI’s first change to it.

The marker needs no application code:

  • A field marked in an earlier turn keeps its mark — and with it the original revert value — when a later turn changes it again, so reverting always restores the value the field held before the AI touched it.

  • The marker clears itself as soon as the user edits or reverts the field, so a stale cue never lingers over a value the user changed.

  • The marker survives detaching and re-attaching the field.

Disabling the Marker

Call setFieldMarkerEnabled(false) for a form that should carry no trace of the AI’s edits:

Source code
Java
controller.setFieldMarkerEnabled(false);

Only the mark is affected: the "AI is working" state shown while a turn runs still applies, and change events still report what the AI wrote. Turning the marker off doesn’t clear marks already shown; fields marked by earlier turns stay marked until the user edits or reverts them.

Localizing the Marker

The marker’s texts default to English. Replace them with setFieldMarkerI18n():

Source code
Java
controller.setFieldMarkerI18n(new FieldMarkerI18n()
        .setMessage("Tekoäly on muuttanut tämän kentän arvoa.")
        .setRevert("Palauta arvo")
        .setBadgeLabel("Tekoälyn tuottama arvo")
        .setBadgeTooltip("Tekoäly on muuttanut kentän arvoa.\nAvaa lisätiedot napsauttamalla"));

Each text has a specific role:

  • setMessage() sets the message shown in the popover that explains the AI fill. The message is also linked to the field as an accessible description and announced to screen readers when the field is marked.

  • setRevert() sets the label of the revert control in the popover.

  • setBadgeLabel() sets the accessible label of the badge button and the popover dialog. The badge itself renders an "AI" icon, so the label is only exposed to assistive technology.

  • setBadgeTooltip() sets the tooltip text shown when the badge is hovered or focused.

The texts are applied to every marker the controller puts on a field, so set them before the first turn to localize them all. Texts left null fall back to the built-in defaults.

Reacting to AI Changes

For applications that need to react to the AI’s edits beyond the marker, addFieldValueChangeListener() registers a listener that fires once per changed field after a successful turn:

Source code
Java
controller.addFieldValueChangeListener(event ->
        auditLog.record(event.getField(), event.getOldValue(),
                event.getNewValue()));

Each event carries the field, its pre-turn value, and its post-turn value, available as getField(), getOldValue(), and getNewValue(). When source tracking is enabled, getFieldSource() on the event also returns the source the model reported for the written value. Events fire in document order, one per changed field. Fields the application has marked with ignoreField() produce no events. The listener is not called when the turn ended in error or when no field’s value changed. Multiple listeners can be registered; each is independent, and the returned Registration removes the listener when its remove() is called.

A field hidden at turn start that is revealed and written into the same turn is reported with its real pre-turn value rather than null, so cascades into conditional fields show up correctly.

The listener runs on the UI thread with the session lock held, so it can update components or call any other Vaadin API directly — no ui.access() wrapper is needed.

Source Tracking

When the values come from an attached document — a resume, an invoice, a scanned contract — a user reviewing the fill often needs more than the values themselves: which part of the document each value came from, and how sure the model was about it. Source tracking records both. For each value the LLM writes, the controller stores the snippets the model reports having read, their locations in the document, and a confidence level.

Source tracking is off by default, since it costs extra output tokens on every fill and brings document snippets to the server. Enable it with setSourceTrackingEnabled() when the application uses the data:

Source code
Java
controller.setSourceTrackingEnabled(true);

Source data is best effort and never blocks a fill: a malformed part, such as an unknown confidence level or a rectangle with no size, is dropped and logged while the value is still written.

Note
One Attachment per Prompt
Send at most one attachment per prompt while source tracking is on. A reported location doesn’t identify which document it points into, so with several attachments there is no way to tell them apart.

Reading Sources

getFieldSource() returns the source recorded for a field’s current value:

Source code
Java
controller.getFieldSource(email).ifPresent(source -> {
    ConfidenceLevel confidence = source.confidence();
    source.extracts().forEach(extract ->
            sourcePanel.add(extract.text()));
});

A ValueSource carries the reported ConfidenceLevel and a list of SourceExtract records. Each extract holds the snippet text the model reports having read and, when reported, a location pointing into the document. The snippet is the model’s own account of what it read; the controller doesn’t verify it against the document.

A source lasts as long as the value it describes. Once the field’s value no longer equals the one the source was reported with — the user edited the field, the application overwrote it, or a revert restored the old value — the source no longer applies and getFieldSource() returns empty. A later fill that writes the field without reporting a source also clears the previous one. Sources are not limited to the turn that produced them: a field filled in an earlier turn keeps its source until the value changes or a later fill overwrites it.

The same data is available per changed field through the change events, via getFieldSource() on the event. A value with nothing to point at, for example one taken from the chat prompt rather than an attachment, carries no source.

Source Locations

An extract’s location is a PageRegion: a 1-based page number and a rectangle. For a single-surface source such as an image, the page is 1. The rectangle’s x(), y(), width(), and height() are fractions of the page dimensions in the 0..1 range, measured from the page’s top-left corner, so they map onto a rendered page regardless of its pixel size:

Source code
Java
source.extracts().forEach(extract -> {
    if (extract.location() instanceof PageRegion region) {
        Rect rect = region.rect();
        documentViewer.highlight(region.page(),
                rect.x() * pageWidth, rect.y() * pageHeight,
                rect.width() * pageWidth, rect.height() * pageHeight);
    }
});

For a rotated PDF page, the coordinates refer to the page as it is displayed.

Confidence Levels

The model rates each sourced value with one of three ConfidenceLevel values:

  • HIGH — the value is written in the document and copied as it is.

  • MEDIUM — the value follows from the document but needed some interpretation: fields combined, units converted, or one candidate chosen over another.

  • LOW — the document is unclear, or the value is a guess.

These meanings are part of the instructions sent to the model. When your domain draws the lines differently, redefine a level’s meaning with describeConfidenceLevel():

Source code
Java
controller.describeConfidenceLevel(ConfidenceLevel.MEDIUM,
        "the value follows from the document but was reformatted "
                + "or combined from several fields");

The description replaces the built-in wording for that level in the LLM-facing instructions. A value reported without a level carries no confidence: it means the model didn’t rate the value, not that the value is uncertain.

Confidence on the Field Marker

While the automatic field marker is enabled, a marked field also shows the reported confidence level as an indicator in the field’s helper area, so a reviewer can see at a glance which values the model itself was unsure about. A value reported without a confidence level shows no indicator, and a later fill that writes the field without a source clears the indicator. While a fill is in progress, the indicator is hidden, since it describes a value that may be about to be replaced.

Reconnecting after Deserialization

FormAIController is not serialized with the orchestrator. After session restore, create a new controller against the same form (and binder, if any), reapply the same describeField(), fieldValueOptions(), and ignoreField() hints, any field-marker and source-tracking configuration, re-register any change listeners, and pass the controller to reconnect():

Source code
Java
FormAIController controller = new FormAIController(form, binder);
controller.describeField(discount, "Discount as a percentage between 0 and 100.")
        .fieldValueOptions(ValueOptions.forField(industry)
                .options(List.of("Software", "Manufacturing", "Healthcare")))
        .ignoreField(internalIdField)
        .setFieldMarkerI18n(markerTexts);

orchestrator.reconnect(provider)
        .withController(controller)
        .apply();

Field ids remain stable across the round-trip because they live on the field components themselves, which Vaadin serializes as part of the UI tree. No separate state object needs saving or restoring; the form fields are the state, and VaadinSession already persists them.

The exception is source data, which lives in the controller and is lost with it. To carry sources across the round-trip, store each field’s ValueSource — the record is serializable — and reattach it to the new controller with restoreFieldSource():

Source code
Java
controller.restoreFieldSource(email, storedEmailSource);

The source binds to the field’s value at the moment of the call and goes stale on the next edit, just like a fresh one. The field values themselves survive with the UI tree, so restoring the sources is the only extra step.

Composing Multiple Forms

FormAIController manages a single container, but a view can host several. Construct one controller per form section and attach each to its own AIOrchestrator. Each orchestrator also needs a dedicated LLMProvider, MessageInput, and MessageList — per the AI Integration overview, none of those instances may be shared across orchestrators.

Updated –